2024-04-17T22:17:41.360 INFO:root:teuthology version: 0.0.1.dev269+g9481b1d 2024-04-17T22:17:41.360 DEBUG:teuthology.run:Teuthology command: teuthology --verbose --description fs/upgrade/upgraded_client/{bluestore-bitmap branch/nautilus centos_8.stream clusters/{1-mds-1-client-micro} conf/{client mds mgr mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} tasks/{0-install 1-mount/mount/fuse 2-clients/fuse-upgrade 3-workload/new_ops/{newops}}} --name teuthology-2024-04-17_21:24:02-fs-squid-distro-default-smithi --owner scheduled_teuthology@teuthology --archive /home/teuthworker/archive/teuthology-2024-04-17_21:24:02-fs-squid-distro-default-smithi/7661154 -- /home/teuthworker/archive/teuthology-2024-04-17_21:24:02-fs-squid-distro-default-smithi/7661154/orig.config.yaml 2024-04-17T22:17:41.406 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-17T22:17:41.505 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-04-17_21:24:02-fs-squid-distro-default-smithi/7661154 branch: squid description: fs/upgrade/upgraded_client/{bluestore-bitmap branch/nautilus centos_8.stream clusters/{1-mds-1-client-micro} conf/{client mds mgr mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} tasks/{0-install 1-mount/mount/fuse 2-clients/fuse-upgrade 3-workload/new_ops/{newops}}} email: ceph-qa@ceph.com first_in_suite: false job_id: '7661154' kernel: kdb: 1 sha1: distro ktype: distro last_in_suite: false machine_type: smithi name: teuthology-2024-04-17_21:24:02-fs-squid-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 10 os_type: centos os_version: 8.stream overrides: admin_socket: branch: squid ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: mon pg warn min per osd: 0 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN sha1: 41c500dd60c93c8d83053b72cffc3dd1a952a336 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 41c500dd60c93c8d83053b72cffc3dd1a952a336 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: squid sha1: 1a6a3e742fd81c6d70be70552499bff767652c32 owner: scheduled_teuthology@teuthology priority: 100 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mon.b - mon.c - mgr.x - mds.a - osd.0 - osd.1 - osd.2 - osd.3 - - client.0 seed: 827 sha1: 41c500dd60c93c8d83053b72cffc3dd1a952a336 sleep_before_teardown: 0 subset: 175/512 suite: fs suite_branch: squid suite_path: /home/teuthworker/src/git.ceph.com_ceph_1a6a3e742fd81c6d70be70552499bff767652c32/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: 1a6a3e742fd81c6d70be70552499bff767652c32 targets: smithi102.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLRJ9qjqTzlTgwiH95uehu4ogBsUfPt8b9MSsgdDu7C2GB9dG25LSJ7HKVK0llq9OfhNJNysQyHz9rcmeUiReok= smithi190.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHYg7CPsZA5CTSHbhqLlD7ulb3ycZN3tg4NvwWBAkpq9tcby13wkhr0sn0M+peU5FOfF7oYZaVTx0DYXHXSUxtk= tasks: - install: branch: nautilus exclude_packages: - cephadm - ceph-mgr-cephadm - ceph-immutable-object-cache - python3-rados - python3-rgw - python3-rbd - python3-cephfs - ceph-volume extra_packages: - python-rados - python-rgw - python-rbd - python-cephfs extra_system_packages: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - print: '**** done installing nautilus' - ceph: conf: global: mon warn on pool no app: false ms bind msgr2: false log-ignorelist: - overall HEALTH_ - \(FS_ - \(MDS_ - \(OSD_ - \(MON_DOWN\) - \(CACHE_POOL_ - \(POOL_ - \(MGR_DOWN\) - \(PG_ - \(SMALLER_PGP_NUM\) - Monitor daemon marked osd - Behind on trimming - Manager daemon - exec: osd.0: - ceph osd set-require-min-compat-client nautilus - print: '**** done ceph' - ceph-fuse: null - ceph-fuse: client.0: mounted: false - print: '**** done unmount client.0' - install.upgrade: client.0: branch: pacific - print: '**** done install.upgrade on client.0' - ceph.healthy: null - ceph-fuse: client.0: null - print: '**** done remount client.0' - cephfs_test_runner: modules: - tasks.cephfs.test_newops teuthology: branch: nautilus fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end - if not is_fuse() then reject() end teuthology_branch: main teuthology_sha1: 9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17 timestamp: 2024-04-17_21:24:02 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-04-17T22:17:41.506 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_1a6a3e742fd81c6d70be70552499bff767652c32/qa; will attempt to use it 2024-04-17T22:17:41.506 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_1a6a3e742fd81c6d70be70552499bff767652c32/qa/tasks 2024-04-17T22:17:41.507 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-17T22:17:41.509 INFO:teuthology.task.internal:Checking packages... 2024-04-17T22:17:41.540 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '41c500dd60c93c8d83053b72cffc3dd1a952a336' 2024-04-17T22:17:41.540 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-17T22:17:41.540 INFO:teuthology.packaging:ref: None 2024-04-17T22:17:41.540 INFO:teuthology.packaging:tag: None 2024-04-17T22:17:41.540 INFO:teuthology.packaging:branch: squid 2024-04-17T22:17:41.540 INFO:teuthology.packaging:sha1: 41c500dd60c93c8d83053b72cffc3dd1a952a336 2024-04-17T22:17:41.540 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=squid 2024-04-17T22:17:41.734 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-2363.g41c500dd 2024-04-17T22:17:41.736 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-17T22:17:41.742 INFO:teuthology.task.internal:no buildpackages task found 2024-04-17T22:17:41.742 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-17T22:17:41.785 INFO:teuthology.task.internal:Saving configuration 2024-04-17T22:17:41.802 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-17T22:17:41.808 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-17T22:17:41.834 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi102.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-04-17_21:24:02-fs-squid-distro-default-smithi/7661154', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-17 22:12:36.194655', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLRJ9qjqTzlTgwiH95uehu4ogBsUfPt8b9MSsgdDu7C2GB9dG25LSJ7HKVK0llq9OfhNJNysQyHz9rcmeUiReok='} 2024-04-17T22:17:41.860 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi190.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-04-17_21:24:02-fs-squid-distro-default-smithi/7661154', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-17 22:12:36.196049', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHYg7CPsZA5CTSHbhqLlD7ulb3ycZN3tg4NvwWBAkpq9tcby13wkhr0sn0M+peU5FOfF7oYZaVTx0DYXHXSUxtk='} 2024-04-17T22:17:41.861 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-17T22:17:41.867 INFO:teuthology.task.internal:roles: ubuntu@smithi102.front.sepia.ceph.com - ['mon.a', 'mon.b', 'mon.c', 'mgr.x', 'mds.a', 'osd.0', 'osd.1', 'osd.2', 'osd.3'] 2024-04-17T22:17:41.867 INFO:teuthology.task.internal:roles: ubuntu@smithi190.front.sepia.ceph.com - ['client.0'] 2024-04-17T22:17:41.867 INFO:teuthology.run_tasks:Running task console_log... 2024-04-17T22:17:41.976 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fa9792cb3a0>, signals=[15]) 2024-04-17T22:17:41.976 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-17T22:17:41.982 INFO:teuthology.task.internal:Opening connections... 2024-04-17T22:17:41.982 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi102.front.sepia.ceph.com 2024-04-17T22:17:41.984 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi102.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-17T22:17:42.062 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi190.front.sepia.ceph.com 2024-04-17T22:17:42.064 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi190.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-17T22:17:42.143 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-17T22:17:42.150 DEBUG:teuthology.orchestra.run.smithi102:> uname -m 2024-04-17T22:17:42.172 INFO:teuthology.orchestra.run.smithi102.stdout:x86_64 2024-04-17T22:17:42.172 DEBUG:teuthology.orchestra.run.smithi102:> cat /etc/os-release 2024-04-17T22:17:42.234 INFO:teuthology.orchestra.run.smithi102.stdout:NAME="CentOS Stream" 2024-04-17T22:17:42.234 INFO:teuthology.orchestra.run.smithi102.stdout:VERSION="8" 2024-04-17T22:17:42.234 INFO:teuthology.orchestra.run.smithi102.stdout:ID="centos" 2024-04-17T22:17:42.235 INFO:teuthology.orchestra.run.smithi102.stdout:ID_LIKE="rhel fedora" 2024-04-17T22:17:42.235 INFO:teuthology.orchestra.run.smithi102.stdout:VERSION_ID="8" 2024-04-17T22:17:42.235 INFO:teuthology.orchestra.run.smithi102.stdout:PLATFORM_ID="platform:el8" 2024-04-17T22:17:42.235 INFO:teuthology.orchestra.run.smithi102.stdout:PRETTY_NAME="CentOS Stream 8" 2024-04-17T22:17:42.235 INFO:teuthology.orchestra.run.smithi102.stdout:ANSI_COLOR="0;31" 2024-04-17T22:17:42.235 INFO:teuthology.orchestra.run.smithi102.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2024-04-17T22:17:42.235 INFO:teuthology.orchestra.run.smithi102.stdout:HOME_URL="https://centos.org/" 2024-04-17T22:17:42.235 INFO:teuthology.orchestra.run.smithi102.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-17T22:17:42.235 INFO:teuthology.orchestra.run.smithi102.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2024-04-17T22:17:42.235 INFO:teuthology.orchestra.run.smithi102.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-17T22:17:42.236 INFO:teuthology.lock.ops:Updating smithi102.front.sepia.ceph.com on lock server 2024-04-17T22:17:42.294 DEBUG:teuthology.orchestra.run.smithi190:> uname -m 2024-04-17T22:17:42.316 INFO:teuthology.orchestra.run.smithi190.stdout:x86_64 2024-04-17T22:17:42.317 DEBUG:teuthology.orchestra.run.smithi190:> cat /etc/os-release 2024-04-17T22:17:42.378 INFO:teuthology.orchestra.run.smithi190.stdout:NAME="CentOS Stream" 2024-04-17T22:17:42.378 INFO:teuthology.orchestra.run.smithi190.stdout:VERSION="8" 2024-04-17T22:17:42.378 INFO:teuthology.orchestra.run.smithi190.stdout:ID="centos" 2024-04-17T22:17:42.378 INFO:teuthology.orchestra.run.smithi190.stdout:ID_LIKE="rhel fedora" 2024-04-17T22:17:42.378 INFO:teuthology.orchestra.run.smithi190.stdout:VERSION_ID="8" 2024-04-17T22:17:42.378 INFO:teuthology.orchestra.run.smithi190.stdout:PLATFORM_ID="platform:el8" 2024-04-17T22:17:42.378 INFO:teuthology.orchestra.run.smithi190.stdout:PRETTY_NAME="CentOS Stream 8" 2024-04-17T22:17:42.379 INFO:teuthology.orchestra.run.smithi190.stdout:ANSI_COLOR="0;31" 2024-04-17T22:17:42.379 INFO:teuthology.orchestra.run.smithi190.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2024-04-17T22:17:42.379 INFO:teuthology.orchestra.run.smithi190.stdout:HOME_URL="https://centos.org/" 2024-04-17T22:17:42.379 INFO:teuthology.orchestra.run.smithi190.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-17T22:17:42.379 INFO:teuthology.orchestra.run.smithi190.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2024-04-17T22:17:42.379 INFO:teuthology.orchestra.run.smithi190.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-17T22:17:42.379 INFO:teuthology.lock.ops:Updating smithi190.front.sepia.ceph.com on lock server 2024-04-17T22:17:42.402 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-17T22:17:42.410 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-17T22:17:42.433 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-17T22:17:42.434 DEBUG:teuthology.orchestra.run.smithi102:> test '!' -e /home/ubuntu/cephtest 2024-04-17T22:17:42.436 DEBUG:teuthology.orchestra.run.smithi190:> test '!' -e /home/ubuntu/cephtest 2024-04-17T22:17:42.456 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-17T22:17:42.462 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-17T22:17:42.462 DEBUG:teuthology.orchestra.run.smithi102:> test -z $(ls -A /var/lib/ceph) 2024-04-17T22:17:42.499 DEBUG:teuthology.orchestra.run.smithi190:> test -z $(ls -A /var/lib/ceph) 2024-04-17T22:17:42.537 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-17T22:17:42.595 INFO:teuthology.run_tasks:Running task kernel... 2024-04-17T22:17:42.610 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-04-17T22:17:42.611 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'client.0': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-04-17T22:17:42.611 DEBUG:teuthology.orchestra.run.smithi102:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-17T22:17:42.611 DEBUG:teuthology.orchestra.run.smithi190:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-17T22:17:42.631 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-17T22:17:42.632 DEBUG:teuthology.orchestra.run.smithi102:> uname -r 2024-04-17T22:17:42.633 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-17T22:17:42.633 DEBUG:teuthology.orchestra.run.smithi190:> uname -r 2024-04-17T22:17:42.694 INFO:teuthology.orchestra.run.smithi190.stdout:4.18.0-489.el8.x86_64 2024-04-17T22:17:42.694 INFO:teuthology.task.kernel:Running kernel on smithi190: 4.18.0-489.el8.x86_64 2024-04-17T22:17:42.695 DEBUG:teuthology.orchestra.run.smithi190:> sudo yum install -y kernel 2024-04-17T22:17:42.696 INFO:teuthology.orchestra.run.smithi102.stdout:4.18.0-489.el8.x86_64 2024-04-17T22:17:42.696 INFO:teuthology.task.kernel:Running kernel on smithi102: 4.18.0-489.el8.x86_64 2024-04-17T22:17:42.696 DEBUG:teuthology.orchestra.run.smithi102:> sudo yum install -y kernel 2024-04-17T22:17:46.667 INFO:teuthology.orchestra.run.smithi102.stdout:CentOS Stream 8 - AppStream 22 MB/s | 28 MB 00:01 2024-04-17T22:17:46.783 INFO:teuthology.orchestra.run.smithi190.stdout:CentOS Stream 8 - AppStream 20 MB/s | 28 MB 00:01 2024-04-17T22:17:52.500 INFO:teuthology.orchestra.run.smithi190.stdout:CentOS Stream 8 - BaseOS 15 MB/s | 10 MB 00:00 2024-04-17T22:17:52.593 INFO:teuthology.orchestra.run.smithi102.stdout:CentOS Stream 8 - BaseOS 12 MB/s | 10 MB 00:00 2024-04-17T22:17:54.611 INFO:teuthology.orchestra.run.smithi102.stdout:CentOS Stream 8 - Extras 43 kB/s | 18 kB 00:00 2024-04-17T22:17:54.642 INFO:teuthology.orchestra.run.smithi190.stdout:CentOS Stream 8 - Extras 33 kB/s | 18 kB 00:00 2024-04-17T22:17:55.343 INFO:teuthology.orchestra.run.smithi190.stdout:CentOS Stream 8 - Extras common packages 13 kB/s | 7.7 kB 00:00 2024-04-17T22:17:55.377 INFO:teuthology.orchestra.run.smithi102.stdout:CentOS Stream 8 - Extras common packages 12 kB/s | 7.7 kB 00:00 2024-04-17T22:17:55.611 INFO:teuthology.orchestra.run.smithi190.stdout:Copr repo for python3-asyncssh owned by ceph 20 kB/s | 3.5 kB 00:00 2024-04-17T22:17:55.629 INFO:teuthology.orchestra.run.smithi102.stdout:Copr repo for python3-asyncssh owned by ceph 22 kB/s | 3.5 kB 00:00 2024-04-17T22:17:56.436 INFO:teuthology.orchestra.run.smithi190.stdout:Extra Packages for Enterprise Linux 22 MB/s | 16 MB 00:00 2024-04-17T22:17:56.882 INFO:teuthology.orchestra.run.smithi102.stdout:Extra Packages for Enterprise Linux 14 MB/s | 16 MB 00:01 2024-04-17T22:18:00.262 INFO:teuthology.orchestra.run.smithi190.stdout:lab-extras 341 kB/s | 24 kB 00:00 2024-04-17T22:18:00.792 INFO:teuthology.orchestra.run.smithi102.stdout:lab-extras 323 kB/s | 24 kB 00:00 2024-04-17T22:18:03.325 INFO:teuthology.orchestra.run.smithi190.stdout:Package kernel-4.18.0-348.el8.x86_64 is already installed. 2024-04-17T22:18:03.325 INFO:teuthology.orchestra.run.smithi190.stdout:Package kernel-4.18.0-489.el8.x86_64 is already installed. 2024-04-17T22:18:03.429 INFO:teuthology.orchestra.run.smithi190.stdout:Dependencies resolved. 2024-04-17T22:18:03.429 INFO:teuthology.orchestra.run.smithi190.stdout:================================================================================ 2024-04-17T22:18:03.430 INFO:teuthology.orchestra.run.smithi190.stdout: Package Architecture Version Repository Size 2024-04-17T22:18:03.430 INFO:teuthology.orchestra.run.smithi190.stdout:================================================================================ 2024-04-17T22:18:03.430 INFO:teuthology.orchestra.run.smithi190.stdout:Installing: 2024-04-17T22:18:03.430 INFO:teuthology.orchestra.run.smithi190.stdout: kernel x86_64 4.18.0-552.el8 baseos 10 M 2024-04-17T22:18:03.430 INFO:teuthology.orchestra.run.smithi190.stdout:Installing dependencies: 2024-04-17T22:18:03.430 INFO:teuthology.orchestra.run.smithi190.stdout: kernel-core x86_64 4.18.0-552.el8 baseos 44 M 2024-04-17T22:18:03.430 INFO:teuthology.orchestra.run.smithi190.stdout: kernel-modules x86_64 4.18.0-552.el8 baseos 36 M 2024-04-17T22:18:03.430 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-04-17T22:18:03.430 INFO:teuthology.orchestra.run.smithi190.stdout:Transaction Summary 2024-04-17T22:18:03.430 INFO:teuthology.orchestra.run.smithi190.stdout:================================================================================ 2024-04-17T22:18:03.430 INFO:teuthology.orchestra.run.smithi190.stdout:Install 3 Packages 2024-04-17T22:18:03.431 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-04-17T22:18:03.431 INFO:teuthology.orchestra.run.smithi190.stdout:Total download size: 90 M 2024-04-17T22:18:03.431 INFO:teuthology.orchestra.run.smithi190.stdout:Installed size: 96 M 2024-04-17T22:18:03.431 INFO:teuthology.orchestra.run.smithi190.stdout:Downloading Packages: 2024-04-17T22:18:03.835 INFO:teuthology.orchestra.run.smithi102.stdout:Package kernel-4.18.0-348.el8.x86_64 is already installed. 2024-04-17T22:18:03.835 INFO:teuthology.orchestra.run.smithi102.stdout:Package kernel-4.18.0-489.el8.x86_64 is already installed. 2024-04-17T22:18:03.971 INFO:teuthology.orchestra.run.smithi102.stdout:Dependencies resolved. 2024-04-17T22:18:03.971 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-04-17T22:18:03.971 INFO:teuthology.orchestra.run.smithi102.stdout: Package Architecture Version Repository Size 2024-04-17T22:18:03.971 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-04-17T22:18:03.971 INFO:teuthology.orchestra.run.smithi102.stdout:Installing: 2024-04-17T22:18:03.972 INFO:teuthology.orchestra.run.smithi102.stdout: kernel x86_64 4.18.0-552.el8 baseos 10 M 2024-04-17T22:18:03.972 INFO:teuthology.orchestra.run.smithi102.stdout:Installing dependencies: 2024-04-17T22:18:03.972 INFO:teuthology.orchestra.run.smithi102.stdout: kernel-core x86_64 4.18.0-552.el8 baseos 44 M 2024-04-17T22:18:03.972 INFO:teuthology.orchestra.run.smithi102.stdout: kernel-modules x86_64 4.18.0-552.el8 baseos 36 M 2024-04-17T22:18:03.972 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-17T22:18:03.972 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction Summary 2024-04-17T22:18:03.972 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-04-17T22:18:03.972 INFO:teuthology.orchestra.run.smithi102.stdout:Install 3 Packages 2024-04-17T22:18:03.972 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-17T22:18:03.972 INFO:teuthology.orchestra.run.smithi102.stdout:Total download size: 90 M 2024-04-17T22:18:03.972 INFO:teuthology.orchestra.run.smithi102.stdout:Installed size: 96 M 2024-04-17T22:18:03.972 INFO:teuthology.orchestra.run.smithi102.stdout:Downloading Packages: 2024-04-17T22:18:04.521 INFO:teuthology.orchestra.run.smithi190.stdout:(1/3): kernel-4.18.0-552.el8.x86_64.rpm 15 MB/s | 10 MB 00:00 2024-04-17T22:18:05.206 INFO:teuthology.orchestra.run.smithi102.stdout:(1/3): kernel-4.18.0-552.el8.x86_64.rpm 10 MB/s | 10 MB 00:01 2024-04-17T22:18:05.938 INFO:teuthology.orchestra.run.smithi190.stdout:(2/3): kernel-modules-4.18.0-552.el8.x86_64.rpm 17 MB/s | 36 MB 00:02 2024-04-17T22:18:06.315 INFO:teuthology.orchestra.run.smithi102.stdout:(2/3): kernel-core-4.18.0-552.el8.x86_64.rpm 20 MB/s | 44 MB 00:02 2024-04-17T22:18:06.421 INFO:teuthology.orchestra.run.smithi190.stdout:(3/3): kernel-core-4.18.0-552.el8.x86_64.rpm 17 MB/s | 44 MB 00:02 2024-04-17T22:18:06.422 INFO:teuthology.orchestra.run.smithi190.stdout:-------------------------------------------------------------------------------- 2024-04-17T22:18:06.422 INFO:teuthology.orchestra.run.smithi190.stdout:Total 30 MB/s | 90 MB 00:02 2024-04-17T22:18:06.815 INFO:teuthology.orchestra.run.smithi102.stdout:(3/3): kernel-modules-4.18.0-552.el8.x86_64.rpm 14 MB/s | 36 MB 00:02 2024-04-17T22:18:06.815 INFO:teuthology.orchestra.run.smithi102.stdout:-------------------------------------------------------------------------------- 2024-04-17T22:18:06.815 INFO:teuthology.orchestra.run.smithi102.stdout:Total 32 MB/s | 90 MB 00:02 2024-04-17T22:18:07.103 INFO:teuthology.orchestra.run.smithi190.stdout:Running transaction check 2024-04-17T22:18:07.498 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction check 2024-04-17T22:18:07.542 INFO:teuthology.orchestra.run.smithi190.stdout:Transaction check succeeded. 2024-04-17T22:18:07.542 INFO:teuthology.orchestra.run.smithi190.stdout:Running transaction test 2024-04-17T22:18:07.916 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction check succeeded. 2024-04-17T22:18:07.916 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction test 2024-04-17T22:18:08.790 INFO:teuthology.orchestra.run.smithi190.stdout:Transaction test succeeded. 2024-04-17T22:18:08.814 INFO:teuthology.orchestra.run.smithi190.stdout:Running transaction 2024-04-17T22:18:09.160 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction test succeeded. 2024-04-17T22:18:09.181 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction 2024-04-17T22:18:11.621 INFO:teuthology.orchestra.run.smithi190.stdout: Preparing : 1/1 2024-04-17T22:18:11.946 INFO:teuthology.orchestra.run.smithi102.stdout: Preparing : 1/1 2024-04-17T22:18:12.209 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : kernel-core-4.18.0-552.el8.x86_64 1/3 2024-04-17T22:18:12.439 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : kernel-core-4.18.0-552.el8.x86_64 1/3 2024-04-17T22:18:15.791 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: kernel-core-4.18.0-552.el8.x86_64 1/3 2024-04-17T22:18:15.976 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: kernel-core-4.18.0-552.el8.x86_64 1/3 2024-04-17T22:18:16.159 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : kernel-modules-4.18.0-552.el8.x86_64 2/3 2024-04-17T22:18:16.397 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : kernel-modules-4.18.0-552.el8.x86_64 2/3 2024-04-17T22:18:19.596 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: kernel-modules-4.18.0-552.el8.x86_64 2/3 2024-04-17T22:18:19.834 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: kernel-modules-4.18.0-552.el8.x86_64 2/3 2024-04-17T22:18:19.891 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : kernel-4.18.0-552.el8.x86_64 3/3 2024-04-17T22:18:20.138 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : kernel-4.18.0-552.el8.x86_64 3/3 2024-04-17T22:18:41.870 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: kernel-core-4.18.0-552.el8.x86_64 3/3 2024-04-17T22:18:41.893 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: kernel-modules-4.18.0-552.el8.x86_64 3/3 2024-04-17T22:18:41.965 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: kernel-core-4.18.0-552.el8.x86_64 3/3 2024-04-17T22:18:41.988 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: kernel-modules-4.18.0-552.el8.x86_64 3/3 2024-04-17T22:18:42.246 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: kernel-4.18.0-552.el8.x86_64 3/3 2024-04-17T22:18:42.246 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : kernel-4.18.0-552.el8.x86_64 1/3 2024-04-17T22:18:42.246 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : kernel-core-4.18.0-552.el8.x86_64 2/3 2024-04-17T22:18:42.333 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: kernel-4.18.0-552.el8.x86_64 3/3 2024-04-17T22:18:42.333 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : kernel-4.18.0-552.el8.x86_64 1/3 2024-04-17T22:18:42.333 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : kernel-core-4.18.0-552.el8.x86_64 2/3 2024-04-17T22:18:42.714 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : kernel-modules-4.18.0-552.el8.x86_64 3/3 2024-04-17T22:18:42.715 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-04-17T22:18:42.715 INFO:teuthology.orchestra.run.smithi190.stdout:Installed: 2024-04-17T22:18:42.715 INFO:teuthology.orchestra.run.smithi190.stdout: kernel-4.18.0-552.el8.x86_64 kernel-core-4.18.0-552.el8.x86_64 2024-04-17T22:18:42.715 INFO:teuthology.orchestra.run.smithi190.stdout: kernel-modules-4.18.0-552.el8.x86_64 2024-04-17T22:18:42.715 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-04-17T22:18:42.715 INFO:teuthology.orchestra.run.smithi190.stdout:Complete! 2024-04-17T22:18:42.829 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : kernel-modules-4.18.0-552.el8.x86_64 3/3 2024-04-17T22:18:42.829 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-17T22:18:42.829 INFO:teuthology.orchestra.run.smithi102.stdout:Installed: 2024-04-17T22:18:42.829 INFO:teuthology.orchestra.run.smithi102.stdout: kernel-4.18.0-552.el8.x86_64 kernel-core-4.18.0-552.el8.x86_64 2024-04-17T22:18:42.830 INFO:teuthology.orchestra.run.smithi102.stdout: kernel-modules-4.18.0-552.el8.x86_64 2024-04-17T22:18:42.830 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-17T22:18:42.830 INFO:teuthology.orchestra.run.smithi102.stdout:Complete! 2024-04-17T22:18:42.869 DEBUG:teuthology.orchestra.run.smithi190:> rpm -q kernel | sort -rV | head -n 1 2024-04-17T22:18:42.969 DEBUG:teuthology.orchestra.run.smithi102:> rpm -q kernel | sort -rV | head -n 1 2024-04-17T22:18:43.087 INFO:teuthology.orchestra.run.smithi190.stdout:kernel-4.18.0-552.el8.x86_64 2024-04-17T22:18:43.088 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-552.el8.x86_64 2024-04-17T22:18:43.088 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 4.18.0-489.el8.x86_64 Expected: 4.18.0-552.el8.x86_64 2024-04-17T22:18:43.088 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-04-17T22:18:43.088 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-04-17T22:18:43.088 INFO:teuthology.task.kernel:Installing distro kernel on client.0... 2024-04-17T22:18:43.088 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi190.front.sepia.ceph.com, path=None, version=distro) 2024-04-17T22:18:43.089 DEBUG:teuthology.orchestra.run.smithi190:> rpm -q kernel | sort -rV | head -n 1 2024-04-17T22:18:43.227 INFO:teuthology.orchestra.run.smithi102.stdout:kernel-4.18.0-552.el8.x86_64 2024-04-17T22:18:43.227 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-552.el8.x86_64 2024-04-17T22:18:43.227 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 4.18.0-489.el8.x86_64 Expected: 4.18.0-552.el8.x86_64 2024-04-17T22:18:43.227 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-04-17T22:18:43.228 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-04-17T22:18:43.228 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-04-17T22:18:43.228 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi102.front.sepia.ceph.com, path=None, version=distro) 2024-04-17T22:18:43.228 DEBUG:teuthology.orchestra.run.smithi102:> rpm -q kernel | sort -rV | head -n 1 2024-04-17T22:18:43.300 INFO:teuthology.orchestra.run.smithi190.stdout:kernel-4.18.0-552.el8.x86_64 2024-04-17T22:18:43.300 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-552.el8.x86_64 2024-04-17T22:18:43.300 DEBUG:teuthology.orchestra.run.smithi190:> sudo rpm -qi grub2-tools 2024-04-17T22:18:43.334 INFO:teuthology.orchestra.run.smithi190.stdout:Name : grub2-tools 2024-04-17T22:18:43.334 INFO:teuthology.orchestra.run.smithi190.stdout:Epoch : 1 2024-04-17T22:18:43.334 INFO:teuthology.orchestra.run.smithi190.stdout:Version : 2.02 2024-04-17T22:18:43.334 INFO:teuthology.orchestra.run.smithi190.stdout:Release : 129.el8 2024-04-17T22:18:43.334 INFO:teuthology.orchestra.run.smithi190.stdout:Architecture: x86_64 2024-04-17T22:18:43.334 INFO:teuthology.orchestra.run.smithi190.stdout:Install Date: Thu 04 May 2023 10:17:48 PM UTC 2024-04-17T22:18:43.334 INFO:teuthology.orchestra.run.smithi190.stdout:Group : System Environment/Base 2024-04-17T22:18:43.334 INFO:teuthology.orchestra.run.smithi190.stdout:Size : 9265586 2024-04-17T22:18:43.335 INFO:teuthology.orchestra.run.smithi190.stdout:License : GPLv3+ 2024-04-17T22:18:43.335 INFO:teuthology.orchestra.run.smithi190.stdout:Signature : RSA/SHA256, Tue 19 Jul 2022 02:52:06 PM UTC, Key ID 05b555b38483c65d 2024-04-17T22:18:43.335 INFO:teuthology.orchestra.run.smithi190.stdout:Source RPM : grub2-2.02-129.el8.src.rpm 2024-04-17T22:18:43.335 INFO:teuthology.orchestra.run.smithi190.stdout:Build Date : Tue 19 Jul 2022 02:30:39 PM UTC 2024-04-17T22:18:43.335 INFO:teuthology.orchestra.run.smithi190.stdout:Build Host : kbuilder.bsys.centos.org 2024-04-17T22:18:43.335 INFO:teuthology.orchestra.run.smithi190.stdout:Relocations : (not relocatable) 2024-04-17T22:18:43.335 INFO:teuthology.orchestra.run.smithi190.stdout:Packager : CentOS BuildSystem 2024-04-17T22:18:43.335 INFO:teuthology.orchestra.run.smithi190.stdout:Vendor : CentOS 2024-04-17T22:18:43.335 INFO:teuthology.orchestra.run.smithi190.stdout:URL : http://www.gnu.org/software/grub/ 2024-04-17T22:18:43.336 INFO:teuthology.orchestra.run.smithi190.stdout:Summary : Support tools for GRUB. 2024-04-17T22:18:43.336 INFO:teuthology.orchestra.run.smithi190.stdout:Description : 2024-04-17T22:18:43.336 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-04-17T22:18:43.336 INFO:teuthology.orchestra.run.smithi190.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-04-17T22:18:43.336 INFO:teuthology.orchestra.run.smithi190.stdout:customizable bootloader with modular architecture. It supports a rich 2024-04-17T22:18:43.336 INFO:teuthology.orchestra.run.smithi190.stdout:variety of kernel formats, file systems, computer architectures and 2024-04-17T22:18:43.336 INFO:teuthology.orchestra.run.smithi190.stdout:hardware devices. 2024-04-17T22:18:43.336 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-04-17T22:18:43.336 INFO:teuthology.orchestra.run.smithi190.stdout:This subpackage provides tools for support of all platforms. 2024-04-17T22:18:43.337 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-04-17T22:18:43.337 INFO:teuthology.task.kernel:Updating grub on smithi190 to boot 4.18.0-552.el8.x86_64 2024-04-17T22:18:43.337 DEBUG:teuthology.orchestra.run.smithi190:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-04-17T22:18:43.438 INFO:teuthology.orchestra.run.smithi102.stdout:kernel-4.18.0-552.el8.x86_64 2024-04-17T22:18:43.438 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-552.el8.x86_64 2024-04-17T22:18:43.439 DEBUG:teuthology.orchestra.run.smithi102:> sudo rpm -qi grub2-tools 2024-04-17T22:18:43.472 INFO:teuthology.orchestra.run.smithi102.stdout:Name : grub2-tools 2024-04-17T22:18:43.472 INFO:teuthology.orchestra.run.smithi102.stdout:Epoch : 1 2024-04-17T22:18:43.472 INFO:teuthology.orchestra.run.smithi102.stdout:Version : 2.02 2024-04-17T22:18:43.472 INFO:teuthology.orchestra.run.smithi102.stdout:Release : 129.el8 2024-04-17T22:18:43.472 INFO:teuthology.orchestra.run.smithi102.stdout:Architecture: x86_64 2024-04-17T22:18:43.473 INFO:teuthology.orchestra.run.smithi102.stdout:Install Date: Thu 04 May 2023 10:17:48 PM UTC 2024-04-17T22:18:43.473 INFO:teuthology.orchestra.run.smithi102.stdout:Group : System Environment/Base 2024-04-17T22:18:43.473 INFO:teuthology.orchestra.run.smithi102.stdout:Size : 9265586 2024-04-17T22:18:43.473 INFO:teuthology.orchestra.run.smithi102.stdout:License : GPLv3+ 2024-04-17T22:18:43.473 INFO:teuthology.orchestra.run.smithi102.stdout:Signature : RSA/SHA256, Tue 19 Jul 2022 02:52:06 PM UTC, Key ID 05b555b38483c65d 2024-04-17T22:18:43.473 INFO:teuthology.orchestra.run.smithi102.stdout:Source RPM : grub2-2.02-129.el8.src.rpm 2024-04-17T22:18:43.473 INFO:teuthology.orchestra.run.smithi102.stdout:Build Date : Tue 19 Jul 2022 02:30:39 PM UTC 2024-04-17T22:18:43.473 INFO:teuthology.orchestra.run.smithi102.stdout:Build Host : kbuilder.bsys.centos.org 2024-04-17T22:18:43.473 INFO:teuthology.orchestra.run.smithi102.stdout:Relocations : (not relocatable) 2024-04-17T22:18:43.473 INFO:teuthology.orchestra.run.smithi102.stdout:Packager : CentOS BuildSystem 2024-04-17T22:18:43.473 INFO:teuthology.orchestra.run.smithi102.stdout:Vendor : CentOS 2024-04-17T22:18:43.473 INFO:teuthology.orchestra.run.smithi102.stdout:URL : http://www.gnu.org/software/grub/ 2024-04-17T22:18:43.473 INFO:teuthology.orchestra.run.smithi102.stdout:Summary : Support tools for GRUB. 2024-04-17T22:18:43.473 INFO:teuthology.orchestra.run.smithi102.stdout:Description : 2024-04-17T22:18:43.473 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-17T22:18:43.474 INFO:teuthology.orchestra.run.smithi102.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-04-17T22:18:43.474 INFO:teuthology.orchestra.run.smithi102.stdout:customizable bootloader with modular architecture. It supports a rich 2024-04-17T22:18:43.474 INFO:teuthology.orchestra.run.smithi102.stdout:variety of kernel formats, file systems, computer architectures and 2024-04-17T22:18:43.474 INFO:teuthology.orchestra.run.smithi102.stdout:hardware devices. 2024-04-17T22:18:43.474 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-17T22:18:43.474 INFO:teuthology.orchestra.run.smithi102.stdout:This subpackage provides tools for support of all platforms. 2024-04-17T22:18:43.475 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-04-17T22:18:43.475 INFO:teuthology.task.kernel:Updating grub on smithi102 to boot 4.18.0-552.el8.x86_64 2024-04-17T22:18:43.475 DEBUG:teuthology.orchestra.run.smithi102:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-04-17T22:18:44.831 INFO:teuthology.orchestra.run.smithi102.stderr:Generating grub configuration file ... 2024-04-17T22:18:45.031 INFO:teuthology.orchestra.run.smithi190.stderr:Generating grub configuration file ... 2024-04-17T22:18:46.919 INFO:teuthology.orchestra.run.smithi190.stderr:done 2024-04-17T22:18:46.920 DEBUG:teuthology.orchestra.run.smithi190:> mktemp 2024-04-17T22:18:46.954 INFO:teuthology.orchestra.run.smithi190.stdout:/tmp/tmp.jAH5nkPdTZ 2024-04-17T22:18:46.954 DEBUG:teuthology.orchestra.run.smithi190:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.jAH5nkPdTZ 2024-04-17T22:18:47.028 DEBUG:teuthology.orchestra.run.smithi190:> sudo chmod 0666 /tmp/tmp.jAH5nkPdTZ 2024-04-17T22:18:47.122 INFO:teuthology.orchestra.run.smithi102.stderr:done 2024-04-17T22:18:47.124 DEBUG:teuthology.orchestra.run.smithi102:> mktemp 2024-04-17T22:18:47.157 INFO:teuthology.orchestra.run.smithi102.stdout:/tmp/tmp.xNLz0CnlD7 2024-04-17T22:18:47.158 DEBUG:teuthology.orchestra.run.smithi102:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.xNLz0CnlD7 2024-04-17T22:18:47.171 DEBUG:teuthology.orchestra.remote:smithi190:/tmp/tmp.jAH5nkPdTZ is 6KB 2024-04-17T22:18:47.229 DEBUG:teuthology.orchestra.run.smithi190:> rm -fr /tmp/tmp.jAH5nkPdTZ 2024-04-17T22:18:47.231 DEBUG:teuthology.orchestra.run.smithi102:> sudo chmod 0666 /tmp/tmp.xNLz0CnlD7 2024-04-17T22:18:47.247 DEBUG:teuthology.orchestra.run.smithi190:> sudo /bin/ls /boot/loader/entries || true 2024-04-17T22:18:47.322 INFO:teuthology.orchestra.run.smithi190.stdout:a1b29d6c289747bdb22930598d39874a-0-rescue.conf 2024-04-17T22:18:47.322 INFO:teuthology.orchestra.run.smithi190.stdout:a1b29d6c289747bdb22930598d39874a-4.18.0-348.el8.x86_64.conf 2024-04-17T22:18:47.322 INFO:teuthology.orchestra.run.smithi190.stdout:a1b29d6c289747bdb22930598d39874a-4.18.0-489.el8.x86_64.conf 2024-04-17T22:18:47.322 INFO:teuthology.orchestra.run.smithi190.stdout:a1b29d6c289747bdb22930598d39874a-4.18.0-552.el8.x86_64.conf 2024-04-17T22:18:47.323 DEBUG:teuthology.orchestra.run.smithi190:> sudo grub2-set-default a1b29d6c289747bdb22930598d39874a-4.18.0-552.el8.x86_64 2024-04-17T22:18:47.378 DEBUG:teuthology.orchestra.remote:smithi102:/tmp/tmp.xNLz0CnlD7 is 6KB 2024-04-17T22:18:47.428 DEBUG:teuthology.orchestra.run.smithi102:> rm -fr /tmp/tmp.xNLz0CnlD7 2024-04-17T22:18:47.447 DEBUG:teuthology.orchestra.run.smithi102:> sudo /bin/ls /boot/loader/entries || true 2024-04-17T22:18:47.482 DEBUG:teuthology.orchestra.run.smithi190:> sudo shutdown -r now 2024-04-17T22:18:47.520 INFO:teuthology.orchestra.run.smithi102.stdout:a1b29d6c289747bdb22930598d39874a-0-rescue.conf 2024-04-17T22:18:47.520 INFO:teuthology.orchestra.run.smithi102.stdout:a1b29d6c289747bdb22930598d39874a-4.18.0-348.el8.x86_64.conf 2024-04-17T22:18:47.520 INFO:teuthology.orchestra.run.smithi102.stdout:a1b29d6c289747bdb22930598d39874a-4.18.0-489.el8.x86_64.conf 2024-04-17T22:18:47.520 INFO:teuthology.orchestra.run.smithi102.stdout:a1b29d6c289747bdb22930598d39874a-4.18.0-552.el8.x86_64.conf 2024-04-17T22:18:47.522 DEBUG:teuthology.orchestra.run.smithi102:> sudo grub2-set-default a1b29d6c289747bdb22930598d39874a-4.18.0-552.el8.x86_64 2024-04-17T22:18:47.719 DEBUG:teuthology.orchestra.run.smithi102:> sudo shutdown -r now 2024-04-17T22:19:17.514 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-04-17T22:19:17.515 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi190.front.sepia.ceph.com' 2024-04-17T22:19:17.516 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi190.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-17T22:19:17.722 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-04-17T22:19:17.722 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi102.front.sepia.ceph.com' 2024-04-17T22:19:17.723 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi102.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-17T22:19:35.839 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.190 2024-04-17T22:19:36.098 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.102 2024-04-17T22:19:44.846 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi190.front.sepia.ceph.com' 2024-04-17T22:19:44.847 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi190.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-17T22:19:45.099 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi102.front.sepia.ceph.com' 2024-04-17T22:19:45.099 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi102.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-17T22:19:47.902 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.190 2024-04-17T22:19:48.162 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.102 2024-04-17T22:19:59.914 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi190.front.sepia.ceph.com' 2024-04-17T22:19:59.915 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi190.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-17T22:20:00.164 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi102.front.sepia.ceph.com' 2024-04-17T22:20:00.164 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi102.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-17T22:20:18.591 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.102 2024-04-17T22:20:32.174 DEBUG:teuthology.orchestra.run.smithi190:> true 2024-04-17T22:20:32.800 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi190.front.sepia.ceph.com' 2024-04-17T22:20:32.801 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "4.18.0-552.el8.x86_64"... 2024-04-17T22:20:32.801 DEBUG:teuthology.orchestra.run.smithi190:> uname -r 2024-04-17T22:20:32.858 INFO:teuthology.orchestra.run.smithi190.stdout:4.18.0-552.el8.x86_64 2024-04-17T22:20:32.858 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-552.el8.x86_64 vs 4.18.0-552.el8.x86_64 2024-04-17T22:20:32.858 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-17T22:20:32.858 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-04-17T22:20:33.592 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi102.front.sepia.ceph.com' 2024-04-17T22:20:33.593 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi102.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-17T22:20:33.859 INFO:teuthology.task.kernel:Enabling kdb on client.0... 2024-04-17T22:20:33.885 DEBUG:teuthology.orchestra.run.smithi190:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-17T22:20:33.886 DEBUG:teuthology.orchestra.run.smithi102:> true 2024-04-17T22:20:33.967 INFO:teuthology.orchestra.run.smithi190.stdout:ttyS1 2024-04-17T22:20:33.980 DEBUG:teuthology.parallel:result is None 2024-04-17T22:20:34.122 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi102.front.sepia.ceph.com' 2024-04-17T22:20:34.122 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "4.18.0-552.el8.x86_64"... 2024-04-17T22:20:34.122 DEBUG:teuthology.orchestra.run.smithi102:> uname -r 2024-04-17T22:20:34.183 INFO:teuthology.orchestra.run.smithi102.stdout:4.18.0-552.el8.x86_64 2024-04-17T22:20:34.183 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-552.el8.x86_64 vs 4.18.0-552.el8.x86_64 2024-04-17T22:20:34.183 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-17T22:20:34.183 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-04-17T22:20:35.184 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-04-17T22:20:35.185 DEBUG:teuthology.orchestra.run.smithi102:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-17T22:20:35.926 INFO:teuthology.orchestra.run.smithi102.stdout:ttyS1 2024-04-17T22:20:35.972 DEBUG:teuthology.parallel:result is None 2024-04-17T22:20:35.972 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-17T22:20:35.979 INFO:teuthology.task.internal:Creating test directory... 2024-04-17T22:20:35.979 DEBUG:teuthology.orchestra.run.smithi102:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-17T22:20:36.015 DEBUG:teuthology.orchestra.run.smithi190:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-17T22:20:36.193 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-17T22:20:36.200 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-17T22:20:36.229 INFO:teuthology.task.internal:Creating archive directory... 2024-04-17T22:20:36.229 DEBUG:teuthology.orchestra.run.smithi102:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-17T22:20:36.261 DEBUG:teuthology.orchestra.run.smithi190:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-17T22:20:36.449 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-17T22:20:36.456 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-17T22:20:36.456 DEBUG:teuthology.orchestra.run.smithi102:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-17T22:20:36.460 DEBUG:teuthology.orchestra.run.smithi190:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-17T22:20:36.499 INFO:teuthology.orchestra.run.smithi102.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-17T22:20:36.501 INFO:teuthology.orchestra.run.smithi190.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-17T22:20:36.509 INFO:teuthology.orchestra.run.smithi102.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-17T22:20:36.513 INFO:teuthology.orchestra.run.smithi190.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-17T22:20:36.514 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-17T22:20:36.521 INFO:teuthology.task.internal:Configuring sudo... 2024-04-17T22:20:36.522 DEBUG:teuthology.orchestra.run.smithi102:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-17T22:20:36.553 DEBUG:teuthology.orchestra.run.smithi190:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-17T22:20:36.584 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-17T22:20:36.593 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-17T22:20:36.594 DEBUG:teuthology.orchestra.run.smithi102:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-17T22:20:36.620 DEBUG:teuthology.orchestra.run.smithi190:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-17T22:20:36.645 DEBUG:teuthology.orchestra.run.smithi102:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-17T22:20:36.723 DEBUG:teuthology.orchestra.run.smithi102:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-17T22:20:36.802 DEBUG:teuthology.orchestra.run.smithi102:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-17T22:20:36.848 DEBUG:teuthology.orchestra.run.smithi102:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-17T22:20:36.890 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-04-17T22:20:36.890 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-17T22:20:36.957 DEBUG:teuthology.orchestra.run.smithi190:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-17T22:20:37.009 DEBUG:teuthology.orchestra.run.smithi190:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-17T22:20:37.059 DEBUG:teuthology.orchestra.run.smithi190:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-17T22:20:37.143 DEBUG:teuthology.orchestra.run.smithi190:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-17T22:20:37.191 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-04-17T22:20:37.191 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-17T22:20:37.261 DEBUG:teuthology.orchestra.run.smithi102:> sudo service rsyslog restart 2024-04-17T22:20:37.263 DEBUG:teuthology.orchestra.run.smithi190:> sudo service rsyslog restart 2024-04-17T22:20:37.321 INFO:teuthology.orchestra.run.smithi102.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-17T22:20:37.354 INFO:teuthology.orchestra.run.smithi190.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-17T22:20:37.630 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-17T22:20:37.638 INFO:teuthology.task.internal:Starting timer... 2024-04-17T22:20:37.638 INFO:teuthology.run_tasks:Running task pcp... 2024-04-17T22:20:37.687 INFO:teuthology.run_tasks:Running task selinux... 2024-04-17T22:20:37.719 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0']} 2024-04-17T22:20:37.720 DEBUG:teuthology.orchestra.run.smithi102:> sudo service auditd rotate 2024-04-17T22:20:37.777 INFO:teuthology.orchestra.run.smithi102.stdout:Rotating logs: 2024-04-17T22:20:37.778 DEBUG:teuthology.orchestra.run.smithi190:> sudo service auditd rotate 2024-04-17T22:20:37.864 INFO:teuthology.orchestra.run.smithi190.stdout:Rotating logs: 2024-04-17T22:20:37.866 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-17T22:20:37.866 DEBUG:teuthology.orchestra.run.smithi102:> /usr/sbin/getenforce 2024-04-17T22:20:37.889 INFO:teuthology.orchestra.run.smithi102.stdout:Permissive 2024-04-17T22:20:37.889 DEBUG:teuthology.orchestra.run.smithi190:> /usr/sbin/getenforce 2024-04-17T22:20:37.931 INFO:teuthology.orchestra.run.smithi190.stdout:Permissive 2024-04-17T22:20:37.932 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi102.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi190.front.sepia.ceph.com': 'permissive'} 2024-04-17T22:20:37.932 DEBUG:teuthology.orchestra.run.smithi102:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-04-17T22:20:37.959 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-17T22:20:37.959 DEBUG:teuthology.orchestra.run.smithi190:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-04-17T22:20:38.001 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-17T22:20:38.001 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-17T22:20:38.001 DEBUG:teuthology.orchestra.run.smithi102:> sudo /usr/sbin/setenforce permissive 2024-04-17T22:20:38.029 DEBUG:teuthology.orchestra.run.smithi190:> sudo /usr/sbin/setenforce permissive 2024-04-17T22:20:38.069 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-17T22:20:38.079 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-04-17T22:20:38.085 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-04-17T22:20:38.229 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-17T22:20:38.244 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-17T22:20:38.245 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi102.front.sepia.ceph.com,smithi190.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-17T22:26:30.644 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi102.front.sepia.ceph.com'), Remote(name='ubuntu@smithi190.front.sepia.ceph.com')] 2024-04-17T22:26:30.646 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi102.front.sepia.ceph.com' 2024-04-17T22:26:30.647 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi102.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-17T22:26:30.752 DEBUG:teuthology.orchestra.run.smithi102:> true 2024-04-17T22:26:30.779 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi102.front.sepia.ceph.com' 2024-04-17T22:26:30.780 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi190.front.sepia.ceph.com' 2024-04-17T22:26:30.780 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi190.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-17T22:26:30.883 DEBUG:teuthology.orchestra.run.smithi190:> true 2024-04-17T22:26:30.952 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi190.front.sepia.ceph.com' 2024-04-17T22:26:30.953 INFO:teuthology.run_tasks:Running task clock... 2024-04-17T22:26:30.963 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-17T22:26:30.963 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-17T22:26:30.964 DEBUG:teuthology.orchestra.run.smithi102:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-17T22:26:30.966 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-17T22:26:30.966 DEBUG:teuthology.orchestra.run.smithi190:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-17T22:26:30.998 INFO:teuthology.orchestra.run.smithi102.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-17T22:26:31.012 INFO:teuthology.orchestra.run.smithi102.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-17T22:26:31.024 INFO:teuthology.orchestra.run.smithi190.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-17T22:26:31.037 INFO:teuthology.orchestra.run.smithi190.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-17T22:26:31.075 INFO:teuthology.orchestra.run.smithi102.stderr:sudo: ntpd: command not found 2024-04-17T22:26:31.086 INFO:teuthology.orchestra.run.smithi102.stdout:506 Cannot talk to daemon 2024-04-17T22:26:31.100 INFO:teuthology.orchestra.run.smithi102.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-17T22:26:31.111 INFO:teuthology.orchestra.run.smithi190.stderr:sudo: ntpd: command not found 2024-04-17T22:26:31.113 INFO:teuthology.orchestra.run.smithi102.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-17T22:26:31.122 INFO:teuthology.orchestra.run.smithi190.stdout:506 Cannot talk to daemon 2024-04-17T22:26:31.135 INFO:teuthology.orchestra.run.smithi190.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-17T22:26:31.148 INFO:teuthology.orchestra.run.smithi190.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-17T22:26:31.177 INFO:teuthology.orchestra.run.smithi102.stderr:bash: ntpq: command not found 2024-04-17T22:26:31.180 INFO:teuthology.orchestra.run.smithi102.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-17T22:26:31.180 INFO:teuthology.orchestra.run.smithi102.stdout:=============================================================================== 2024-04-17T22:26:31.180 INFO:teuthology.orchestra.run.smithi102.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-17T22:26:31.180 INFO:teuthology.orchestra.run.smithi102.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-17T22:26:31.180 INFO:teuthology.orchestra.run.smithi102.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-17T22:26:31.180 INFO:teuthology.orchestra.run.smithi102.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-17T22:26:31.210 INFO:teuthology.orchestra.run.smithi190.stderr:bash: ntpq: command not found 2024-04-17T22:26:31.213 INFO:teuthology.orchestra.run.smithi190.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-17T22:26:31.213 INFO:teuthology.orchestra.run.smithi190.stdout:=============================================================================== 2024-04-17T22:26:31.213 INFO:teuthology.orchestra.run.smithi190.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-17T22:26:31.214 INFO:teuthology.orchestra.run.smithi190.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-17T22:26:31.214 INFO:teuthology.orchestra.run.smithi190.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-17T22:26:31.214 INFO:teuthology.orchestra.run.smithi190.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-17T22:26:31.215 INFO:teuthology.run_tasks:Running task install... 2024-04-17T22:26:31.223 DEBUG:teuthology.task.install:project ceph 2024-04-17T22:26:31.223 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '41c500dd60c93c8d83053b72cffc3dd1a952a336'}} 2024-04-17T22:26:31.224 DEBUG:teuthology.task.install:config {'branch': 'nautilus', 'exclude_packages': ['cephadm', 'ceph-mgr-cephadm', 'ceph-immutable-object-cache', 'python3-rados', 'python3-rgw', 'python3-rbd', 'python3-cephfs', 'ceph-volume'], 'extra_packages': ['python-rados', 'python-rgw', 'python-rbd', 'python-cephfs'], 'extra_system_packages': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux'], 'flavor': 'default', 'sha1': '41c500dd60c93c8d83053b72cffc3dd1a952a336'} 2024-04-17T22:26:31.224 INFO:teuthology.task.install:Using flavor: default 2024-04-17T22:26:31.230 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'radosgw', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-fuse', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-17T22:26:31.230 INFO:teuthology.task.install:extra packages: ['python-rados', 'python-rgw', 'python-rbd', 'python-cephfs'] 2024-04-17T22:26:31.230 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': 'nautilus', 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': ['cephadm', 'ceph-mgr-cephadm', 'ceph-immutable-object-cache', 'python3-rados', 'python3-rgw', 'python3-rbd', 'python3-cephfs', 'ceph-volume'], 'extra_packages': ['python-rados', 'python-rgw', 'python-rbd', 'python-cephfs'], 'extra_system_packages': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux'], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '41c500dd60c93c8d83053b72cffc3dd1a952a336', 'tag': None, 'wait_for_package': False} 2024-04-17T22:26:31.231 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-17T22:26:31.231 INFO:teuthology.packaging:ref: None 2024-04-17T22:26:31.231 INFO:teuthology.packaging:tag: None 2024-04-17T22:26:31.231 INFO:teuthology.packaging:branch: nautilus 2024-04-17T22:26:31.231 INFO:teuthology.packaging:sha1: 41c500dd60c93c8d83053b72cffc3dd1a952a336 2024-04-17T22:26:31.231 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=nautilus 2024-04-17T22:26:31.233 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': 'nautilus', 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': ['cephadm', 'ceph-mgr-cephadm', 'ceph-immutable-object-cache', 'python3-rados', 'python3-rgw', 'python3-rbd', 'python3-cephfs', 'ceph-volume'], 'extra_packages': ['python-rados', 'python-rgw', 'python-rbd', 'python-cephfs'], 'extra_system_packages': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux'], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '41c500dd60c93c8d83053b72cffc3dd1a952a336', 'tag': None, 'wait_for_package': False} 2024-04-17T22:26:31.233 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-17T22:26:31.233 INFO:teuthology.packaging:ref: None 2024-04-17T22:26:31.234 INFO:teuthology.packaging:tag: None 2024-04-17T22:26:31.234 INFO:teuthology.packaging:branch: nautilus 2024-04-17T22:26:31.234 INFO:teuthology.packaging:sha1: 41c500dd60c93c8d83053b72cffc3dd1a952a336 2024-04-17T22:26:31.234 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=nautilus 2024-04-17T22:26:31.402 INFO:teuthology.task.install.rpm:Pulling from https://chacra.ceph.com/r/ceph/nautilus/ca74598065096e6fcbd8433c8779a2be0c889351/centos/8/flavors/default/ 2024-04-17T22:26:31.402 INFO:teuthology.task.install.rpm:Package version is 14.2.22-0 2024-04-17T22:26:31.433 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://chacra.ceph.com/r/ceph/nautilus/ca74598065096e6fcbd8433c8779a2be0c889351/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://chacra.ceph.com/r/ceph/nautilus/ca74598065096e6fcbd8433c8779a2be0c889351/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://chacra.ceph.com/r/ceph/nautilus/ca74598065096e6fcbd8433c8779a2be0c889351/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-17T22:26:31.434 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-04-17T22:26:31.434 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-17T22:26:31.462 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-fuse, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, rbd-fuse, rbd-mirror, rbd-nbd, python-rados, python-rgw, python-rbd, python-cephfs, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux on remote rpm x86_64 2024-04-17T22:26:31.462 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-17T22:26:31.462 INFO:teuthology.packaging:ref: None 2024-04-17T22:26:31.462 INFO:teuthology.packaging:tag: None 2024-04-17T22:26:31.462 INFO:teuthology.packaging:branch: nautilus 2024-04-17T22:26:31.462 INFO:teuthology.packaging:sha1: 41c500dd60c93c8d83053b72cffc3dd1a952a336 2024-04-17T22:26:31.466 DEBUG:teuthology.orchestra.run.smithi190:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;ref/nautilus/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-17T22:26:31.489 INFO:teuthology.task.install.rpm:Pulling from https://chacra.ceph.com/r/ceph/nautilus/ca74598065096e6fcbd8433c8779a2be0c889351/centos/8/flavors/default/ 2024-04-17T22:26:31.490 INFO:teuthology.task.install.rpm:Package version is 14.2.22-0 2024-04-17T22:26:31.522 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://chacra.ceph.com/r/ceph/nautilus/ca74598065096e6fcbd8433c8779a2be0c889351/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://chacra.ceph.com/r/ceph/nautilus/ca74598065096e6fcbd8433c8779a2be0c889351/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://chacra.ceph.com/r/ceph/nautilus/ca74598065096e6fcbd8433c8779a2be0c889351/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-17T22:26:31.522 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-04-17T22:26:31.522 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-17T22:26:31.534 DEBUG:teuthology.orchestra.run.smithi190:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-17T22:26:31.551 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-fuse, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, rbd-fuse, rbd-mirror, rbd-nbd, python-rados, python-rgw, python-rbd, python-cephfs, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux on remote rpm x86_64 2024-04-17T22:26:31.551 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-17T22:26:31.552 INFO:teuthology.packaging:ref: None 2024-04-17T22:26:31.552 INFO:teuthology.packaging:tag: None 2024-04-17T22:26:31.552 INFO:teuthology.packaging:branch: nautilus 2024-04-17T22:26:31.552 INFO:teuthology.packaging:sha1: 41c500dd60c93c8d83053b72cffc3dd1a952a336 2024-04-17T22:26:31.552 DEBUG:teuthology.orchestra.run.smithi102:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;ref/nautilus/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-17T22:26:31.610 DEBUG:teuthology.orchestra.run.smithi190:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-17T22:26:31.620 DEBUG:teuthology.orchestra.run.smithi102:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-17T22:26:31.676 INFO:teuthology.orchestra.run.smithi190.stdout:check_obsoletes = 1 2024-04-17T22:26:31.678 DEBUG:teuthology.orchestra.run.smithi190:> sudo yum clean all 2024-04-17T22:26:31.698 DEBUG:teuthology.orchestra.run.smithi102:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-17T22:26:31.765 INFO:teuthology.orchestra.run.smithi102.stdout:check_obsoletes = 1 2024-04-17T22:26:31.767 DEBUG:teuthology.orchestra.run.smithi102:> sudo yum clean all 2024-04-17T22:26:31.944 INFO:teuthology.orchestra.run.smithi190.stdout:78 files removed 2024-04-17T22:26:31.972 DEBUG:teuthology.orchestra.run.smithi190:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-fuse librados-devel libcephfs2 libcephfs-devel librados2 librbd1 rbd-fuse rbd-mirror rbd-nbd python-rados python-rgw python-rbd python-cephfs bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux 2024-04-17T22:26:32.033 INFO:teuthology.orchestra.run.smithi102.stdout:78 files removed 2024-04-17T22:26:32.060 DEBUG:teuthology.orchestra.run.smithi102:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-fuse librados-devel libcephfs2 libcephfs-devel librados2 librbd1 rbd-fuse rbd-mirror rbd-nbd python-rados python-rgw python-rbd python-cephfs bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux 2024-04-17T22:26:32.457 INFO:teuthology.orchestra.run.smithi190.stdout:ceph packages for x86_64 806 kB/s | 65 kB 00:00 2024-04-17T22:26:32.533 INFO:teuthology.orchestra.run.smithi102.stdout:ceph packages for x86_64 922 kB/s | 65 kB 00:00 2024-04-17T22:26:32.566 INFO:teuthology.orchestra.run.smithi190.stdout:ceph noarch packages 158 kB/s | 11 kB 00:00 2024-04-17T22:26:32.642 INFO:teuthology.orchestra.run.smithi102.stdout:ceph noarch packages 159 kB/s | 11 kB 00:00 2024-04-17T22:26:32.650 INFO:teuthology.orchestra.run.smithi190.stdout:ceph source packages 30 kB/s | 1.7 kB 00:00 2024-04-17T22:26:32.742 INFO:teuthology.orchestra.run.smithi102.stdout:ceph source packages 23 kB/s | 1.7 kB 00:00 2024-04-17T22:26:33.975 INFO:teuthology.orchestra.run.smithi190.stdout:CentOS Stream 8 - AppStream 22 MB/s | 28 MB 00:01 2024-04-17T22:26:34.243 INFO:teuthology.orchestra.run.smithi102.stdout:CentOS Stream 8 - AppStream 19 MB/s | 28 MB 00:01 2024-04-17T22:26:39.743 INFO:teuthology.orchestra.run.smithi190.stdout:CentOS Stream 8 - BaseOS 13 MB/s | 10 MB 00:00 2024-04-17T22:26:40.052 INFO:teuthology.orchestra.run.smithi102.stdout:CentOS Stream 8 - BaseOS 12 MB/s | 10 MB 00:00 2024-04-17T22:26:41.751 INFO:teuthology.orchestra.run.smithi190.stdout:CentOS Stream 8 - Extras 43 kB/s | 18 kB 00:00 2024-04-17T22:26:42.186 INFO:teuthology.orchestra.run.smithi102.stdout:CentOS Stream 8 - Extras 32 kB/s | 18 kB 00:00 2024-04-17T22:26:42.268 INFO:teuthology.orchestra.run.smithi190.stdout:CentOS Stream 8 - Extras common packages 19 kB/s | 7.7 kB 00:00 2024-04-17T22:26:42.803 INFO:teuthology.orchestra.run.smithi102.stdout:CentOS Stream 8 - Extras common packages 15 kB/s | 7.7 kB 00:00 2024-04-17T22:26:43.138 INFO:teuthology.orchestra.run.smithi190.stdout:CentOS Stream 8 - PowerTools 6.4 MB/s | 5.0 MB 00:00 2024-04-17T22:26:43.596 INFO:teuthology.orchestra.run.smithi102.stdout:CentOS Stream 8 - PowerTools 7.1 MB/s | 5.0 MB 00:00 2024-04-17T22:26:44.221 INFO:teuthology.orchestra.run.smithi190.stdout:Copr repo for python3-asyncssh owned by ceph 22 kB/s | 3.5 kB 00:00 2024-04-17T22:26:44.714 INFO:teuthology.orchestra.run.smithi102.stdout:Copr repo for python3-asyncssh owned by ceph 18 kB/s | 3.5 kB 00:00 2024-04-17T22:26:45.714 INFO:teuthology.orchestra.run.smithi190.stdout:Extra Packages for Enterprise Linux 12 MB/s | 16 MB 00:01 2024-04-17T22:26:46.276 INFO:teuthology.orchestra.run.smithi102.stdout:Extra Packages for Enterprise Linux 11 MB/s | 16 MB 00:01 2024-04-17T22:26:49.581 INFO:teuthology.orchestra.run.smithi190.stdout:lab-extras 199 kB/s | 24 kB 00:00 2024-04-17T22:26:50.141 INFO:teuthology.orchestra.run.smithi102.stdout:lab-extras 285 kB/s | 24 kB 00:00 2024-04-17T22:26:53.175 INFO:teuthology.orchestra.run.smithi190.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2024-04-17T22:26:53.175 INFO:teuthology.orchestra.run.smithi190.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2024-04-17T22:26:53.193 INFO:teuthology.orchestra.run.smithi190.stdout:Package NetworkManager-1:1.40.16-4.el8.x86_64 is already installed. 2024-04-17T22:26:53.194 INFO:teuthology.orchestra.run.smithi190.stdout:Package iproute-6.2.0-1.el8.x86_64 is already installed. 2024-04-17T22:26:53.194 INFO:teuthology.orchestra.run.smithi190.stdout:Package util-linux-2.32.1-41.el8.x86_64 is already installed. 2024-04-17T22:26:53.261 INFO:teuthology.orchestra.run.smithi190.stdout:Dependencies resolved. 2024-04-17T22:26:53.272 INFO:teuthology.orchestra.run.smithi190.stdout:============================================================================================ 2024-04-17T22:26:53.273 INFO:teuthology.orchestra.run.smithi190.stdout: Package Arch Version Repository Size 2024-04-17T22:26:53.273 INFO:teuthology.orchestra.run.smithi190.stdout:============================================================================================ 2024-04-17T22:26:53.273 INFO:teuthology.orchestra.run.smithi190.stdout:Installing: 2024-04-17T22:26:53.273 INFO:teuthology.orchestra.run.smithi190.stdout: bison x86_64 3.0.4-10.el8 appstream 688 k 2024-04-17T22:26:53.273 INFO:teuthology.orchestra.run.smithi190.stdout: ceph x86_64 2:14.2.22-0.el8 ceph 6.3 k 2024-04-17T22:26:53.273 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-base x86_64 2:14.2.22-0.el8 ceph 5.4 M 2024-04-17T22:26:53.273 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-fuse x86_64 2:14.2.22-0.el8 ceph 590 k 2024-04-17T22:26:53.273 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr x86_64 2:14.2.22-0.el8 ceph 1.6 M 2024-04-17T22:26:53.273 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr-dashboard noarch 2:14.2.22-0.el8 ceph-noarch 3.7 M 2024-04-17T22:26:53.273 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr-diskprediction-local noarch 2:14.2.22-0.el8 ceph-noarch 1.1 M 2024-04-17T22:26:53.273 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr-rook noarch 2:14.2.22-0.el8 ceph-noarch 15 k 2024-04-17T22:26:53.274 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-radosgw x86_64 2:14.2.22-0.el8 ceph 6.0 M 2024-04-17T22:26:53.274 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-test x86_64 2:14.2.22-0.el8 ceph 35 M 2024-04-17T22:26:53.274 INFO:teuthology.orchestra.run.smithi190.stdout: elfutils-libelf-devel x86_64 0.190-2.el8 baseos 62 k 2024-04-17T22:26:53.274 INFO:teuthology.orchestra.run.smithi190.stdout: flex x86_64 2.6.1-9.el8 appstream 320 k 2024-04-17T22:26:53.274 INFO:teuthology.orchestra.run.smithi190.stdout: libcephfs-devel x86_64 2:14.2.22-0.el8 ceph 22 k 2024-04-17T22:26:53.274 INFO:teuthology.orchestra.run.smithi190.stdout: libcephfs2 x86_64 2:14.2.22-0.el8 ceph 520 k 2024-04-17T22:26:53.274 INFO:teuthology.orchestra.run.smithi190.stdout: librados-devel x86_64 2:14.2.22-0.el8 ceph 90 k 2024-04-17T22:26:53.274 INFO:teuthology.orchestra.run.smithi190.stdout: openssl-devel x86_64 1:1.1.1k-12.el8 baseos 3.2 M 2024-04-17T22:26:53.274 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cephfs x86_64 2:14.2.22-0.el8 ceph 159 k 2024-04-17T22:26:53.274 INFO:teuthology.orchestra.run.smithi190.stdout: python3-rados x86_64 2:14.2.22-0.el8 ceph 323 k 2024-04-17T22:26:53.274 INFO:teuthology.orchestra.run.smithi190.stdout: python3-rbd x86_64 2:14.2.22-0.el8 ceph 291 k 2024-04-17T22:26:53.274 INFO:teuthology.orchestra.run.smithi190.stdout: python3-rgw x86_64 2:14.2.22-0.el8 ceph 113 k 2024-04-17T22:26:53.274 INFO:teuthology.orchestra.run.smithi190.stdout: rbd-fuse x86_64 2:14.2.22-0.el8 ceph 73 k 2024-04-17T22:26:53.274 INFO:teuthology.orchestra.run.smithi190.stdout: rbd-mirror x86_64 2:14.2.22-0.el8 ceph 2.3 M 2024-04-17T22:26:53.275 INFO:teuthology.orchestra.run.smithi190.stdout: rbd-nbd x86_64 2:14.2.22-0.el8 ceph 142 k 2024-04-17T22:26:53.275 INFO:teuthology.orchestra.run.smithi190.stdout:Upgrading: 2024-04-17T22:26:53.275 INFO:teuthology.orchestra.run.smithi190.stdout: NetworkManager x86_64 1:1.40.16-15.el8 baseos 2.3 M 2024-04-17T22:26:53.275 INFO:teuthology.orchestra.run.smithi190.stdout: NetworkManager-libnm x86_64 1:1.40.16-15.el8 baseos 1.9 M 2024-04-17T22:26:53.275 INFO:teuthology.orchestra.run.smithi190.stdout: NetworkManager-team x86_64 1:1.40.16-15.el8 baseos 161 k 2024-04-17T22:26:53.275 INFO:teuthology.orchestra.run.smithi190.stdout: NetworkManager-tui x86_64 1:1.40.16-15.el8 baseos 356 k 2024-04-17T22:26:53.275 INFO:teuthology.orchestra.run.smithi190.stdout: elfutils-debuginfod-client x86_64 0.190-2.el8 baseos 77 k 2024-04-17T22:26:53.275 INFO:teuthology.orchestra.run.smithi190.stdout: elfutils-libelf x86_64 0.190-2.el8 baseos 233 k 2024-04-17T22:26:53.275 INFO:teuthology.orchestra.run.smithi190.stdout: elfutils-libs x86_64 0.190-2.el8 baseos 305 k 2024-04-17T22:26:53.275 INFO:teuthology.orchestra.run.smithi190.stdout: iproute x86_64 6.2.0-5.el8 baseos 854 k 2024-04-17T22:26:53.275 INFO:teuthology.orchestra.run.smithi190.stdout: libblkid x86_64 2.32.1-46.el8 baseos 221 k 2024-04-17T22:26:53.275 INFO:teuthology.orchestra.run.smithi190.stdout: libfdisk x86_64 2.32.1-46.el8 baseos 254 k 2024-04-17T22:26:53.275 INFO:teuthology.orchestra.run.smithi190.stdout: libmount x86_64 2.32.1-46.el8 baseos 237 k 2024-04-17T22:26:53.275 INFO:teuthology.orchestra.run.smithi190.stdout: librados2 x86_64 2:14.2.22-0.el8 ceph 3.5 M 2024-04-17T22:26:53.276 INFO:teuthology.orchestra.run.smithi190.stdout: librbd1 x86_64 2:14.2.22-0.el8 ceph 1.7 M 2024-04-17T22:26:53.276 INFO:teuthology.orchestra.run.smithi190.stdout: libsmartcols x86_64 2.32.1-46.el8 baseos 180 k 2024-04-17T22:26:53.276 INFO:teuthology.orchestra.run.smithi190.stdout: libuuid x86_64 2.32.1-46.el8 baseos 99 k 2024-04-17T22:26:53.276 INFO:teuthology.orchestra.run.smithi190.stdout: openssl x86_64 1:1.1.1k-12.el8 baseos 738 k 2024-04-17T22:26:53.276 INFO:teuthology.orchestra.run.smithi190.stdout: openssl-libs x86_64 1:1.1.1k-12.el8 baseos 1.5 M 2024-04-17T22:26:53.276 INFO:teuthology.orchestra.run.smithi190.stdout: util-linux x86_64 2.32.1-46.el8 baseos 2.6 M 2024-04-17T22:26:53.276 INFO:teuthology.orchestra.run.smithi190.stdout: util-linux-user x86_64 2.32.1-46.el8 baseos 103 k 2024-04-17T22:26:53.276 INFO:teuthology.orchestra.run.smithi190.stdout:Installing dependencies: 2024-04-17T22:26:53.276 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-common x86_64 2:14.2.22-0.el8 ceph 19 M 2024-04-17T22:26:53.276 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-grafana-dashboards noarch 2:14.2.22-0.el8 ceph-noarch 24 k 2024-04-17T22:26:53.276 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mds x86_64 2:14.2.22-0.el8 ceph 2.0 M 2024-04-17T22:26:53.276 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mon x86_64 2:14.2.22-0.el8 ceph 4.5 M 2024-04-17T22:26:53.276 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-osd x86_64 2:14.2.22-0.el8 ceph 17 M 2024-04-17T22:26:53.276 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-selinux x86_64 2:14.2.22-0.el8 ceph 25 k 2024-04-17T22:26:53.276 INFO:teuthology.orchestra.run.smithi190.stdout: fuse x86_64 2.9.7-19.el8 baseos 83 k 2024-04-17T22:26:53.276 INFO:teuthology.orchestra.run.smithi190.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2024-04-17T22:26:53.277 INFO:teuthology.orchestra.run.smithi190.stdout: jq x86_64 1.6-8.el8 appstream 203 k 2024-04-17T22:26:53.277 INFO:teuthology.orchestra.run.smithi190.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 baseos 48 k 2024-04-17T22:26:53.277 INFO:teuthology.orchestra.run.smithi190.stdout: krb5-devel x86_64 1.18.2-22.el8 baseos 561 k 2024-04-17T22:26:53.277 INFO:teuthology.orchestra.run.smithi190.stdout: leveldb x86_64 1.22-1.el8 epel 181 k 2024-04-17T22:26:53.277 INFO:teuthology.orchestra.run.smithi190.stdout: libcom_err-devel x86_64 1.45.6-5.el8 baseos 39 k 2024-04-17T22:26:53.277 INFO:teuthology.orchestra.run.smithi190.stdout: libgfortran x86_64 8.5.0-21.el8 baseos 645 k 2024-04-17T22:26:53.277 INFO:teuthology.orchestra.run.smithi190.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2024-04-17T22:26:53.277 INFO:teuthology.orchestra.run.smithi190.stdout: libquadmath x86_64 8.5.0-21.el8 baseos 172 k 2024-04-17T22:26:53.277 INFO:teuthology.orchestra.run.smithi190.stdout: librabbitmq x86_64 0.9.0-5.el8 baseos 47 k 2024-04-17T22:26:53.277 INFO:teuthology.orchestra.run.smithi190.stdout: libradosstriper1 x86_64 2:14.2.22-0.el8 ceph 376 k 2024-04-17T22:26:53.277 INFO:teuthology.orchestra.run.smithi190.stdout: librdkafka x86_64 1.6.1-1.el8 appstream 677 k 2024-04-17T22:26:53.277 INFO:teuthology.orchestra.run.smithi190.stdout: librgw2 x86_64 2:14.2.22-0.el8 ceph 5.3 M 2024-04-17T22:26:53.277 INFO:teuthology.orchestra.run.smithi190.stdout: libselinux-devel x86_64 2.9-8.el8 baseos 200 k 2024-04-17T22:26:53.277 INFO:teuthology.orchestra.run.smithi190.stdout: libsepol-devel x86_64 2.9-3.el8 baseos 87 k 2024-04-17T22:26:53.277 INFO:teuthology.orchestra.run.smithi190.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2024-04-17T22:26:53.278 INFO:teuthology.orchestra.run.smithi190.stdout: libverto-devel x86_64 0.3.2-2.el8 baseos 18 k 2024-04-17T22:26:53.278 INFO:teuthology.orchestra.run.smithi190.stdout: libzstd-devel x86_64 1.4.4-1.el8 baseos 44 k 2024-04-17T22:26:53.278 INFO:teuthology.orchestra.run.smithi190.stdout: lttng-ust x86_64 2.8.1-11.el8 appstream 259 k 2024-04-17T22:26:53.278 INFO:teuthology.orchestra.run.smithi190.stdout: oniguruma x86_64 6.8.2-3.el8 appstream 188 k 2024-04-17T22:26:53.278 INFO:teuthology.orchestra.run.smithi190.stdout: openblas x86_64 0.3.15-4.el8 appstream 4.7 M 2024-04-17T22:26:53.278 INFO:teuthology.orchestra.run.smithi190.stdout: openblas-threads x86_64 0.3.15-4.el8 appstream 4.9 M 2024-04-17T22:26:53.278 INFO:teuthology.orchestra.run.smithi190.stdout: pcre2-devel x86_64 10.32-3.el8 baseos 605 k 2024-04-17T22:26:53.278 INFO:teuthology.orchestra.run.smithi190.stdout: pcre2-utf16 x86_64 10.32-3.el8 baseos 229 k 2024-04-17T22:26:53.278 INFO:teuthology.orchestra.run.smithi190.stdout: pcre2-utf32 x86_64 10.32-3.el8 baseos 220 k 2024-04-17T22:26:53.278 INFO:teuthology.orchestra.run.smithi190.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2024-04-17T22:26:53.278 INFO:teuthology.orchestra.run.smithi190.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2024-04-17T22:26:53.278 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2024-04-17T22:26:53.278 INFO:teuthology.orchestra.run.smithi190.stdout: python3-ceph-argparse x86_64 2:14.2.22-0.el8 ceph 39 k 2024-04-17T22:26:53.278 INFO:teuthology.orchestra.run.smithi190.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2024-04-17T22:26:53.279 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cffi x86_64 1.11.5-6.el8 baseos 249 k 2024-04-17T22:26:53.279 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2024-04-17T22:26:53.279 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2024-04-17T22:26:53.279 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cryptography x86_64 3.2.1-7.el8 baseos 641 k 2024-04-17T22:26:53.279 INFO:teuthology.orchestra.run.smithi190.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2024-04-17T22:26:53.279 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2024-04-17T22:26:53.279 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2024-04-17T22:26:53.279 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jwt noarch 1.6.1-2.el8 baseos 43 k 2024-04-17T22:26:53.279 INFO:teuthology.orchestra.run.smithi190.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2024-04-17T22:26:53.279 INFO:teuthology.orchestra.run.smithi190.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2024-04-17T22:26:53.279 INFO:teuthology.orchestra.run.smithi190.stdout: python3-mako noarch 1.0.6-14.el8 appstream 157 k 2024-04-17T22:26:53.279 INFO:teuthology.orchestra.run.smithi190.stdout: python3-markupsafe x86_64 0.23-19.el8 appstream 39 k 2024-04-17T22:26:53.279 INFO:teuthology.orchestra.run.smithi190.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2024-04-17T22:26:53.279 INFO:teuthology.orchestra.run.smithi190.stdout: python3-numpy x86_64 1:1.14.3-10.el8 appstream 3.7 M 2024-04-17T22:26:53.280 INFO:teuthology.orchestra.run.smithi190.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 appstream 225 k 2024-04-17T22:26:53.280 INFO:teuthology.orchestra.run.smithi190.stdout: python3-oauthlib noarch 2.1.0-1.el8 baseos 155 k 2024-04-17T22:26:53.280 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2024-04-17T22:26:53.280 INFO:teuthology.orchestra.run.smithi190.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2024-04-17T22:26:53.280 INFO:teuthology.orchestra.run.smithi190.stdout: python3-prettytable noarch 0.7.2-14.el8 appstream 44 k 2024-04-17T22:26:53.280 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 appstream 103 k 2024-04-17T22:26:53.280 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pyasn1 noarch 0.3.7-6.el8 appstream 126 k 2024-04-17T22:26:53.280 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 appstream 110 k 2024-04-17T22:26:53.280 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pycparser noarch 2.14-14.el8 baseos 109 k 2024-04-17T22:26:53.280 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pytz noarch 2017.2-11.el8 appstream 57 k 2024-04-17T22:26:53.280 INFO:teuthology.orchestra.run.smithi190.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2024-04-17T22:26:53.280 INFO:teuthology.orchestra.run.smithi190.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 baseos 43 k 2024-04-17T22:26:53.280 INFO:teuthology.orchestra.run.smithi190.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2024-04-17T22:26:53.280 INFO:teuthology.orchestra.run.smithi190.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2024-04-17T22:26:53.280 INFO:teuthology.orchestra.run.smithi190.stdout: python3-scipy x86_64 1.0.0-21.module_el8+762+77bd8591 appstream 14 M 2024-04-17T22:26:53.281 INFO:teuthology.orchestra.run.smithi190.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2024-04-17T22:26:53.281 INFO:teuthology.orchestra.run.smithi190.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2024-04-17T22:26:53.281 INFO:teuthology.orchestra.run.smithi190.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2024-04-17T22:26:53.281 INFO:teuthology.orchestra.run.smithi190.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2024-04-17T22:26:53.281 INFO:teuthology.orchestra.run.smithi190.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2024-04-17T22:26:53.281 INFO:teuthology.orchestra.run.smithi190.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2024-04-17T22:26:53.281 INFO:teuthology.orchestra.run.smithi190.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2024-04-17T22:26:53.281 INFO:teuthology.orchestra.run.smithi190.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2024-04-17T22:26:53.281 INFO:teuthology.orchestra.run.smithi190.stdout: python3-werkzeug noarch 0.12.2-4.el8 appstream 457 k 2024-04-17T22:26:53.281 INFO:teuthology.orchestra.run.smithi190.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2024-04-17T22:26:53.281 INFO:teuthology.orchestra.run.smithi190.stdout: socat x86_64 1.7.4.1-1.el8 appstream 323 k 2024-04-17T22:26:53.281 INFO:teuthology.orchestra.run.smithi190.stdout: xmlstarlet x86_64 1.6.1-20.el8 appstream 71 k 2024-04-17T22:26:53.281 INFO:teuthology.orchestra.run.smithi190.stdout: zlib-devel x86_64 1.2.11-21.el8 baseos 58 k 2024-04-17T22:26:53.281 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-04-17T22:26:53.282 INFO:teuthology.orchestra.run.smithi190.stdout:Transaction Summary 2024-04-17T22:26:53.282 INFO:teuthology.orchestra.run.smithi190.stdout:============================================================================================ 2024-04-17T22:26:53.282 INFO:teuthology.orchestra.run.smithi190.stdout:Install 102 Packages 2024-04-17T22:26:53.282 INFO:teuthology.orchestra.run.smithi190.stdout:Upgrade 19 Packages 2024-04-17T22:26:53.282 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-04-17T22:26:53.283 INFO:teuthology.orchestra.run.smithi190.stdout:Total download size: 166 M 2024-04-17T22:26:53.283 INFO:teuthology.orchestra.run.smithi190.stdout:Downloading Packages: 2024-04-17T22:26:53.754 INFO:teuthology.orchestra.run.smithi102.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2024-04-17T22:26:53.755 INFO:teuthology.orchestra.run.smithi102.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2024-04-17T22:26:53.773 INFO:teuthology.orchestra.run.smithi102.stdout:Package NetworkManager-1:1.40.16-4.el8.x86_64 is already installed. 2024-04-17T22:26:53.774 INFO:teuthology.orchestra.run.smithi102.stdout:Package iproute-6.2.0-1.el8.x86_64 is already installed. 2024-04-17T22:26:53.775 INFO:teuthology.orchestra.run.smithi102.stdout:Package util-linux-2.32.1-41.el8.x86_64 is already installed. 2024-04-17T22:26:53.799 INFO:teuthology.orchestra.run.smithi190.stdout:(1/121): ceph-14.2.22-0.el8.x86_64.rpm 112 kB/s | 6.3 kB 00:00 2024-04-17T22:26:53.841 INFO:teuthology.orchestra.run.smithi190.stdout:(2/121): ceph-fuse-14.2.22-0.el8.x86_64.rpm 14 MB/s | 590 kB 00:00 2024-04-17T22:26:53.842 INFO:teuthology.orchestra.run.smithi102.stdout:Dependencies resolved. 2024-04-17T22:26:53.854 INFO:teuthology.orchestra.run.smithi102.stdout:============================================================================================ 2024-04-17T22:26:53.854 INFO:teuthology.orchestra.run.smithi102.stdout: Package Arch Version Repository Size 2024-04-17T22:26:53.854 INFO:teuthology.orchestra.run.smithi102.stdout:============================================================================================ 2024-04-17T22:26:53.854 INFO:teuthology.orchestra.run.smithi102.stdout:Installing: 2024-04-17T22:26:53.855 INFO:teuthology.orchestra.run.smithi102.stdout: bison x86_64 3.0.4-10.el8 appstream 688 k 2024-04-17T22:26:53.855 INFO:teuthology.orchestra.run.smithi102.stdout: ceph x86_64 2:14.2.22-0.el8 ceph 6.3 k 2024-04-17T22:26:53.855 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-base x86_64 2:14.2.22-0.el8 ceph 5.4 M 2024-04-17T22:26:53.855 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-fuse x86_64 2:14.2.22-0.el8 ceph 590 k 2024-04-17T22:26:53.855 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mgr x86_64 2:14.2.22-0.el8 ceph 1.6 M 2024-04-17T22:26:53.855 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mgr-dashboard noarch 2:14.2.22-0.el8 ceph-noarch 3.7 M 2024-04-17T22:26:53.855 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mgr-diskprediction-local noarch 2:14.2.22-0.el8 ceph-noarch 1.1 M 2024-04-17T22:26:53.855 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mgr-rook noarch 2:14.2.22-0.el8 ceph-noarch 15 k 2024-04-17T22:26:53.855 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-radosgw x86_64 2:14.2.22-0.el8 ceph 6.0 M 2024-04-17T22:26:53.855 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-test x86_64 2:14.2.22-0.el8 ceph 35 M 2024-04-17T22:26:53.856 INFO:teuthology.orchestra.run.smithi102.stdout: elfutils-libelf-devel x86_64 0.190-2.el8 baseos 62 k 2024-04-17T22:26:53.856 INFO:teuthology.orchestra.run.smithi102.stdout: flex x86_64 2.6.1-9.el8 appstream 320 k 2024-04-17T22:26:53.856 INFO:teuthology.orchestra.run.smithi102.stdout: libcephfs-devel x86_64 2:14.2.22-0.el8 ceph 22 k 2024-04-17T22:26:53.856 INFO:teuthology.orchestra.run.smithi102.stdout: libcephfs2 x86_64 2:14.2.22-0.el8 ceph 520 k 2024-04-17T22:26:53.856 INFO:teuthology.orchestra.run.smithi102.stdout: librados-devel x86_64 2:14.2.22-0.el8 ceph 90 k 2024-04-17T22:26:53.856 INFO:teuthology.orchestra.run.smithi102.stdout: openssl-devel x86_64 1:1.1.1k-12.el8 baseos 3.2 M 2024-04-17T22:26:53.856 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cephfs x86_64 2:14.2.22-0.el8 ceph 159 k 2024-04-17T22:26:53.856 INFO:teuthology.orchestra.run.smithi102.stdout: python3-rados x86_64 2:14.2.22-0.el8 ceph 323 k 2024-04-17T22:26:53.856 INFO:teuthology.orchestra.run.smithi102.stdout: python3-rbd x86_64 2:14.2.22-0.el8 ceph 291 k 2024-04-17T22:26:53.856 INFO:teuthology.orchestra.run.smithi102.stdout: python3-rgw x86_64 2:14.2.22-0.el8 ceph 113 k 2024-04-17T22:26:53.856 INFO:teuthology.orchestra.run.smithi102.stdout: rbd-fuse x86_64 2:14.2.22-0.el8 ceph 73 k 2024-04-17T22:26:53.856 INFO:teuthology.orchestra.run.smithi102.stdout: rbd-mirror x86_64 2:14.2.22-0.el8 ceph 2.3 M 2024-04-17T22:26:53.856 INFO:teuthology.orchestra.run.smithi102.stdout: rbd-nbd x86_64 2:14.2.22-0.el8 ceph 142 k 2024-04-17T22:26:53.857 INFO:teuthology.orchestra.run.smithi102.stdout:Upgrading: 2024-04-17T22:26:53.857 INFO:teuthology.orchestra.run.smithi102.stdout: NetworkManager x86_64 1:1.40.16-15.el8 baseos 2.3 M 2024-04-17T22:26:53.857 INFO:teuthology.orchestra.run.smithi102.stdout: NetworkManager-libnm x86_64 1:1.40.16-15.el8 baseos 1.9 M 2024-04-17T22:26:53.857 INFO:teuthology.orchestra.run.smithi102.stdout: NetworkManager-team x86_64 1:1.40.16-15.el8 baseos 161 k 2024-04-17T22:26:53.857 INFO:teuthology.orchestra.run.smithi102.stdout: NetworkManager-tui x86_64 1:1.40.16-15.el8 baseos 356 k 2024-04-17T22:26:53.857 INFO:teuthology.orchestra.run.smithi102.stdout: elfutils-debuginfod-client x86_64 0.190-2.el8 baseos 77 k 2024-04-17T22:26:53.857 INFO:teuthology.orchestra.run.smithi102.stdout: elfutils-libelf x86_64 0.190-2.el8 baseos 233 k 2024-04-17T22:26:53.857 INFO:teuthology.orchestra.run.smithi102.stdout: elfutils-libs x86_64 0.190-2.el8 baseos 305 k 2024-04-17T22:26:53.857 INFO:teuthology.orchestra.run.smithi102.stdout: iproute x86_64 6.2.0-5.el8 baseos 854 k 2024-04-17T22:26:53.857 INFO:teuthology.orchestra.run.smithi102.stdout: libblkid x86_64 2.32.1-46.el8 baseos 221 k 2024-04-17T22:26:53.857 INFO:teuthology.orchestra.run.smithi102.stdout: libfdisk x86_64 2.32.1-46.el8 baseos 254 k 2024-04-17T22:26:53.857 INFO:teuthology.orchestra.run.smithi102.stdout: libmount x86_64 2.32.1-46.el8 baseos 237 k 2024-04-17T22:26:53.857 INFO:teuthology.orchestra.run.smithi102.stdout: librados2 x86_64 2:14.2.22-0.el8 ceph 3.5 M 2024-04-17T22:26:53.858 INFO:teuthology.orchestra.run.smithi102.stdout: librbd1 x86_64 2:14.2.22-0.el8 ceph 1.7 M 2024-04-17T22:26:53.858 INFO:teuthology.orchestra.run.smithi102.stdout: libsmartcols x86_64 2.32.1-46.el8 baseos 180 k 2024-04-17T22:26:53.858 INFO:teuthology.orchestra.run.smithi102.stdout: libuuid x86_64 2.32.1-46.el8 baseos 99 k 2024-04-17T22:26:53.858 INFO:teuthology.orchestra.run.smithi102.stdout: openssl x86_64 1:1.1.1k-12.el8 baseos 738 k 2024-04-17T22:26:53.858 INFO:teuthology.orchestra.run.smithi102.stdout: openssl-libs x86_64 1:1.1.1k-12.el8 baseos 1.5 M 2024-04-17T22:26:53.858 INFO:teuthology.orchestra.run.smithi102.stdout: util-linux x86_64 2.32.1-46.el8 baseos 2.6 M 2024-04-17T22:26:53.858 INFO:teuthology.orchestra.run.smithi102.stdout: util-linux-user x86_64 2.32.1-46.el8 baseos 103 k 2024-04-17T22:26:53.858 INFO:teuthology.orchestra.run.smithi102.stdout:Installing dependencies: 2024-04-17T22:26:53.858 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-common x86_64 2:14.2.22-0.el8 ceph 19 M 2024-04-17T22:26:53.858 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-grafana-dashboards noarch 2:14.2.22-0.el8 ceph-noarch 24 k 2024-04-17T22:26:53.858 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mds x86_64 2:14.2.22-0.el8 ceph 2.0 M 2024-04-17T22:26:53.858 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mon x86_64 2:14.2.22-0.el8 ceph 4.5 M 2024-04-17T22:26:53.858 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-osd x86_64 2:14.2.22-0.el8 ceph 17 M 2024-04-17T22:26:53.859 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-selinux x86_64 2:14.2.22-0.el8 ceph 25 k 2024-04-17T22:26:53.859 INFO:teuthology.orchestra.run.smithi102.stdout: fuse x86_64 2.9.7-19.el8 baseos 83 k 2024-04-17T22:26:53.859 INFO:teuthology.orchestra.run.smithi102.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2024-04-17T22:26:53.859 INFO:teuthology.orchestra.run.smithi102.stdout: jq x86_64 1.6-8.el8 appstream 203 k 2024-04-17T22:26:53.859 INFO:teuthology.orchestra.run.smithi102.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 baseos 48 k 2024-04-17T22:26:53.859 INFO:teuthology.orchestra.run.smithi102.stdout: krb5-devel x86_64 1.18.2-22.el8 baseos 561 k 2024-04-17T22:26:53.859 INFO:teuthology.orchestra.run.smithi102.stdout: leveldb x86_64 1.22-1.el8 epel 181 k 2024-04-17T22:26:53.859 INFO:teuthology.orchestra.run.smithi102.stdout: libcom_err-devel x86_64 1.45.6-5.el8 baseos 39 k 2024-04-17T22:26:53.859 INFO:teuthology.orchestra.run.smithi102.stdout: libgfortran x86_64 8.5.0-21.el8 baseos 645 k 2024-04-17T22:26:53.859 INFO:teuthology.orchestra.run.smithi102.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2024-04-17T22:26:53.859 INFO:teuthology.orchestra.run.smithi102.stdout: libquadmath x86_64 8.5.0-21.el8 baseos 172 k 2024-04-17T22:26:53.859 INFO:teuthology.orchestra.run.smithi102.stdout: librabbitmq x86_64 0.9.0-5.el8 baseos 47 k 2024-04-17T22:26:53.860 INFO:teuthology.orchestra.run.smithi102.stdout: libradosstriper1 x86_64 2:14.2.22-0.el8 ceph 376 k 2024-04-17T22:26:53.860 INFO:teuthology.orchestra.run.smithi102.stdout: librdkafka x86_64 1.6.1-1.el8 appstream 677 k 2024-04-17T22:26:53.860 INFO:teuthology.orchestra.run.smithi102.stdout: librgw2 x86_64 2:14.2.22-0.el8 ceph 5.3 M 2024-04-17T22:26:53.860 INFO:teuthology.orchestra.run.smithi102.stdout: libselinux-devel x86_64 2.9-8.el8 baseos 200 k 2024-04-17T22:26:53.860 INFO:teuthology.orchestra.run.smithi102.stdout: libsepol-devel x86_64 2.9-3.el8 baseos 87 k 2024-04-17T22:26:53.860 INFO:teuthology.orchestra.run.smithi102.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2024-04-17T22:26:53.860 INFO:teuthology.orchestra.run.smithi102.stdout: libverto-devel x86_64 0.3.2-2.el8 baseos 18 k 2024-04-17T22:26:53.860 INFO:teuthology.orchestra.run.smithi102.stdout: libzstd-devel x86_64 1.4.4-1.el8 baseos 44 k 2024-04-17T22:26:53.860 INFO:teuthology.orchestra.run.smithi102.stdout: lttng-ust x86_64 2.8.1-11.el8 appstream 259 k 2024-04-17T22:26:53.860 INFO:teuthology.orchestra.run.smithi102.stdout: oniguruma x86_64 6.8.2-3.el8 appstream 188 k 2024-04-17T22:26:53.860 INFO:teuthology.orchestra.run.smithi102.stdout: openblas x86_64 0.3.15-4.el8 appstream 4.7 M 2024-04-17T22:26:53.860 INFO:teuthology.orchestra.run.smithi102.stdout: openblas-threads x86_64 0.3.15-4.el8 appstream 4.9 M 2024-04-17T22:26:53.860 INFO:teuthology.orchestra.run.smithi102.stdout: pcre2-devel x86_64 10.32-3.el8 baseos 605 k 2024-04-17T22:26:53.860 INFO:teuthology.orchestra.run.smithi102.stdout: pcre2-utf16 x86_64 10.32-3.el8 baseos 229 k 2024-04-17T22:26:53.861 INFO:teuthology.orchestra.run.smithi102.stdout: pcre2-utf32 x86_64 10.32-3.el8 baseos 220 k 2024-04-17T22:26:53.861 INFO:teuthology.orchestra.run.smithi102.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2024-04-17T22:26:53.861 INFO:teuthology.orchestra.run.smithi102.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2024-04-17T22:26:53.861 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2024-04-17T22:26:53.861 INFO:teuthology.orchestra.run.smithi102.stdout: python3-ceph-argparse x86_64 2:14.2.22-0.el8 ceph 39 k 2024-04-17T22:26:53.861 INFO:teuthology.orchestra.run.smithi102.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2024-04-17T22:26:53.861 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cffi x86_64 1.11.5-6.el8 baseos 249 k 2024-04-17T22:26:53.861 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2024-04-17T22:26:53.861 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2024-04-17T22:26:53.861 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cryptography x86_64 3.2.1-7.el8 baseos 641 k 2024-04-17T22:26:53.861 INFO:teuthology.orchestra.run.smithi102.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2024-04-17T22:26:53.861 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2024-04-17T22:26:53.861 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2024-04-17T22:26:53.861 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jwt noarch 1.6.1-2.el8 baseos 43 k 2024-04-17T22:26:53.862 INFO:teuthology.orchestra.run.smithi102.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2024-04-17T22:26:53.862 INFO:teuthology.orchestra.run.smithi102.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2024-04-17T22:26:53.862 INFO:teuthology.orchestra.run.smithi102.stdout: python3-mako noarch 1.0.6-14.el8 appstream 157 k 2024-04-17T22:26:53.862 INFO:teuthology.orchestra.run.smithi102.stdout: python3-markupsafe x86_64 0.23-19.el8 appstream 39 k 2024-04-17T22:26:53.862 INFO:teuthology.orchestra.run.smithi102.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2024-04-17T22:26:53.862 INFO:teuthology.orchestra.run.smithi102.stdout: python3-numpy x86_64 1:1.14.3-10.el8 appstream 3.7 M 2024-04-17T22:26:53.862 INFO:teuthology.orchestra.run.smithi102.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 appstream 225 k 2024-04-17T22:26:53.862 INFO:teuthology.orchestra.run.smithi102.stdout: python3-oauthlib noarch 2.1.0-1.el8 baseos 155 k 2024-04-17T22:26:53.862 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2024-04-17T22:26:53.862 INFO:teuthology.orchestra.run.smithi102.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2024-04-17T22:26:53.862 INFO:teuthology.orchestra.run.smithi102.stdout: python3-prettytable noarch 0.7.2-14.el8 appstream 44 k 2024-04-17T22:26:53.862 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 appstream 103 k 2024-04-17T22:26:53.862 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pyasn1 noarch 0.3.7-6.el8 appstream 126 k 2024-04-17T22:26:53.862 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 appstream 110 k 2024-04-17T22:26:53.863 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pycparser noarch 2.14-14.el8 baseos 109 k 2024-04-17T22:26:53.863 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pytz noarch 2017.2-11.el8 appstream 57 k 2024-04-17T22:26:53.863 INFO:teuthology.orchestra.run.smithi102.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2024-04-17T22:26:53.863 INFO:teuthology.orchestra.run.smithi102.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 baseos 43 k 2024-04-17T22:26:53.863 INFO:teuthology.orchestra.run.smithi102.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2024-04-17T22:26:53.863 INFO:teuthology.orchestra.run.smithi102.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2024-04-17T22:26:53.863 INFO:teuthology.orchestra.run.smithi102.stdout: python3-scipy x86_64 1.0.0-21.module_el8+762+77bd8591 appstream 14 M 2024-04-17T22:26:53.863 INFO:teuthology.orchestra.run.smithi102.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2024-04-17T22:26:53.863 INFO:teuthology.orchestra.run.smithi102.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2024-04-17T22:26:53.863 INFO:teuthology.orchestra.run.smithi102.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2024-04-17T22:26:53.863 INFO:teuthology.orchestra.run.smithi102.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2024-04-17T22:26:53.863 INFO:teuthology.orchestra.run.smithi102.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2024-04-17T22:26:53.863 INFO:teuthology.orchestra.run.smithi102.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2024-04-17T22:26:53.863 INFO:teuthology.orchestra.run.smithi102.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2024-04-17T22:26:53.863 INFO:teuthology.orchestra.run.smithi102.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2024-04-17T22:26:53.864 INFO:teuthology.orchestra.run.smithi102.stdout: python3-werkzeug noarch 0.12.2-4.el8 appstream 457 k 2024-04-17T22:26:53.864 INFO:teuthology.orchestra.run.smithi102.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2024-04-17T22:26:53.864 INFO:teuthology.orchestra.run.smithi102.stdout: socat x86_64 1.7.4.1-1.el8 appstream 323 k 2024-04-17T22:26:53.864 INFO:teuthology.orchestra.run.smithi102.stdout: xmlstarlet x86_64 1.6.1-20.el8 appstream 71 k 2024-04-17T22:26:53.864 INFO:teuthology.orchestra.run.smithi102.stdout: zlib-devel x86_64 1.2.11-21.el8 baseos 58 k 2024-04-17T22:26:53.864 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-17T22:26:53.864 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction Summary 2024-04-17T22:26:53.864 INFO:teuthology.orchestra.run.smithi102.stdout:============================================================================================ 2024-04-17T22:26:53.864 INFO:teuthology.orchestra.run.smithi102.stdout:Install 102 Packages 2024-04-17T22:26:53.864 INFO:teuthology.orchestra.run.smithi102.stdout:Upgrade 19 Packages 2024-04-17T22:26:53.864 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-17T22:26:53.865 INFO:teuthology.orchestra.run.smithi102.stdout:Total download size: 166 M 2024-04-17T22:26:53.865 INFO:teuthology.orchestra.run.smithi102.stdout:Downloading Packages: 2024-04-17T22:26:53.899 INFO:teuthology.orchestra.run.smithi190.stdout:(3/121): ceph-mds-14.2.22-0.el8.x86_64.rpm 34 MB/s | 2.0 MB 00:00 2024-04-17T22:26:53.975 INFO:teuthology.orchestra.run.smithi190.stdout:(4/121): ceph-base-14.2.22-0.el8.x86_64.rpm 23 MB/s | 5.4 MB 00:00 2024-04-17T22:26:54.016 INFO:teuthology.orchestra.run.smithi190.stdout:(5/121): ceph-mgr-14.2.22-0.el8.x86_64.rpm 13 MB/s | 1.6 MB 00:00 2024-04-17T22:26:54.108 INFO:teuthology.orchestra.run.smithi190.stdout:(6/121): ceph-mon-14.2.22-0.el8.x86_64.rpm 33 MB/s | 4.5 MB 00:00 2024-04-17T22:26:54.234 INFO:teuthology.orchestra.run.smithi190.stdout:(7/121): ceph-radosgw-14.2.22-0.el8.x86_64.rpm 48 MB/s | 6.0 MB 00:00 2024-04-17T22:26:54.259 INFO:teuthology.orchestra.run.smithi190.stdout:(8/121): ceph-selinux-14.2.22-0.el8.x86_64.rpm 1.0 MB/s | 25 kB 00:00 2024-04-17T22:26:54.417 INFO:teuthology.orchestra.run.smithi102.stdout:(1/121): ceph-14.2.22-0.el8.x86_64.rpm 147 kB/s | 6.3 kB 00:00 2024-04-17T22:26:54.459 INFO:teuthology.orchestra.run.smithi190.stdout:(9/121): ceph-common-14.2.22-0.el8.x86_64.rpm 27 MB/s | 19 MB 00:00 2024-04-17T22:26:54.484 INFO:teuthology.orchestra.run.smithi102.stdout:(2/121): ceph-fuse-14.2.22-0.el8.x86_64.rpm 8.6 MB/s | 590 kB 00:00 2024-04-17T22:26:54.485 INFO:teuthology.orchestra.run.smithi190.stdout:(10/121): libcephfs-devel-14.2.22-0.el8.x86_64. 868 kB/s | 22 kB 00:00 2024-04-17T22:26:54.526 INFO:teuthology.orchestra.run.smithi190.stdout:(11/121): libcephfs2-14.2.22-0.el8.x86_64.rpm 12 MB/s | 520 kB 00:00 2024-04-17T22:26:54.559 INFO:teuthology.orchestra.run.smithi102.stdout:(3/121): ceph-mds-14.2.22-0.el8.x86_64.rpm 26 MB/s | 2.0 MB 00:00 2024-04-17T22:26:54.561 INFO:teuthology.orchestra.run.smithi190.stdout:(12/121): librados-devel-14.2.22-0.el8.x86_64.r 2.7 MB/s | 90 kB 00:00 2024-04-17T22:26:54.594 INFO:teuthology.orchestra.run.smithi190.stdout:(13/121): libradosstriper1-14.2.22-0.el8.x86_64 11 MB/s | 376 kB 00:00 2024-04-17T22:26:54.635 INFO:teuthology.orchestra.run.smithi102.stdout:(4/121): ceph-base-14.2.22-0.el8.x86_64.rpm 21 MB/s | 5.4 MB 00:00 2024-04-17T22:26:54.677 INFO:teuthology.orchestra.run.smithi102.stdout:(5/121): ceph-mgr-14.2.22-0.el8.x86_64.rpm 13 MB/s | 1.6 MB 00:00 2024-04-17T22:26:54.760 INFO:teuthology.orchestra.run.smithi102.stdout:(6/121): ceph-mon-14.2.22-0.el8.x86_64.rpm 36 MB/s | 4.5 MB 00:00 2024-04-17T22:26:54.782 INFO:teuthology.orchestra.run.smithi190.stdout:(14/121): ceph-osd-14.2.22-0.el8.x86_64.rpm 22 MB/s | 17 MB 00:00 2024-04-17T22:26:54.812 INFO:teuthology.orchestra.run.smithi190.stdout:(15/121): python3-ceph-argparse-14.2.22-0.el8.x 1.3 MB/s | 39 kB 00:00 2024-04-17T22:26:54.836 INFO:teuthology.orchestra.run.smithi190.stdout:(16/121): python3-cephfs-14.2.22-0.el8.x86_64.r 6.2 MB/s | 159 kB 00:00 2024-04-17T22:26:54.862 INFO:teuthology.orchestra.run.smithi190.stdout:(17/121): python3-rados-14.2.22-0.el8.x86_64.rp 13 MB/s | 323 kB 00:00 2024-04-17T22:26:54.887 INFO:teuthology.orchestra.run.smithi190.stdout:(18/121): python3-rbd-14.2.22-0.el8.x86_64.rpm 11 MB/s | 291 kB 00:00 2024-04-17T22:26:54.903 INFO:teuthology.orchestra.run.smithi102.stdout:(7/121): ceph-radosgw-14.2.22-0.el8.x86_64.rpm 42 MB/s | 6.0 MB 00:00 2024-04-17T22:26:54.912 INFO:teuthology.orchestra.run.smithi190.stdout:(19/121): python3-rgw-14.2.22-0.el8.x86_64.rpm 4.4 MB/s | 113 kB 00:00 2024-04-17T22:26:54.928 INFO:teuthology.orchestra.run.smithi102.stdout:(8/121): ceph-selinux-14.2.22-0.el8.x86_64.rpm 1.0 MB/s | 25 kB 00:00 2024-04-17T22:26:54.938 INFO:teuthology.orchestra.run.smithi190.stdout:(20/121): rbd-fuse-14.2.22-0.el8.x86_64.rpm 2.9 MB/s | 73 kB 00:00 2024-04-17T22:26:55.038 INFO:teuthology.orchestra.run.smithi190.stdout:(21/121): librgw2-14.2.22-0.el8.x86_64.rpm 12 MB/s | 5.3 MB 00:00 2024-04-17T22:26:55.063 INFO:teuthology.orchestra.run.smithi190.stdout:(22/121): rbd-nbd-14.2.22-0.el8.x86_64.rpm 5.6 MB/s | 142 kB 00:00 2024-04-17T22:26:55.080 INFO:teuthology.orchestra.run.smithi190.stdout:(23/121): ceph-grafana-dashboards-14.2.22-0.el8 1.4 MB/s | 24 kB 00:00 2024-04-17T22:26:55.128 INFO:teuthology.orchestra.run.smithi102.stdout:(9/121): ceph-common-14.2.22-0.el8.x86_64.rpm 26 MB/s | 19 MB 00:00 2024-04-17T22:26:55.153 INFO:teuthology.orchestra.run.smithi102.stdout:(10/121): libcephfs-devel-14.2.22-0.el8.x86_64. 873 kB/s | 22 kB 00:00 2024-04-17T22:26:55.184 INFO:teuthology.orchestra.run.smithi190.stdout:(24/121): rbd-mirror-14.2.22-0.el8.x86_64.rpm 9.4 MB/s | 2.3 MB 00:00 2024-04-17T22:26:55.187 INFO:teuthology.orchestra.run.smithi102.stdout:(11/121): libcephfs2-14.2.22-0.el8.x86_64.rpm 15 MB/s | 520 kB 00:00 2024-04-17T22:26:55.212 INFO:teuthology.orchestra.run.smithi102.stdout:(12/121): librados-devel-14.2.22-0.el8.x86_64.r 3.5 MB/s | 90 kB 00:00 2024-04-17T22:26:55.246 INFO:teuthology.orchestra.run.smithi102.stdout:(13/121): libradosstriper1-14.2.22-0.el8.x86_64 11 MB/s | 376 kB 00:00 2024-04-17T22:26:55.364 INFO:teuthology.orchestra.run.smithi190.stdout:(25/121): ceph-mgr-diskprediction-local-14.2.22 6.2 MB/s | 1.1 MB 00:00 2024-04-17T22:26:55.430 INFO:teuthology.orchestra.run.smithi102.stdout:(14/121): ceph-osd-14.2.22-0.el8.x86_64.rpm 22 MB/s | 17 MB 00:00 2024-04-17T22:26:55.455 INFO:teuthology.orchestra.run.smithi102.stdout:(15/121): python3-ceph-argparse-14.2.22-0.el8.x 1.6 MB/s | 39 kB 00:00 2024-04-17T22:26:55.480 INFO:teuthology.orchestra.run.smithi102.stdout:(16/121): python3-cephfs-14.2.22-0.el8.x86_64.r 6.2 MB/s | 159 kB 00:00 2024-04-17T22:26:55.505 INFO:teuthology.orchestra.run.smithi102.stdout:(17/121): python3-rados-14.2.22-0.el8.x86_64.rp 13 MB/s | 323 kB 00:00 2024-04-17T22:26:55.525 INFO:teuthology.orchestra.run.smithi190.stdout:(26/121): ceph-mgr-rook-14.2.22-0.el8.noarch.rp 95 kB/s | 15 kB 00:00 2024-04-17T22:26:55.539 INFO:teuthology.orchestra.run.smithi102.stdout:(18/121): python3-rbd-14.2.22-0.el8.x86_64.rpm 8.5 MB/s | 291 kB 00:00 2024-04-17T22:26:55.582 INFO:teuthology.orchestra.run.smithi190.stdout:(27/121): ceph-mgr-dashboard-14.2.22-0.el8.noar 7.5 MB/s | 3.7 MB 00:00 2024-04-17T22:26:55.592 INFO:teuthology.orchestra.run.smithi102.stdout:(19/121): python3-rgw-14.2.22-0.el8.x86_64.rpm 2.1 MB/s | 113 kB 00:00 2024-04-17T22:26:55.898 INFO:teuthology.orchestra.run.smithi102.stdout:(20/121): librgw2-14.2.22-0.el8.x86_64.rpm 8.1 MB/s | 5.3 MB 00:00 2024-04-17T22:26:55.940 INFO:teuthology.orchestra.run.smithi102.stdout:(21/121): rbd-fuse-14.2.22-0.el8.x86_64.rpm 210 kB/s | 73 kB 00:00 2024-04-17T22:26:55.948 INFO:teuthology.orchestra.run.smithi190.stdout:(28/121): ceph-test-14.2.22-0.el8.x86_64.rpm 21 MB/s | 35 MB 00:01 2024-04-17T22:26:55.965 INFO:teuthology.orchestra.run.smithi102.stdout:(22/121): rbd-nbd-14.2.22-0.el8.x86_64.rpm 5.6 MB/s | 142 kB 00:00 2024-04-17T22:26:55.990 INFO:teuthology.orchestra.run.smithi102.stdout:(23/121): ceph-grafana-dashboards-14.2.22-0.el8 963 kB/s | 24 kB 00:00 2024-04-17T22:26:56.057 INFO:teuthology.orchestra.run.smithi102.stdout:(24/121): rbd-mirror-14.2.22-0.el8.x86_64.rpm 15 MB/s | 2.3 MB 00:00 2024-04-17T22:26:56.065 INFO:teuthology.orchestra.run.smithi190.stdout:(29/121): flex-2.6.1-9.el8.x86_64.rpm 661 kB/s | 320 kB 00:00 2024-04-17T22:26:56.116 INFO:teuthology.orchestra.run.smithi102.stdout:(25/121): ceph-mgr-diskprediction-local-14.2.22 19 MB/s | 1.1 MB 00:00 2024-04-17T22:26:56.141 INFO:teuthology.orchestra.run.smithi102.stdout:(26/121): ceph-mgr-rook-14.2.22-0.el8.noarch.rp 616 kB/s | 15 kB 00:00 2024-04-17T22:26:56.200 INFO:teuthology.orchestra.run.smithi102.stdout:(27/121): ceph-mgr-dashboard-14.2.22-0.el8.noar 18 MB/s | 3.7 MB 00:00 2024-04-17T22:26:56.216 INFO:teuthology.orchestra.run.smithi190.stdout:(30/121): bison-3.0.4-10.el8.x86_64.rpm 998 kB/s | 688 kB 00:00 2024-04-17T22:26:56.249 INFO:teuthology.orchestra.run.smithi190.stdout:(31/121): librdkafka-1.6.1-1.el8.x86_64.rpm 3.6 MB/s | 677 kB 00:00 2024-04-17T22:26:56.274 INFO:teuthology.orchestra.run.smithi190.stdout:(32/121): jq-1.6-8.el8.x86_64.rpm 624 kB/s | 203 kB 00:00 2024-04-17T22:26:56.333 INFO:teuthology.orchestra.run.smithi190.stdout:(33/121): oniguruma-6.8.2-3.el8.x86_64.rpm 2.2 MB/s | 188 kB 00:00 2024-04-17T22:26:56.375 INFO:teuthology.orchestra.run.smithi190.stdout:(34/121): lttng-ust-2.8.1-11.el8.x86_64.rpm 1.6 MB/s | 259 kB 00:00 2024-04-17T22:26:56.500 INFO:teuthology.orchestra.run.smithi190.stdout:(35/121): python3-mako-1.0.6-14.el8.noarch.rpm 1.2 MB/s | 157 kB 00:00 2024-04-17T22:26:56.609 INFO:teuthology.orchestra.run.smithi102.stdout:(28/121): ceph-test-14.2.22-0.el8.x86_64.rpm 21 MB/s | 35 MB 00:01 2024-04-17T22:26:56.626 INFO:teuthology.orchestra.run.smithi190.stdout:(36/121): openblas-threads-0.3.15-4.el8.x86_64. 17 MB/s | 4.9 MB 00:00 2024-04-17T22:26:56.651 INFO:teuthology.orchestra.run.smithi190.stdout:(37/121): python3-markupsafe-0.23-19.el8.x86_64 258 kB/s | 39 kB 00:00 2024-04-17T22:26:56.709 INFO:teuthology.orchestra.run.smithi102.stdout:(29/121): flex-2.6.1-9.el8.x86_64.rpm 629 kB/s | 320 kB 00:00 2024-04-17T22:26:56.718 INFO:teuthology.orchestra.run.smithi190.stdout:(38/121): openblas-0.3.15-4.el8.x86_64.rpm 11 MB/s | 4.7 MB 00:00 2024-04-17T22:26:56.793 INFO:teuthology.orchestra.run.smithi190.stdout:(39/121): python3-numpy-f2py-1.14.3-10.el8.x86_ 1.5 MB/s | 225 kB 00:00 2024-04-17T22:26:56.818 INFO:teuthology.orchestra.run.smithi190.stdout:(40/121): python3-prettytable-0.7.2-14.el8.noar 439 kB/s | 44 kB 00:00 2024-04-17T22:26:56.869 INFO:teuthology.orchestra.run.smithi190.stdout:(41/121): python3-numpy-1.14.3-10.el8.x86_64.rp 15 MB/s | 3.7 MB 00:00 2024-04-17T22:26:56.876 INFO:teuthology.orchestra.run.smithi102.stdout:(30/121): bison-3.0.4-10.el8.x86_64.rpm 938 kB/s | 688 kB 00:00 2024-04-17T22:26:56.885 INFO:teuthology.orchestra.run.smithi190.stdout:(42/121): python3-pyasn1-0.3.7-6.el8.noarch.rpm 1.8 MB/s | 126 kB 00:00 2024-04-17T22:26:56.919 INFO:teuthology.orchestra.run.smithi190.stdout:(43/121): python3-pyOpenSSL-19.0.0-1.el8.noarch 823 kB/s | 103 kB 00:00 2024-04-17T22:26:56.935 INFO:teuthology.orchestra.run.smithi102.stdout:(31/121): jq-1.6-8.el8.x86_64.rpm 624 kB/s | 203 kB 00:00 2024-04-17T22:26:56.961 INFO:teuthology.orchestra.run.smithi190.stdout:(44/121): python3-pyasn1-modules-0.3.7-6.el8.no 1.2 MB/s | 110 kB 00:00 2024-04-17T22:26:56.987 INFO:teuthology.orchestra.run.smithi190.stdout:(45/121): python3-pytz-2017.2-11.el8.noarch.rpm 567 kB/s | 57 kB 00:00 2024-04-17T22:26:57.035 INFO:teuthology.orchestra.run.smithi102.stdout:(32/121): lttng-ust-2.8.1-11.el8.x86_64.rpm 1.6 MB/s | 259 kB 00:00 2024-04-17T22:26:57.045 INFO:teuthology.orchestra.run.smithi190.stdout:(46/121): python3-werkzeug-0.12.2-4.el8.noarch. 5.4 MB/s | 457 kB 00:00 2024-04-17T22:26:57.060 INFO:teuthology.orchestra.run.smithi102.stdout:(33/121): oniguruma-6.8.2-3.el8.x86_64.rpm 1.5 MB/s | 188 kB 00:00 2024-04-17T22:26:57.120 INFO:teuthology.orchestra.run.smithi190.stdout:(47/121): xmlstarlet-1.6.1-20.el8.x86_64.rpm 944 kB/s | 71 kB 00:00 2024-04-17T22:26:57.146 INFO:teuthology.orchestra.run.smithi190.stdout:(48/121): socat-1.7.4.1-1.el8.x86_64.rpm 2.0 MB/s | 323 kB 00:00 2024-04-17T22:26:57.152 INFO:teuthology.orchestra.run.smithi102.stdout:(34/121): librdkafka-1.6.1-1.el8.x86_64.rpm 1.5 MB/s | 677 kB 00:00 2024-04-17T22:26:57.286 INFO:teuthology.orchestra.run.smithi102.stdout:(35/121): python3-mako-1.0.6-14.el8.noarch.rpm 1.1 MB/s | 157 kB 00:00 2024-04-17T22:26:57.430 INFO:teuthology.orchestra.run.smithi190.stdout:(49/121): python3-scipy-1.0.0-21.module_el8+762 28 MB/s | 14 MB 00:00 2024-04-17T22:26:57.445 INFO:teuthology.orchestra.run.smithi102.stdout:(36/121): openblas-threads-0.3.15-4.el8.x86_64. 13 MB/s | 4.9 MB 00:00 2024-04-17T22:26:57.446 INFO:teuthology.orchestra.run.smithi190.stdout:(50/121): elfutils-libelf-devel-0.190-2.el8.x86 191 kB/s | 62 kB 00:00 2024-04-17T22:26:57.470 INFO:teuthology.orchestra.run.smithi102.stdout:(37/121): python3-markupsafe-0.23-19.el8.x86_64 211 kB/s | 39 kB 00:00 2024-04-17T22:26:57.472 INFO:teuthology.orchestra.run.smithi190.stdout:(51/121): fuse-2.9.7-19.el8.x86_64.rpm 256 kB/s | 83 kB 00:00 2024-04-17T22:26:57.497 INFO:teuthology.orchestra.run.smithi190.stdout:(52/121): keyutils-libs-devel-1.5.10-9.el8.x86_ 718 kB/s | 48 kB 00:00 2024-04-17T22:26:57.531 INFO:teuthology.orchestra.run.smithi190.stdout:(53/121): libcom_err-devel-1.45.6-5.el8.x86_64. 663 kB/s | 39 kB 00:00 2024-04-17T22:26:57.647 INFO:teuthology.orchestra.run.smithi190.stdout:(54/121): libquadmath-8.5.0-21.el8.x86_64.rpm 1.4 MB/s | 172 kB 00:00 2024-04-17T22:26:57.653 INFO:teuthology.orchestra.run.smithi102.stdout:(38/121): python3-numpy-1.14.3-10.el8.x86_64.rp 18 MB/s | 3.7 MB 00:00 2024-04-17T22:26:57.673 INFO:teuthology.orchestra.run.smithi190.stdout:(55/121): libgfortran-8.5.0-21.el8.x86_64.rpm 3.6 MB/s | 645 kB 00:00 2024-04-17T22:26:57.679 INFO:teuthology.orchestra.run.smithi102.stdout:(39/121): python3-numpy-f2py-1.14.3-10.el8.x86_ 1.1 MB/s | 225 kB 00:00 2024-04-17T22:26:57.698 INFO:teuthology.orchestra.run.smithi190.stdout:(56/121): librabbitmq-0.9.0-5.el8.x86_64.rpm 937 kB/s | 47 kB 00:00 2024-04-17T22:26:57.737 INFO:teuthology.orchestra.run.smithi102.stdout:(40/121): python3-prettytable-0.7.2-14.el8.noar 526 kB/s | 44 kB 00:00 2024-04-17T22:26:57.740 INFO:teuthology.orchestra.run.smithi190.stdout:(57/121): krb5-devel-1.18.2-22.el8.x86_64.rpm 1.9 MB/s | 561 kB 00:00 2024-04-17T22:26:57.765 INFO:teuthology.orchestra.run.smithi190.stdout:(58/121): libselinux-devel-2.9-8.el8.x86_64.rpm 2.1 MB/s | 200 kB 00:00 2024-04-17T22:26:57.786 INFO:teuthology.orchestra.run.smithi102.stdout:(41/121): python3-pyOpenSSL-19.0.0-1.el8.noarch 964 kB/s | 103 kB 00:00 2024-04-17T22:26:57.790 INFO:teuthology.orchestra.run.smithi190.stdout:(59/121): libsepol-devel-2.9-3.el8.x86_64.rpm 945 kB/s | 87 kB 00:00 2024-04-17T22:26:57.816 INFO:teuthology.orchestra.run.smithi190.stdout:(60/121): libverto-devel-0.3.2-2.el8.x86_64.rpm 239 kB/s | 18 kB 00:00 2024-04-17T22:26:57.841 INFO:teuthology.orchestra.run.smithi190.stdout:(61/121): libzstd-devel-1.4.4-1.el8.x86_64.rpm 580 kB/s | 44 kB 00:00 2024-04-17T22:26:57.846 INFO:teuthology.orchestra.run.smithi102.stdout:(42/121): python3-pyasn1-0.3.7-6.el8.noarch.rpm 1.1 MB/s | 126 kB 00:00 2024-04-17T22:26:57.871 INFO:teuthology.orchestra.run.smithi102.stdout:(43/121): python3-pyasn1-modules-0.3.7-6.el8.no 1.3 MB/s | 110 kB 00:00 2024-04-17T22:26:57.908 INFO:teuthology.orchestra.run.smithi190.stdout:(62/121): pcre2-utf16-10.32-3.el8.x86_64.rpm 3.4 MB/s | 229 kB 00:00 2024-04-17T22:26:57.933 INFO:teuthology.orchestra.run.smithi190.stdout:(63/121): pcre2-devel-10.32-3.el8.x86_64.rpm 5.0 MB/s | 605 kB 00:00 2024-04-17T22:26:57.947 INFO:teuthology.orchestra.run.smithi102.stdout:(44/121): openblas-0.3.15-4.el8.x86_64.rpm 5.2 MB/s | 4.7 MB 00:00 2024-04-17T22:26:57.972 INFO:teuthology.orchestra.run.smithi102.stdout:(45/121): python3-pytz-2017.2-11.el8.noarch.rpm 454 kB/s | 57 kB 00:00 2024-04-17T22:26:57.975 INFO:teuthology.orchestra.run.smithi190.stdout:(64/121): pcre2-utf32-10.32-3.el8.x86_64.rpm 3.2 MB/s | 220 kB 00:00 2024-04-17T22:26:58.000 INFO:teuthology.orchestra.run.smithi190.stdout:(65/121): python3-cffi-1.11.5-6.el8.x86_64.rpm 3.6 MB/s | 249 kB 00:00 2024-04-17T22:26:58.075 INFO:teuthology.orchestra.run.smithi190.stdout:(66/121): openssl-devel-1.1.1k-12.el8.x86_64.rp 11 MB/s | 3.2 MB 00:00 2024-04-17T22:26:58.097 INFO:teuthology.orchestra.run.smithi102.stdout:(46/121): python3-werkzeug-0.12.2-4.el8.noarch. 3.0 MB/s | 457 kB 00:00 2024-04-17T22:26:58.101 INFO:teuthology.orchestra.run.smithi190.stdout:(67/121): python3-jwt-1.6.1-2.el8.noarch.rpm 425 kB/s | 43 kB 00:00 2024-04-17T22:26:58.126 INFO:teuthology.orchestra.run.smithi190.stdout:(68/121): python3-cryptography-3.2.1-7.el8.x86_ 4.2 MB/s | 641 kB 00:00 2024-04-17T22:26:58.148 INFO:teuthology.orchestra.run.smithi102.stdout:(47/121): socat-1.7.4.1-1.el8.x86_64.rpm 1.8 MB/s | 323 kB 00:00 2024-04-17T22:26:58.151 INFO:teuthology.orchestra.run.smithi190.stdout:(69/121): python3-oauthlib-2.1.0-1.el8.noarch.r 2.0 MB/s | 155 kB 00:00 2024-04-17T22:26:58.176 INFO:teuthology.orchestra.run.smithi190.stdout:(70/121): python3-pycparser-2.14-14.el8.noarch. 1.4 MB/s | 109 kB 00:00 2024-04-17T22:26:58.202 INFO:teuthology.orchestra.run.smithi190.stdout:(71/121): python3-requests-oauthlib-1.0.0-1.el8 565 kB/s | 43 kB 00:00 2024-04-17T22:26:58.206 INFO:teuthology.orchestra.run.smithi102.stdout:(48/121): xmlstarlet-1.6.1-20.el8.x86_64.rpm 651 kB/s | 71 kB 00:00 2024-04-17T22:26:58.227 INFO:teuthology.orchestra.run.smithi190.stdout:(72/121): zlib-devel-1.2.11-21.el8.x86_64.rpm 768 kB/s | 58 kB 00:00 2024-04-17T22:26:58.407 INFO:teuthology.orchestra.run.smithi102.stdout:(49/121): python3-scipy-1.0.0-21.module_el8+762 26 MB/s | 14 MB 00:00 2024-04-17T22:26:58.432 INFO:teuthology.orchestra.run.smithi102.stdout:(50/121): elfutils-libelf-devel-0.190-2.el8.x86 219 kB/s | 62 kB 00:00 2024-04-17T22:26:58.449 INFO:teuthology.orchestra.run.smithi102.stdout:(51/121): fuse-2.9.7-19.el8.x86_64.rpm 343 kB/s | 83 kB 00:00 2024-04-17T22:26:58.516 INFO:teuthology.orchestra.run.smithi102.stdout:(52/121): libcom_err-devel-1.45.6-5.el8.x86_64. 583 kB/s | 39 kB 00:00 2024-04-17T22:26:58.577 INFO:teuthology.orchestra.run.smithi190.stdout:(73/121): liboath-2.6.2-3.el8.x86_64.rpm 168 kB/s | 59 kB 00:00 2024-04-17T22:26:58.599 INFO:teuthology.orchestra.run.smithi102.stdout:(53/121): keyutils-libs-devel-1.5.10-9.el8.x86_ 250 kB/s | 48 kB 00:00 2024-04-17T22:26:58.625 INFO:teuthology.orchestra.run.smithi102.stdout:(54/121): krb5-devel-1.18.2-22.el8.x86_64.rpm 2.8 MB/s | 561 kB 00:00 2024-04-17T22:26:58.627 INFO:teuthology.orchestra.run.smithi190.stdout:(74/121): leveldb-1.22-1.el8.x86_64.rpm 425 kB/s | 181 kB 00:00 2024-04-17T22:26:58.683 INFO:teuthology.orchestra.run.smithi102.stdout:(55/121): librabbitmq-0.9.0-5.el8.x86_64.rpm 806 kB/s | 47 kB 00:00 2024-04-17T22:26:58.708 INFO:teuthology.orchestra.run.smithi102.stdout:(56/121): libgfortran-8.5.0-21.el8.x86_64.rpm 3.3 MB/s | 645 kB 00:00 2024-04-17T22:26:58.711 INFO:teuthology.orchestra.run.smithi190.stdout:(75/121): gperftools-libs-2.7-9.el8.x86_64.rpm 573 kB/s | 306 kB 00:00 2024-04-17T22:26:58.734 INFO:teuthology.orchestra.run.smithi102.stdout:(57/121): libquadmath-8.5.0-21.el8.x86_64.rpm 1.3 MB/s | 172 kB 00:00 2024-04-17T22:26:58.768 INFO:teuthology.orchestra.run.smithi102.stdout:(58/121): libselinux-devel-2.9-8.el8.x86_64.rpm 2.3 MB/s | 200 kB 00:00 2024-04-17T22:26:58.793 INFO:teuthology.orchestra.run.smithi102.stdout:(59/121): libverto-devel-0.3.2-2.el8.x86_64.rpm 306 kB/s | 18 kB 00:00 2024-04-17T22:26:58.810 INFO:teuthology.orchestra.run.smithi102.stdout:(60/121): libsepol-devel-2.9-3.el8.x86_64.rpm 860 kB/s | 87 kB 00:00 2024-04-17T22:26:58.835 INFO:teuthology.orchestra.run.smithi102.stdout:(61/121): libzstd-devel-1.4.4-1.el8.x86_64.rpm 651 kB/s | 44 kB 00:00 2024-04-17T22:26:58.894 INFO:teuthology.orchestra.run.smithi102.stdout:(62/121): pcre2-devel-10.32-3.el8.x86_64.rpm 7.1 MB/s | 605 kB 00:00 2024-04-17T22:26:58.919 INFO:teuthology.orchestra.run.smithi102.stdout:(63/121): pcre2-utf16-10.32-3.el8.x86_64.rpm 2.7 MB/s | 229 kB 00:00 2024-04-17T22:26:58.920 INFO:teuthology.orchestra.run.smithi190.stdout:(76/121): libunwind-1.3.1-3.el8.x86_64.rpm 219 kB/s | 75 kB 00:00 2024-04-17T22:26:58.979 INFO:teuthology.orchestra.run.smithi190.stdout:(77/121): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 124 kB/s | 44 kB 00:00 2024-04-17T22:26:58.986 INFO:teuthology.orchestra.run.smithi102.stdout:(64/121): pcre2-utf32-10.32-3.el8.x86_64.rpm 2.3 MB/s | 220 kB 00:00 2024-04-17T22:26:59.011 INFO:teuthology.orchestra.run.smithi102.stdout:(65/121): python3-cffi-1.11.5-6.el8.x86_64.rpm 2.6 MB/s | 249 kB 00:00 2024-04-17T22:26:59.078 INFO:teuthology.orchestra.run.smithi102.stdout:(66/121): openssl-devel-1.1.1k-12.el8.x86_64.rp 11 MB/s | 3.2 MB 00:00 2024-04-17T22:26:59.098 INFO:teuthology.orchestra.run.smithi102.stdout:(67/121): python3-jwt-1.6.1-2.el8.noarch.rpm 510 kB/s | 43 kB 00:00 2024-04-17T22:26:59.129 INFO:teuthology.orchestra.run.smithi102.stdout:(68/121): python3-cryptography-3.2.1-7.el8.x86_ 4.4 MB/s | 641 kB 00:00 2024-04-17T22:26:59.137 INFO:teuthology.orchestra.run.smithi190.stdout:(78/121): python3-beautifulsoup4-4.6.3-2.el8.1. 435 kB/s | 185 kB 00:00 2024-04-17T22:26:59.154 INFO:teuthology.orchestra.run.smithi102.stdout:(69/121): python3-oauthlib-2.1.0-1.el8.noarch.r 2.0 MB/s | 155 kB 00:00 2024-04-17T22:26:59.179 INFO:teuthology.orchestra.run.smithi102.stdout:(70/121): python3-pycparser-2.14-14.el8.noarch. 1.3 MB/s | 109 kB 00:00 2024-04-17T22:26:59.196 INFO:teuthology.orchestra.run.smithi190.stdout:(79/121): python3-cachetools-3.1.1-4.el8.noarch 120 kB/s | 33 kB 00:00 2024-04-17T22:26:59.204 INFO:teuthology.orchestra.run.smithi102.stdout:(71/121): python3-requests-oauthlib-1.0.0-1.el8 564 kB/s | 43 kB 00:00 2024-04-17T22:26:59.230 INFO:teuthology.orchestra.run.smithi102.stdout:(72/121): zlib-devel-1.2.11-21.el8.x86_64.rpm 770 kB/s | 58 kB 00:00 2024-04-17T22:26:59.246 INFO:teuthology.orchestra.run.smithi190.stdout:(80/121): python3-certifi-2018.10.15-7.el8.noar 61 kB/s | 16 kB 00:00 2024-04-17T22:26:59.563 INFO:teuthology.orchestra.run.smithi190.stdout:(81/121): python3-cheroot-8.5.2-1.el8.noarch.rp 405 kB/s | 173 kB 00:00 2024-04-17T22:26:59.588 INFO:teuthology.orchestra.run.smithi190.stdout:(82/121): python3-google-auth-1.1.1-10.el8.noar 238 kB/s | 81 kB 00:00 2024-04-17T22:26:59.705 INFO:teuthology.orchestra.run.smithi190.stdout:(83/121): python3-cherrypy-18.4.0-1.el8.noarch. 755 kB/s | 384 kB 00:00 2024-04-17T22:26:59.747 INFO:teuthology.orchestra.run.smithi190.stdout:(84/121): python3-jaraco-6.2-6.el8.noarch.rpm 60 kB/s | 11 kB 00:00 2024-04-17T22:26:59.769 INFO:teuthology.orchestra.run.smithi102.stdout:(73/121): liboath-2.6.2-3.el8.x86_64.rpm 109 kB/s | 59 kB 00:00 2024-04-17T22:26:59.856 INFO:teuthology.orchestra.run.smithi190.stdout:(85/121): python3-jaraco-functools-2.0-4.el8.no 66 kB/s | 18 kB 00:00 2024-04-17T22:26:59.880 INFO:teuthology.orchestra.run.smithi102.stdout:(74/121): leveldb-1.22-1.el8.x86_64.rpm 268 kB/s | 181 kB 00:00 2024-04-17T22:26:59.947 INFO:teuthology.orchestra.run.smithi102.stdout:(75/121): libunwind-1.3.1-3.el8.x86_64.rpm 423 kB/s | 75 kB 00:00 2024-04-17T22:26:59.981 INFO:teuthology.orchestra.run.smithi102.stdout:(76/121): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 435 kB/s | 44 kB 00:00 2024-04-17T22:27:00.031 INFO:teuthology.orchestra.run.smithi102.stdout:(77/121): gperftools-libs-2.7-9.el8.x86_64.rpm 359 kB/s | 306 kB 00:00 2024-04-17T22:27:00.081 INFO:teuthology.orchestra.run.smithi102.stdout:(78/121): python3-cachetools-3.1.1-4.el8.noarch 329 kB/s | 33 kB 00:00 2024-04-17T22:27:00.098 INFO:teuthology.orchestra.run.smithi190.stdout:(86/121): python3-logutils-0.3.5-11.el8.noarch. 139 kB/s | 49 kB 00:00 2024-04-17T22:27:00.164 INFO:teuthology.orchestra.run.smithi102.stdout:(79/121): python3-certifi-2018.10.15-7.el8.noar 123 kB/s | 16 kB 00:00 2024-04-17T22:27:00.206 INFO:teuthology.orchestra.run.smithi190.stdout:(87/121): python3-more-itertools-7.2.0-3.el8.no 168 kB/s | 59 kB 00:00 2024-04-17T22:27:00.215 INFO:teuthology.orchestra.run.smithi102.stdout:(80/121): python3-beautifulsoup4-4.6.3-2.el8.1. 692 kB/s | 185 kB 00:00 2024-04-17T22:27:00.349 INFO:teuthology.orchestra.run.smithi102.stdout:(81/121): python3-cheroot-8.5.2-1.el8.noarch.rp 645 kB/s | 173 kB 00:00 2024-04-17T22:27:00.399 INFO:teuthology.orchestra.run.smithi102.stdout:(82/121): python3-google-auth-1.1.1-10.el8.noar 442 kB/s | 81 kB 00:00 2024-04-17T22:27:00.407 INFO:teuthology.orchestra.run.smithi190.stdout:(88/121): python3-kubernetes-11.0.0-6.el8.noarc 1.7 MB/s | 1.2 MB 00:00 2024-04-17T22:27:00.449 INFO:teuthology.orchestra.run.smithi102.stdout:(83/121): python3-jaraco-6.2-6.el8.noarch.rpm 111 kB/s | 11 kB 00:00 2024-04-17T22:27:00.473 INFO:teuthology.orchestra.run.smithi190.stdout:(89/121): python3-portend-2.6-1.el8.noarch.rpm 58 kB/s | 16 kB 00:00 2024-04-17T22:27:00.500 INFO:teuthology.orchestra.run.smithi102.stdout:(84/121): python3-jaraco-functools-2.0-4.el8.no 175 kB/s | 18 kB 00:00 2024-04-17T22:27:00.608 INFO:teuthology.orchestra.run.smithi102.stdout:(85/121): python3-logutils-0.3.5-11.el8.noarch. 449 kB/s | 49 kB 00:00 2024-04-17T22:27:00.624 INFO:teuthology.orchestra.run.smithi190.stdout:(90/121): python3-pecan-1.3.2-9.el8.noarch.rpm 538 kB/s | 283 kB 00:00 2024-04-17T22:27:00.659 INFO:teuthology.orchestra.run.smithi102.stdout:(86/121): python3-cherrypy-18.4.0-1.el8.noarch. 778 kB/s | 384 kB 00:00 2024-04-17T22:27:00.674 INFO:teuthology.orchestra.run.smithi190.stdout:(91/121): python3-repoze-lru-0.7-6.el8.noarch.r 126 kB/s | 34 kB 00:00 2024-04-17T22:27:00.792 INFO:teuthology.orchestra.run.smithi102.stdout:(87/121): python3-more-itertools-7.2.0-3.el8.no 320 kB/s | 59 kB 00:00 2024-04-17T22:27:00.891 INFO:teuthology.orchestra.run.smithi190.stdout:(92/121): python3-routes-2.4.1-12.el8.noarch.rp 470 kB/s | 196 kB 00:00 2024-04-17T22:27:00.893 INFO:teuthology.orchestra.run.smithi102.stdout:(88/121): python3-portend-2.6-1.el8.noarch.rpm 156 kB/s | 16 kB 00:00 2024-04-17T22:27:00.933 INFO:teuthology.orchestra.run.smithi190.stdout:(93/121): python3-simplegeneric-0.8.1-17.el8.no 72 kB/s | 19 kB 00:00 2024-04-17T22:27:00.967 INFO:teuthology.orchestra.run.smithi190.stdout:(94/121): python3-rsa-4.9-2.el8.noarch.rpm 187 kB/s | 64 kB 00:00 2024-04-17T22:27:01.001 INFO:teuthology.orchestra.run.smithi102.stdout:(89/121): python3-repoze-lru-0.7-6.el8.noarch.r 309 kB/s | 34 kB 00:00 2024-04-17T22:27:01.102 INFO:teuthology.orchestra.run.smithi102.stdout:(90/121): python3-pecan-1.3.2-9.el8.noarch.rpm 638 kB/s | 283 kB 00:00 2024-04-17T22:27:01.150 INFO:teuthology.orchestra.run.smithi190.stdout:(95/121): python3-singledispatch-3.4.0.3-18.el8 92 kB/s | 24 kB 00:00 2024-04-17T22:27:01.192 INFO:teuthology.orchestra.run.smithi190.stdout:(96/121): python3-tempora-1.14.1-5.el8.noarch.r 113 kB/s | 29 kB 00:00 2024-04-17T22:27:01.251 INFO:teuthology.orchestra.run.smithi190.stdout:(97/121): python3-trustme-0.6.0-4.el8.noarch.rp 97 kB/s | 27 kB 00:00 2024-04-17T22:27:01.286 INFO:teuthology.orchestra.run.smithi102.stdout:(91/121): python3-rsa-4.9-2.el8.noarch.rpm 350 kB/s | 64 kB 00:00 2024-04-17T22:27:01.394 INFO:teuthology.orchestra.run.smithi102.stdout:(92/121): python3-routes-2.4.1-12.el8.noarch.rp 501 kB/s | 196 kB 00:00 2024-04-17T22:27:01.419 INFO:teuthology.orchestra.run.smithi102.stdout:(93/121): python3-simplegeneric-0.8.1-17.el8.no 140 kB/s | 19 kB 00:00 2024-04-17T22:27:01.495 INFO:teuthology.orchestra.run.smithi102.stdout:(94/121): python3-singledispatch-3.4.0.3-18.el8 238 kB/s | 24 kB 00:00 2024-04-17T22:27:01.520 INFO:teuthology.orchestra.run.smithi102.stdout:(95/121): python3-tempora-1.14.1-5.el8.noarch.r 292 kB/s | 29 kB 00:00 2024-04-17T22:27:01.593 INFO:teuthology.orchestra.run.smithi190.stdout:(98/121): python3-websocket-client-0.56.0-5.el8 179 kB/s | 61 kB 00:00 2024-04-17T22:27:01.595 INFO:teuthology.orchestra.run.smithi102.stdout:(96/121): python3-trustme-0.6.0-4.el8.noarch.rp 274 kB/s | 27 kB 00:00 2024-04-17T22:27:01.676 INFO:teuthology.orchestra.run.smithi190.stdout:(99/121): python3-waitress-1.4.3-1.el8.noarch.r 486 kB/s | 255 kB 00:00 2024-04-17T22:27:01.702 INFO:teuthology.orchestra.run.smithi190.stdout:(100/121): python3-webob-1.8.5-1.el8.1.noarch.r 493 kB/s | 251 kB 00:00 2024-04-17T22:27:01.769 INFO:teuthology.orchestra.run.smithi190.stdout:(101/121): librados2-14.2.22-0.el8.x86_64.rpm 53 MB/s | 3.5 MB 00:00 2024-04-17T22:27:01.819 INFO:teuthology.orchestra.run.smithi190.stdout:(102/121): librbd1-14.2.22-0.el8.x86_64.rpm 35 MB/s | 1.7 MB 00:00 2024-04-17T22:27:01.862 INFO:teuthology.orchestra.run.smithi102.stdout:(97/121): python3-waitress-1.4.3-1.el8.noarch.r 746 kB/s | 255 kB 00:00 2024-04-17T22:27:01.938 INFO:teuthology.orchestra.run.smithi102.stdout:(98/121): python3-webob-1.8.5-1.el8.1.noarch.rp 734 kB/s | 251 kB 00:00 2024-04-17T22:27:01.944 INFO:teuthology.orchestra.run.smithi190.stdout:(103/121): python3-webtest-2.0.33-1.el8.noarch. 243 kB/s | 85 kB 00:00 2024-04-17T22:27:01.970 INFO:teuthology.orchestra.run.smithi190.stdout:(104/121): python3-zc-lockfile-2.0-2.el8.noarch 78 kB/s | 23 kB 00:00 2024-04-17T22:27:01.980 INFO:teuthology.orchestra.run.smithi102.stdout:(99/121): python3-kubernetes-11.0.0-6.el8.noarc 780 kB/s | 1.2 MB 00:01 2024-04-17T22:27:02.046 INFO:teuthology.orchestra.run.smithi102.stdout:(100/121): python3-websocket-client-0.56.0-5.el 333 kB/s | 61 kB 00:00 2024-04-17T22:27:02.080 INFO:teuthology.orchestra.run.smithi102.stdout:(101/121): python3-zc-lockfile-2.0-2.el8.noarch 229 kB/s | 23 kB 00:00 2024-04-17T22:27:02.156 INFO:teuthology.orchestra.run.smithi102.stdout:(102/121): librados2-14.2.22-0.el8.x86_64.rpm 32 MB/s | 3.5 MB 00:00 2024-04-17T22:27:02.178 INFO:teuthology.orchestra.run.smithi190.stdout:(105/121): NetworkManager-1.40.16-15.el8.x86_64 6.4 MB/s | 2.3 MB 00:00 2024-04-17T22:27:02.181 INFO:teuthology.orchestra.run.smithi102.stdout:(103/121): python3-webtest-2.0.33-1.el8.noarch. 351 kB/s | 85 kB 00:00 2024-04-17T22:27:02.204 INFO:teuthology.orchestra.run.smithi190.stdout:(106/121): NetworkManager-team-1.40.16-15.el8.x 688 kB/s | 161 kB 00:00 2024-04-17T22:27:02.223 INFO:teuthology.orchestra.run.smithi102.stdout:(104/121): librbd1-14.2.22-0.el8.x86_64.rpm 12 MB/s | 1.7 MB 00:00 2024-04-17T22:27:02.237 INFO:teuthology.orchestra.run.smithi190.stdout:(107/121): NetworkManager-tui-1.40.16-15.el8.x8 5.9 MB/s | 356 kB 00:00 2024-04-17T22:27:02.279 INFO:teuthology.orchestra.run.smithi190.stdout:(108/121): elfutils-debuginfod-client-0.190-2.e 1.0 MB/s | 77 kB 00:00 2024-04-17T22:27:02.304 INFO:teuthology.orchestra.run.smithi190.stdout:(109/121): elfutils-libelf-0.190-2.el8.x86_64.r 3.4 MB/s | 233 kB 00:00 2024-04-17T22:27:02.355 INFO:teuthology.orchestra.run.smithi190.stdout:(110/121): NetworkManager-libnm-1.40.16-15.el8. 4.6 MB/s | 1.9 MB 00:00 2024-04-17T22:27:02.388 INFO:teuthology.orchestra.run.smithi190.stdout:(111/121): iproute-6.2.0-5.el8.x86_64.rpm 10 MB/s | 854 kB 00:00 2024-04-17T22:27:02.414 INFO:teuthology.orchestra.run.smithi190.stdout:(112/121): libblkid-2.32.1-46.el8.x86_64.rpm 3.7 MB/s | 221 kB 00:00 2024-04-17T22:27:02.439 INFO:teuthology.orchestra.run.smithi190.stdout:(113/121): elfutils-libs-0.190-2.el8.x86_64.rpm 1.9 MB/s | 305 kB 00:00 2024-04-17T22:27:02.440 INFO:teuthology.orchestra.run.smithi102.stdout:(105/121): NetworkManager-team-1.40.16-15.el8.x 743 kB/s | 161 kB 00:00 2024-04-17T22:27:02.464 INFO:teuthology.orchestra.run.smithi190.stdout:(114/121): libfdisk-2.32.1-46.el8.x86_64.rpm 3.3 MB/s | 254 kB 00:00 2024-04-17T22:27:02.489 INFO:teuthology.orchestra.run.smithi190.stdout:(115/121): libmount-2.32.1-46.el8.x86_64.rpm 3.1 MB/s | 237 kB 00:00 2024-04-17T22:27:02.548 INFO:teuthology.orchestra.run.smithi190.stdout:(116/121): libsmartcols-2.32.1-46.el8.x86_64.rp 1.6 MB/s | 180 kB 00:00 2024-04-17T22:27:02.565 INFO:teuthology.orchestra.run.smithi102.stdout:(106/121): NetworkManager-1.40.16-15.el8.x86_64 5.6 MB/s | 2.3 MB 00:00 2024-04-17T22:27:02.573 INFO:teuthology.orchestra.run.smithi190.stdout:(117/121): libuuid-2.32.1-46.el8.x86_64.rpm 912 kB/s | 99 kB 00:00 2024-04-17T22:27:02.598 INFO:teuthology.orchestra.run.smithi102.stdout:(107/121): NetworkManager-libnm-1.40.16-15.el8. 4.6 MB/s | 1.9 MB 00:00 2024-04-17T22:27:02.615 INFO:teuthology.orchestra.run.smithi190.stdout:(118/121): openssl-1.1.1k-12.el8.x86_64.rpm 5.8 MB/s | 738 kB 00:00 2024-04-17T22:27:02.624 INFO:teuthology.orchestra.run.smithi102.stdout:(108/121): NetworkManager-tui-1.40.16-15.el8.x8 1.9 MB/s | 356 kB 00:00 2024-04-17T22:27:02.649 INFO:teuthology.orchestra.run.smithi102.stdout:(109/121): elfutils-debuginfod-client-0.190-2.e 915 kB/s | 77 kB 00:00 2024-04-17T22:27:02.674 INFO:teuthology.orchestra.run.smithi102.stdout:(110/121): elfutils-libelf-0.190-2.el8.x86_64.r 3.0 MB/s | 233 kB 00:00 2024-04-17T22:27:02.682 INFO:teuthology.orchestra.run.smithi190.stdout:(119/121): util-linux-2.32.1-46.el8.x86_64.rpm 24 MB/s | 2.6 MB 00:00 2024-04-17T22:27:02.699 INFO:teuthology.orchestra.run.smithi102.stdout:(111/121): elfutils-libs-0.190-2.el8.x86_64.rpm 3.9 MB/s | 305 kB 00:00 2024-04-17T22:27:02.707 INFO:teuthology.orchestra.run.smithi190.stdout:(120/121): util-linux-user-2.32.1-46.el8.x86_64 1.1 MB/s | 103 kB 00:00 2024-04-17T22:27:02.733 INFO:teuthology.orchestra.run.smithi102.stdout:(112/121): iproute-6.2.0-5.el8.x86_64.rpm 10 MB/s | 854 kB 00:00 2024-04-17T22:27:02.741 INFO:teuthology.orchestra.run.smithi190.stdout:(121/121): openssl-libs-1.1.1k-12.el8.x86_64.rp 7.7 MB/s | 1.5 MB 00:00 2024-04-17T22:27:02.743 INFO:teuthology.orchestra.run.smithi190.stdout:-------------------------------------------------------------------------------- 2024-04-17T22:27:02.744 INFO:teuthology.orchestra.run.smithi190.stdout:Total 18 MB/s | 166 MB 00:09 2024-04-17T22:27:02.758 INFO:teuthology.orchestra.run.smithi102.stdout:(113/121): libblkid-2.32.1-46.el8.x86_64.rpm 2.6 MB/s | 221 kB 00:00 2024-04-17T22:27:02.784 INFO:teuthology.orchestra.run.smithi102.stdout:(114/121): libfdisk-2.32.1-46.el8.x86_64.rpm 3.0 MB/s | 254 kB 00:00 2024-04-17T22:27:02.809 INFO:teuthology.orchestra.run.smithi102.stdout:(115/121): libmount-2.32.1-46.el8.x86_64.rpm 3.1 MB/s | 237 kB 00:00 2024-04-17T22:27:02.835 INFO:teuthology.orchestra.run.smithi102.stdout:(116/121): libsmartcols-2.32.1-46.el8.x86_64.rp 2.3 MB/s | 180 kB 00:00 2024-04-17T22:27:02.860 INFO:teuthology.orchestra.run.smithi102.stdout:(117/121): libuuid-2.32.1-46.el8.x86_64.rpm 1.3 MB/s | 99 kB 00:00 2024-04-17T22:27:02.922 INFO:teuthology.orchestra.run.smithi102.stdout:(118/121): openssl-1.1.1k-12.el8.x86_64.rpm 6.4 MB/s | 738 kB 00:00 2024-04-17T22:27:02.960 INFO:teuthology.orchestra.run.smithi102.stdout:(119/121): openssl-libs-1.1.1k-12.el8.x86_64.rp 12 MB/s | 1.5 MB 00:00 2024-04-17T22:27:02.986 INFO:teuthology.orchestra.run.smithi102.stdout:(120/121): util-linux-user-2.32.1-46.el8.x86_64 1.6 MB/s | 103 kB 00:00 2024-04-17T22:27:03.052 INFO:teuthology.orchestra.run.smithi102.stdout:(121/121): util-linux-2.32.1-46.el8.x86_64.rpm 13 MB/s | 2.6 MB 00:00 2024-04-17T22:27:03.057 INFO:teuthology.orchestra.run.smithi102.stdout:-------------------------------------------------------------------------------- 2024-04-17T22:27:03.057 INFO:teuthology.orchestra.run.smithi102.stdout:Total 18 MB/s | 166 MB 00:09 2024-04-17T22:27:03.531 INFO:teuthology.orchestra.run.smithi190.stdout:Running transaction check 2024-04-17T22:27:03.790 INFO:teuthology.orchestra.run.smithi190.stdout:Transaction check succeeded. 2024-04-17T22:27:03.790 INFO:teuthology.orchestra.run.smithi190.stdout:Running transaction test 2024-04-17T22:27:03.821 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction check 2024-04-17T22:27:04.081 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction check succeeded. 2024-04-17T22:27:04.081 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction test 2024-04-17T22:27:05.031 INFO:teuthology.orchestra.run.smithi190.stdout:Transaction test succeeded. 2024-04-17T22:27:05.034 INFO:teuthology.orchestra.run.smithi190.stdout:Running transaction 2024-04-17T22:27:05.350 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction test succeeded. 2024-04-17T22:27:05.353 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction 2024-04-17T22:27:09.660 INFO:teuthology.orchestra.run.smithi190.stdout: Preparing : 1/1 2024-04-17T22:27:09.827 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: openssl-libs-1:1.1.1k-12.el8.x86_64 1/1 2024-04-17T22:27:10.015 INFO:teuthology.orchestra.run.smithi190.stdout: Upgrading : openssl-libs-1:1.1.1k-12.el8.x86_64 1/140 2024-04-17T22:27:10.083 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: openssl-libs-1:1.1.1k-12.el8.x86_64 1/140 2024-04-17T22:27:10.216 INFO:teuthology.orchestra.run.smithi102.stdout: Preparing : 1/1 2024-04-17T22:27:10.287 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/140 2024-04-17T22:27:10.390 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: openssl-libs-1:1.1.1k-12.el8.x86_64 1/1 2024-04-17T22:27:10.458 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : leveldb-1.22-1.el8.x86_64 3/140 2024-04-17T22:27:10.616 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : librabbitmq-0.9.0-5.el8.x86_64 4/140 2024-04-17T22:27:10.624 INFO:teuthology.orchestra.run.smithi102.stdout: Upgrading : openssl-libs-1:1.1.1k-12.el8.x86_64 1/140 2024-04-17T22:27:10.702 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: openssl-libs-1:1.1.1k-12.el8.x86_64 1/140 2024-04-17T22:27:10.754 INFO:teuthology.orchestra.run.smithi190.stdout: Upgrading : libuuid-2.32.1-46.el8.x86_64 5/140 2024-04-17T22:27:10.840 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: libuuid-2.32.1-46.el8.x86_64 5/140 2024-04-17T22:27:10.909 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/140 2024-04-17T22:27:11.089 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : leveldb-1.22-1.el8.x86_64 3/140 2024-04-17T22:27:11.287 INFO:teuthology.orchestra.run.smithi190.stdout: Upgrading : libblkid-2.32.1-46.el8.x86_64 6/140 2024-04-17T22:27:11.298 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : librabbitmq-0.9.0-5.el8.x86_64 4/140 2024-04-17T22:27:11.367 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: libblkid-2.32.1-46.el8.x86_64 6/140 2024-04-17T22:27:11.615 INFO:teuthology.orchestra.run.smithi190.stdout: Upgrading : elfutils-libelf-0.190-2.el8.x86_64 7/140 2024-04-17T22:27:11.742 INFO:teuthology.orchestra.run.smithi102.stdout: Upgrading : libuuid-2.32.1-46.el8.x86_64 5/140 2024-04-17T22:27:11.795 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 8/140 2024-04-17T22:27:11.833 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: libuuid-2.32.1-46.el8.x86_64 5/140 2024-04-17T22:27:12.034 INFO:teuthology.orchestra.run.smithi102.stdout: Upgrading : libblkid-2.32.1-46.el8.x86_64 6/140 2024-04-17T22:27:12.112 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: libblkid-2.32.1-46.el8.x86_64 6/140 2024-04-17T22:27:12.165 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 8/140 2024-04-17T22:27:12.361 INFO:teuthology.orchestra.run.smithi190.stdout: Upgrading : librados2-2:14.2.22-0.el8.x86_64 9/140 2024-04-17T22:27:12.371 INFO:teuthology.orchestra.run.smithi102.stdout: Upgrading : elfutils-libelf-0.190-2.el8.x86_64 7/140 2024-04-17T22:27:12.551 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 8/140 2024-04-17T22:27:12.568 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: librados2-2:14.2.22-0.el8.x86_64 9/140 2024-04-17T22:27:12.794 INFO:teuthology.orchestra.run.smithi190.stdout: Upgrading : librbd1-2:14.2.22-0.el8.x86_64 10/140 2024-04-17T22:27:12.899 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: librbd1-2:14.2.22-0.el8.x86_64 10/140 2024-04-17T22:27:12.929 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 8/140 2024-04-17T22:27:13.061 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libcephfs2-2:14.2.22-0.el8.x86_64 11/140 2024-04-17T22:27:13.154 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: libcephfs2-2:14.2.22-0.el8.x86_64 11/140 2024-04-17T22:27:13.175 INFO:teuthology.orchestra.run.smithi102.stdout: Upgrading : librados2-2:14.2.22-0.el8.x86_64 9/140 2024-04-17T22:27:13.387 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: librados2-2:14.2.22-0.el8.x86_64 9/140 2024-04-17T22:27:13.584 INFO:teuthology.orchestra.run.smithi102.stdout: Upgrading : librbd1-2:14.2.22-0.el8.x86_64 10/140 2024-04-17T22:27:13.683 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: librbd1-2:14.2.22-0.el8.x86_64 10/140 2024-04-17T22:27:13.777 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-rados-2:14.2.22-0.el8.x86_64 12/140 2024-04-17T22:27:13.892 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : libcephfs2-2:14.2.22-0.el8.x86_64 11/140 2024-04-17T22:27:13.936 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : librgw2-2:14.2.22-0.el8.x86_64 13/140 2024-04-17T22:27:13.963 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: libcephfs2-2:14.2.22-0.el8.x86_64 11/140 2024-04-17T22:27:13.996 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: librgw2-2:14.2.22-0.el8.x86_64 13/140 2024-04-17T22:27:14.232 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 14/140 2024-04-17T22:27:14.434 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 15/140 2024-04-17T22:27:14.561 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libradosstriper1-2:14.2.22-0.el8.x86_64 16/140 2024-04-17T22:27:14.633 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-rados-2:14.2.22-0.el8.x86_64 12/140 2024-04-17T22:27:14.817 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : librgw2-2:14.2.22-0.el8.x86_64 13/140 2024-04-17T22:27:14.889 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: librgw2-2:14.2.22-0.el8.x86_64 13/140 2024-04-17T22:27:14.936 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: libradosstriper1-2:14.2.22-0.el8.x86_64 16/140 2024-04-17T22:27:15.137 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 14/140 2024-04-17T22:27:15.186 INFO:teuthology.orchestra.run.smithi190.stdout: Upgrading : NetworkManager-libnm-1:1.40.16-15.el8.x86_64 17/140 2024-04-17T22:27:15.233 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: NetworkManager-libnm-1:1.40.16-15.el8.x86_64 17/140 2024-04-17T22:27:15.349 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 15/140 2024-04-17T22:27:15.477 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: NetworkManager-1:1.40.16-15.el8.x86_64 18/140 2024-04-17T22:27:15.491 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : libradosstriper1-2:14.2.22-0.el8.x86_64 16/140 2024-04-17T22:27:15.698 INFO:teuthology.orchestra.run.smithi190.stdout: Upgrading : NetworkManager-1:1.40.16-15.el8.x86_64 18/140 2024-04-17T22:27:15.835 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: libradosstriper1-2:14.2.22-0.el8.x86_64 16/140 2024-04-17T22:27:16.086 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: NetworkManager-1:1.40.16-15.el8.x86_64 18/140 2024-04-17T22:27:16.091 INFO:teuthology.orchestra.run.smithi102.stdout: Upgrading : NetworkManager-libnm-1:1.40.16-15.el8.x86_64 17/140 2024-04-17T22:27:16.184 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: NetworkManager-libnm-1:1.40.16-15.el8.x86_64 17/140 2024-04-17T22:27:16.266 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 19/140 2024-04-17T22:27:16.409 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : zlib-devel-1.2.11-21.el8.x86_64 20/140 2024-04-17T22:27:16.419 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: NetworkManager-1:1.40.16-15.el8.x86_64 18/140 2024-04-17T22:27:16.536 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libquadmath-8.5.0-21.el8.x86_64 21/140 2024-04-17T22:27:16.699 INFO:teuthology.orchestra.run.smithi102.stdout: Upgrading : NetworkManager-1:1.40.16-15.el8.x86_64 18/140 2024-04-17T22:27:16.755 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: libquadmath-8.5.0-21.el8.x86_64 21/140 2024-04-17T22:27:16.919 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libgfortran-8.5.0-21.el8.x86_64 22/140 2024-04-17T22:27:17.100 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: NetworkManager-1:1.40.16-15.el8.x86_64 18/140 2024-04-17T22:27:17.364 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 19/140 2024-04-17T22:27:17.512 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: libgfortran-8.5.0-21.el8.x86_64 22/140 2024-04-17T22:27:17.522 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : zlib-devel-1.2.11-21.el8.x86_64 20/140 2024-04-17T22:27:17.666 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : libquadmath-8.5.0-21.el8.x86_64 21/140 2024-04-17T22:27:17.711 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : openblas-0.3.15-4.el8.x86_64 23/140 2024-04-17T22:27:17.813 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: libquadmath-8.5.0-21.el8.x86_64 21/140 2024-04-17T22:27:17.958 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : libgfortran-8.5.0-21.el8.x86_64 22/140 2024-04-17T22:27:18.324 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: openblas-0.3.15-4.el8.x86_64 23/140 2024-04-17T22:27:18.532 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : openblas-threads-0.3.15-4.el8.x86_64 24/140 2024-04-17T22:27:18.553 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: libgfortran-8.5.0-21.el8.x86_64 22/140 2024-04-17T22:27:18.741 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : openblas-0.3.15-4.el8.x86_64 23/140 2024-04-17T22:27:19.212 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: openblas-threads-0.3.15-4.el8.x86_64 24/140 2024-04-17T22:27:19.355 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: openblas-0.3.15-4.el8.x86_64 23/140 2024-04-17T22:27:19.534 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 25/140 2024-04-17T22:27:19.558 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : openblas-threads-0.3.15-4.el8.x86_64 24/140 2024-04-17T22:27:19.732 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 26/140 2024-04-17T22:27:19.944 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-ceph-argparse-2:14.2.22-0.el8.x86_64 27/140 2024-04-17T22:27:20.164 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-cephfs-2:14.2.22-0.el8.x86_64 28/140 2024-04-17T22:27:20.248 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: openblas-threads-0.3.15-4.el8.x86_64 24/140 2024-04-17T22:27:20.335 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/140 2024-04-17T22:27:20.600 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 25/140 2024-04-17T22:27:20.796 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 26/140 2024-04-17T22:27:21.008 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-ceph-argparse-2:14.2.22-0.el8.x86_64 27/140 2024-04-17T22:27:21.245 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-cephfs-2:14.2.22-0.el8.x86_64 28/140 2024-04-17T22:27:21.458 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/140 2024-04-17T22:27:22.121 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/140 2024-04-17T22:27:22.429 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-scipy-1.0.0-21.module_el8+762+77bd8591.x 30/140 2024-04-17T22:27:22.609 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 31/140 2024-04-17T22:27:22.796 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-rsa-4.9-2.el8.noarch 32/140 2024-04-17T22:27:22.995 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-rgw-2:14.2.22-0.el8.x86_64 33/140 2024-04-17T22:27:23.178 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-rbd-2:14.2.22-0.el8.x86_64 34/140 2024-04-17T22:27:23.250 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/140 2024-04-17T22:27:23.366 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : librados-devel-2:14.2.22-0.el8.x86_64 35/140 2024-04-17T22:27:23.552 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-scipy-1.0.0-21.module_el8+762+77bd8591.x 30/140 2024-04-17T22:27:23.553 INFO:teuthology.orchestra.run.smithi190.stdout: Upgrading : elfutils-libs-0.190-2.el8.x86_64 36/140 2024-04-17T22:27:23.750 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 31/140 2024-04-17T22:27:23.760 INFO:teuthology.orchestra.run.smithi190.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el8.x86_64 37/140 2024-04-17T22:27:23.919 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-rsa-4.9-2.el8.noarch 32/140 2024-04-17T22:27:23.984 INFO:teuthology.orchestra.run.smithi190.stdout: Upgrading : libfdisk-2.32.1-46.el8.x86_64 38/140 2024-04-17T22:27:24.058 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: libfdisk-2.32.1-46.el8.x86_64 38/140 2024-04-17T22:27:24.159 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-rgw-2:14.2.22-0.el8.x86_64 33/140 2024-04-17T22:27:24.243 INFO:teuthology.orchestra.run.smithi190.stdout: Upgrading : libmount-2.32.1-46.el8.x86_64 39/140 2024-04-17T22:27:24.350 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-rbd-2:14.2.22-0.el8.x86_64 34/140 2024-04-17T22:27:24.397 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: libmount-2.32.1-46.el8.x86_64 39/140 2024-04-17T22:27:24.565 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : librdkafka-1.6.1-1.el8.x86_64 40/140 2024-04-17T22:27:24.572 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : librados-devel-2:14.2.22-0.el8.x86_64 35/140 2024-04-17T22:27:24.746 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 41/140 2024-04-17T22:27:24.765 INFO:teuthology.orchestra.run.smithi102.stdout: Upgrading : elfutils-libs-0.190-2.el8.x86_64 36/140 2024-04-17T22:27:24.959 INFO:teuthology.orchestra.run.smithi190.stdout: Upgrading : libsmartcols-2.32.1-46.el8.x86_64 42/140 2024-04-17T22:27:24.974 INFO:teuthology.orchestra.run.smithi102.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el8.x86_64 37/140 2024-04-17T22:27:25.182 INFO:teuthology.orchestra.run.smithi102.stdout: Upgrading : libfdisk-2.32.1-46.el8.x86_64 38/140 2024-04-17T22:27:25.244 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: libfdisk-2.32.1-46.el8.x86_64 38/140 2024-04-17T22:27:25.437 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: libsmartcols-2.32.1-46.el8.x86_64 42/140 2024-04-17T22:27:25.474 INFO:teuthology.orchestra.run.smithi102.stdout: Upgrading : libmount-2.32.1-46.el8.x86_64 39/140 2024-04-17T22:27:25.598 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: libmount-2.32.1-46.el8.x86_64 39/140 2024-04-17T22:27:25.701 INFO:teuthology.orchestra.run.smithi190.stdout: Upgrading : util-linux-2.32.1-46.el8.x86_64 43/140 2024-04-17T22:27:25.715 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: util-linux-2.32.1-46.el8.x86_64 43/140 2024-04-17T22:27:25.808 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : librdkafka-1.6.1-1.el8.x86_64 40/140 2024-04-17T22:27:25.886 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 44/140 2024-04-17T22:27:25.977 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 41/140 2024-04-17T22:27:26.116 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 45/140 2024-04-17T22:27:26.199 INFO:teuthology.orchestra.run.smithi102.stdout: Upgrading : libsmartcols-2.32.1-46.el8.x86_64 42/140 2024-04-17T22:27:26.316 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 46/140 2024-04-17T22:27:26.497 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 47/140 2024-04-17T22:27:26.627 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 48/140 2024-04-17T22:27:26.676 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: libsmartcols-2.32.1-46.el8.x86_64 42/140 2024-04-17T22:27:26.809 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 49/140 2024-04-17T22:27:26.957 INFO:teuthology.orchestra.run.smithi102.stdout: Upgrading : util-linux-2.32.1-46.el8.x86_64 43/140 2024-04-17T22:27:26.971 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: util-linux-2.32.1-46.el8.x86_64 43/140 2024-04-17T22:27:27.143 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 44/140 2024-04-17T22:27:27.453 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 50/140 2024-04-17T22:27:27.620 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 51/140 2024-04-17T22:27:27.673 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 45/140 2024-04-17T22:27:27.765 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 52/140 2024-04-17T22:27:27.917 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 53/140 2024-04-17T22:27:28.005 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 46/140 2024-04-17T22:27:28.083 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 54/140 2024-04-17T22:27:28.219 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 47/140 2024-04-17T22:27:28.332 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 55/140 2024-04-17T22:27:28.384 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 48/140 2024-04-17T22:27:28.549 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 56/140 2024-04-17T22:27:28.598 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 49/140 2024-04-17T22:27:28.728 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 57/140 2024-04-17T22:27:28.802 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 50/140 2024-04-17T22:27:28.968 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 51/140 2024-04-17T22:27:28.969 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 58/140 2024-04-17T22:27:29.112 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 52/140 2024-04-17T22:27:29.180 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 59/140 2024-04-17T22:27:29.306 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 53/140 2024-04-17T22:27:29.381 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 60/140 2024-04-17T22:27:29.456 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 54/140 2024-04-17T22:27:29.606 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 61/140 2024-04-17T22:27:29.688 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 55/140 2024-04-17T22:27:29.913 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 56/140 2024-04-17T22:27:29.943 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-cffi-1.11.5-6.el8.x86_64 62/140 2024-04-17T22:27:30.141 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 57/140 2024-04-17T22:27:30.203 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-cryptography-3.2.1-7.el8.x86_64 63/140 2024-04-17T22:27:30.352 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 58/140 2024-04-17T22:27:30.384 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 64/140 2024-04-17T22:27:30.583 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 59/140 2024-04-17T22:27:30.612 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 65/140 2024-04-17T22:27:30.796 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 60/140 2024-04-17T22:27:30.870 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 66/140 2024-04-17T22:27:31.020 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 61/140 2024-04-17T22:27:31.366 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-cffi-1.11.5-6.el8.x86_64 62/140 2024-04-17T22:27:31.600 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-cryptography-3.2.1-7.el8.x86_64 63/140 2024-04-17T22:27:31.883 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 64/140 2024-04-17T22:27:31.910 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 67/140 2024-04-17T22:27:32.138 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 65/140 2024-04-17T22:27:32.141 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 68/140 2024-04-17T22:27:32.340 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 69/140 2024-04-17T22:27:32.386 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 66/140 2024-04-17T22:27:32.525 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 70/140 2024-04-17T22:27:32.760 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 71/140 2024-04-17T22:27:32.936 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : pcre2-utf32-10.32-3.el8.x86_64 72/140 2024-04-17T22:27:33.220 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : pcre2-utf16-10.32-3.el8.x86_64 73/140 2024-04-17T22:27:33.388 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : pcre2-devel-10.32-3.el8.x86_64 74/140 2024-04-17T22:27:33.454 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 67/140 2024-04-17T22:27:33.510 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libzstd-devel-1.4.4-1.el8.x86_64 75/140 2024-04-17T22:27:33.661 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libverto-devel-0.3.2-2.el8.x86_64 76/140 2024-04-17T22:27:33.706 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 68/140 2024-04-17T22:27:33.915 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 69/140 2024-04-17T22:27:33.985 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 77/140 2024-04-17T22:27:34.140 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 70/140 2024-04-17T22:27:34.195 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libselinux-devel-2.9-8.el8.x86_64 78/140 2024-04-17T22:27:34.328 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libcom_err-devel-1.45.6-5.el8.x86_64 79/140 2024-04-17T22:27:34.351 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 71/140 2024-04-17T22:27:34.529 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 80/140 2024-04-17T22:27:34.568 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : pcre2-utf32-10.32-3.el8.x86_64 72/140 2024-04-17T22:27:34.750 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : krb5-devel-1.18.2-22.el8.x86_64 81/140 2024-04-17T22:27:34.866 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : pcre2-utf16-10.32-3.el8.x86_64 73/140 2024-04-17T22:27:34.934 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : fuse-2.9.7-19.el8.x86_64 82/140 2024-04-17T22:27:35.061 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : pcre2-devel-10.32-3.el8.x86_64 74/140 2024-04-17T22:27:35.138 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 83/140 2024-04-17T22:27:35.191 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : libzstd-devel-1.4.4-1.el8.x86_64 75/140 2024-04-17T22:27:35.335 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-pytz-2017.2-11.el8.noarch 84/140 2024-04-17T22:27:35.361 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : libverto-devel-0.3.2-2.el8.x86_64 76/140 2024-04-17T22:27:35.546 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 85/140 2024-04-17T22:27:35.676 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 77/140 2024-04-17T22:27:35.804 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-portend-2.6-1.el8.noarch 86/140 2024-04-17T22:27:35.893 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : libselinux-devel-2.9-8.el8.x86_64 78/140 2024-04-17T22:27:36.018 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : libcom_err-devel-1.45.6-5.el8.x86_64 79/140 2024-04-17T22:27:36.040 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 87/140 2024-04-17T22:27:36.183 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 88/140 2024-04-17T22:27:36.254 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 80/140 2024-04-17T22:27:36.507 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : krb5-devel-1.18.2-22.el8.x86_64 81/140 2024-04-17T22:27:36.699 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : fuse-2.9.7-19.el8.x86_64 82/140 2024-04-17T22:27:36.921 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 83/140 2024-04-17T22:27:37.101 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-pytz-2017.2-11.el8.noarch 84/140 2024-04-17T22:27:37.345 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 85/140 2024-04-17T22:27:37.607 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-portend-2.6-1.el8.noarch 86/140 2024-04-17T22:27:37.855 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 87/140 2024-04-17T22:27:37.989 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 88/140 2024-04-17T22:27:38.299 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-common-2:14.2.22-0.el8.x86_64 89/140 2024-04-17T22:27:38.633 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-common-2:14.2.22-0.el8.x86_64 89/140 2024-04-17T22:27:39.249 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-common-2:14.2.22-0.el8.x86_64 89/140 2024-04-17T22:27:39.528 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-base-2:14.2.22-0.el8.x86_64 90/140 2024-04-17T22:27:39.634 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-base-2:14.2.22-0.el8.x86_64 90/140 2024-04-17T22:27:39.790 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-selinux-2:14.2.22-0.el8.x86_64 91/140 2024-04-17T22:27:40.161 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-common-2:14.2.22-0.el8.x86_64 89/140 2024-04-17T22:27:40.406 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-common-2:14.2.22-0.el8.x86_64 89/140 2024-04-17T22:27:41.027 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-common-2:14.2.22-0.el8.x86_64 89/140 2024-04-17T22:27:41.337 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-base-2:14.2.22-0.el8.x86_64 90/140 2024-04-17T22:27:41.438 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-base-2:14.2.22-0.el8.x86_64 90/140 2024-04-17T22:27:41.588 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-selinux-2:14.2.22-0.el8.x86_64 91/140 2024-04-17T22:27:59.013 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-selinux-2:14.2.22-0.el8.x86_64 91/140 2024-04-17T22:27:59.013 INFO:teuthology.orchestra.run.smithi190.stdout:skipping the directory /sys 2024-04-17T22:27:59.014 INFO:teuthology.orchestra.run.smithi190.stdout:skipping the directory /proc 2024-04-17T22:27:59.014 INFO:teuthology.orchestra.run.smithi190.stdout:skipping the directory /dev 2024-04-17T22:27:59.014 INFO:teuthology.orchestra.run.smithi190.stdout:skipping the directory /run 2024-04-17T22:27:59.014 INFO:teuthology.orchestra.run.smithi190.stdout:skipping the directory /mnt 2024-04-17T22:27:59.014 INFO:teuthology.orchestra.run.smithi190.stdout:skipping the directory /var/tmp 2024-04-17T22:27:59.014 INFO:teuthology.orchestra.run.smithi190.stdout:skipping the directory /home 2024-04-17T22:27:59.014 INFO:teuthology.orchestra.run.smithi190.stdout:skipping the directory /tmp 2024-04-17T22:27:59.014 INFO:teuthology.orchestra.run.smithi190.stdout:skipping the directory /dev 2024-04-17T22:27:59.014 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-04-17T22:27:59.363 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-mds-2:14.2.22-0.el8.x86_64 92/140 2024-04-17T22:27:59.745 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-mds-2:14.2.22-0.el8.x86_64 92/140 2024-04-17T22:27:59.938 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-mon-2:14.2.22-0.el8.x86_64 93/140 2024-04-17T22:28:00.673 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-selinux-2:14.2.22-0.el8.x86_64 91/140 2024-04-17T22:28:00.674 INFO:teuthology.orchestra.run.smithi102.stdout:skipping the directory /sys 2024-04-17T22:28:00.674 INFO:teuthology.orchestra.run.smithi102.stdout:skipping the directory /proc 2024-04-17T22:28:00.674 INFO:teuthology.orchestra.run.smithi102.stdout:skipping the directory /dev 2024-04-17T22:28:00.674 INFO:teuthology.orchestra.run.smithi102.stdout:skipping the directory /run 2024-04-17T22:28:00.674 INFO:teuthology.orchestra.run.smithi102.stdout:skipping the directory /mnt 2024-04-17T22:28:00.674 INFO:teuthology.orchestra.run.smithi102.stdout:skipping the directory /var/tmp 2024-04-17T22:28:00.674 INFO:teuthology.orchestra.run.smithi102.stdout:skipping the directory /home 2024-04-17T22:28:00.674 INFO:teuthology.orchestra.run.smithi102.stdout:skipping the directory /tmp 2024-04-17T22:28:00.674 INFO:teuthology.orchestra.run.smithi102.stdout:skipping the directory /dev 2024-04-17T22:28:00.674 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-17T22:28:01.028 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-mds-2:14.2.22-0.el8.x86_64 92/140 2024-04-17T22:28:01.315 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-mon-2:14.2.22-0.el8.x86_64 93/140 2024-04-17T22:28:01.411 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-mds-2:14.2.22-0.el8.x86_64 92/140 2024-04-17T22:28:01.532 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-osd-2:14.2.22-0.el8.x86_64 94/140 2024-04-17T22:28:01.566 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-osd-2:14.2.22-0.el8.x86_64 94/140 2024-04-17T22:28:01.611 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-mon-2:14.2.22-0.el8.x86_64 93/140 2024-04-17T22:28:01.821 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 95/140 2024-04-17T22:28:02.132 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-mako-1.0.6-14.el8.noarch 96/140 2024-04-17T22:28:02.414 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 97/140 2024-04-17T22:28:02.553 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : oniguruma-6.8.2-3.el8.x86_64 98/140 2024-04-17T22:28:02.619 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: oniguruma-6.8.2-3.el8.x86_64 98/140 2024-04-17T22:28:02.827 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : jq-1.6-8.el8.x86_64 99/140 2024-04-17T22:28:02.989 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-mon-2:14.2.22-0.el8.x86_64 93/140 2024-04-17T22:28:03.201 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-osd-2:14.2.22-0.el8.x86_64 94/140 2024-04-17T22:28:03.235 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-osd-2:14.2.22-0.el8.x86_64 94/140 2024-04-17T22:28:03.429 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 95/140 2024-04-17T22:28:03.740 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-grafana-dashboards-2:14.2.22-0.el8.noarch 100/140 2024-04-17T22:28:03.768 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-mako-1.0.6-14.el8.noarch 96/140 2024-04-17T22:28:03.990 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-mgr-dashboard-2:14.2.22-0.el8.noarch 101/140 2024-04-17T22:28:04.071 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 97/140 2024-04-17T22:28:04.210 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : oniguruma-6.8.2-3.el8.x86_64 98/140 2024-04-17T22:28:04.271 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: oniguruma-6.8.2-3.el8.x86_64 98/140 2024-04-17T22:28:04.306 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-mgr-dashboard-2:14.2.22-0.el8.noarch 101/140 2024-04-17T22:28:04.478 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-mgr-diskprediction-local-2:14.2.22-0.el8.no 102/140 2024-04-17T22:28:04.483 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : jq-1.6-8.el8.x86_64 99/140 2024-04-17T22:28:04.491 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:14.2.22-0.el8.no 102/140 2024-04-17T22:28:04.636 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-mgr-rook-2:14.2.22-0.el8.noarch 103/140 2024-04-17T22:28:04.824 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-mgr-rook-2:14.2.22-0.el8.noarch 103/140 2024-04-17T22:28:05.074 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-mgr-2:14.2.22-0.el8.x86_64 104/140 2024-04-17T22:28:05.090 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-mgr-2:14.2.22-0.el8.x86_64 104/140 2024-04-17T22:28:05.392 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-grafana-dashboards-2:14.2.22-0.el8.noarch 100/140 2024-04-17T22:28:05.667 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-mgr-dashboard-2:14.2.22-0.el8.noarch 101/140 2024-04-17T22:28:05.983 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-mgr-dashboard-2:14.2.22-0.el8.noarch 101/140 2024-04-17T22:28:06.160 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-mgr-diskprediction-local-2:14.2.22-0.el8.no 102/140 2024-04-17T22:28:06.173 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:14.2.22-0.el8.no 102/140 2024-04-17T22:28:06.318 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-mgr-rook-2:14.2.22-0.el8.noarch 103/140 2024-04-17T22:28:06.506 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-mgr-rook-2:14.2.22-0.el8.noarch 103/140 2024-04-17T22:28:06.726 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-mgr-2:14.2.22-0.el8.x86_64 104/140 2024-04-17T22:28:06.743 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-mgr-2:14.2.22-0.el8.x86_64 104/140 2024-04-17T22:28:08.233 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-2:14.2.22-0.el8.x86_64 105/140 2024-04-17T22:28:08.954 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-test-2:14.2.22-0.el8.x86_64 106/140 2024-04-17T22:28:09.198 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-radosgw-2:14.2.22-0.el8.x86_64 107/140 2024-04-17T22:28:09.416 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-radosgw-2:14.2.22-0.el8.x86_64 107/140 2024-04-17T22:28:09.586 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : rbd-mirror-2:14.2.22-0.el8.x86_64 108/140 2024-04-17T22:28:09.653 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: rbd-mirror-2:14.2.22-0.el8.x86_64 108/140 2024-04-17T22:28:09.924 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-2:14.2.22-0.el8.x86_64 105/140 2024-04-17T22:28:10.485 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-fuse-2:14.2.22-0.el8.x86_64 109/140 2024-04-17T22:28:10.689 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-test-2:14.2.22-0.el8.x86_64 106/140 2024-04-17T22:28:10.765 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : openssl-devel-1:1.1.1k-12.el8.x86_64 110/140 2024-04-17T22:28:10.924 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : elfutils-libelf-devel-0.190-2.el8.x86_64 111/140 2024-04-17T22:28:10.950 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-radosgw-2:14.2.22-0.el8.x86_64 107/140 2024-04-17T22:28:11.115 INFO:teuthology.orchestra.run.smithi190.stdout: Upgrading : util-linux-user-2.32.1-46.el8.x86_64 112/140 2024-04-17T22:28:11.168 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-radosgw-2:14.2.22-0.el8.x86_64 107/140 2024-04-17T22:28:11.283 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libcephfs-devel-2:14.2.22-0.el8.x86_64 113/140 2024-04-17T22:28:11.351 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : rbd-mirror-2:14.2.22-0.el8.x86_64 108/140 2024-04-17T22:28:11.418 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: rbd-mirror-2:14.2.22-0.el8.x86_64 108/140 2024-04-17T22:28:11.492 INFO:teuthology.orchestra.run.smithi190.stdout: Upgrading : NetworkManager-team-1:1.40.16-15.el8.x86_64 114/140 2024-04-17T22:28:11.658 INFO:teuthology.orchestra.run.smithi190.stdout: Upgrading : NetworkManager-tui-1:1.40.16-15.el8.x86_64 115/140 2024-04-17T22:28:11.852 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : rbd-fuse-2:14.2.22-0.el8.x86_64 116/140 2024-04-17T22:28:12.176 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : rbd-nbd-2:14.2.22-0.el8.x86_64 117/140 2024-04-17T22:28:12.294 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-fuse-2:14.2.22-0.el8.x86_64 109/140 2024-04-17T22:28:12.502 INFO:teuthology.orchestra.run.smithi190.stdout: Upgrading : iproute-6.2.0-5.el8.x86_64 118/140 2024-04-17T22:28:12.547 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : openssl-devel-1:1.1.1k-12.el8.x86_64 110/140 2024-04-17T22:28:12.748 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : elfutils-libelf-devel-0.190-2.el8.x86_64 111/140 2024-04-17T22:28:12.764 INFO:teuthology.orchestra.run.smithi190.stdout: Upgrading : openssl-1:1.1.1k-12.el8.x86_64 119/140 2024-04-17T22:28:12.903 INFO:teuthology.orchestra.run.smithi102.stdout: Upgrading : util-linux-user-2.32.1-46.el8.x86_64 112/140 2024-04-17T22:28:12.976 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : flex-2.6.1-9.el8.x86_64 120/140 2024-04-17T22:28:13.064 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 120/140 2024-04-17T22:28:13.110 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : libcephfs-devel-2:14.2.22-0.el8.x86_64 113/140 2024-04-17T22:28:13.287 INFO:teuthology.orchestra.run.smithi102.stdout: Upgrading : NetworkManager-team-1:1.40.16-15.el8.x86_64 114/140 2024-04-17T22:28:13.362 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : bison-3.0.4-10.el8.x86_64 121/140 2024-04-17T22:28:13.371 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 121/140 2024-04-17T22:28:13.449 INFO:teuthology.orchestra.run.smithi102.stdout: Upgrading : NetworkManager-tui-1:1.40.16-15.el8.x86_64 115/140 2024-04-17T22:28:13.512 INFO:teuthology.orchestra.run.smithi190.stdout: Cleanup : NetworkManager-tui-1:1.40.16-4.el8.x86_64 122/140 2024-04-17T22:28:13.679 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : rbd-fuse-2:14.2.22-0.el8.x86_64 116/140 2024-04-17T22:28:13.686 INFO:teuthology.orchestra.run.smithi190.stdout: Cleanup : elfutils-debuginfod-client-0.189-1.el8.x86_64 123/140 2024-04-17T22:28:13.834 INFO:teuthology.orchestra.run.smithi190.stdout: Cleanup : elfutils-libs-0.189-1.el8.x86_64 124/140 2024-04-17T22:28:14.006 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : rbd-nbd-2:14.2.22-0.el8.x86_64 117/140 2024-04-17T22:28:14.028 INFO:teuthology.orchestra.run.smithi190.stdout: Cleanup : openssl-1:1.1.1k-9.el8.x86_64 125/140 2024-04-17T22:28:14.028 INFO:teuthology.orchestra.run.smithi190.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 126/140 2024-04-17T22:28:14.220 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 126/140 2024-04-17T22:28:14.220 INFO:teuthology.orchestra.run.smithi190.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 127/140 2024-04-17T22:28:14.349 INFO:teuthology.orchestra.run.smithi102.stdout: Upgrading : iproute-6.2.0-5.el8.x86_64 118/140 2024-04-17T22:28:14.442 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 127/140 2024-04-17T22:28:14.614 INFO:teuthology.orchestra.run.smithi102.stdout: Upgrading : openssl-1:1.1.1k-12.el8.x86_64 119/140 2024-04-17T22:28:14.653 INFO:teuthology.orchestra.run.smithi190.stdout: Cleanup : iproute-6.2.0-1.el8.x86_64 128/140 2024-04-17T22:28:14.832 INFO:teuthology.orchestra.run.smithi190.stdout: Cleanup : util-linux-user-2.32.1-41.el8.x86_64 129/140 2024-04-17T22:28:14.833 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : flex-2.6.1-9.el8.x86_64 120/140 2024-04-17T22:28:14.923 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 120/140 2024-04-17T22:28:15.028 INFO:teuthology.orchestra.run.smithi190.stdout: Cleanup : util-linux-2.32.1-41.el8.x86_64 130/140 2024-04-17T22:28:15.028 INFO:teuthology.orchestra.run.smithi190.stdout: Cleanup : libfdisk-2.32.1-41.el8.x86_64 131/140 2024-04-17T22:28:15.242 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : bison-3.0.4-10.el8.x86_64 121/140 2024-04-17T22:28:15.254 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 121/140 2024-04-17T22:28:15.277 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: libfdisk-2.32.1-41.el8.x86_64 131/140 2024-04-17T22:28:15.277 INFO:teuthology.orchestra.run.smithi190.stdout: Cleanup : libmount-2.32.1-41.el8.x86_64 132/140 2024-04-17T22:28:15.411 INFO:teuthology.orchestra.run.smithi102.stdout: Cleanup : NetworkManager-tui-1:1.40.16-4.el8.x86_64 122/140 2024-04-17T22:28:15.479 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: libmount-2.32.1-41.el8.x86_64 132/140 2024-04-17T22:28:15.479 INFO:teuthology.orchestra.run.smithi190.stdout: Cleanup : libblkid-2.32.1-41.el8.x86_64 133/140 2024-04-17T22:28:15.627 INFO:teuthology.orchestra.run.smithi102.stdout: Cleanup : elfutils-debuginfod-client-0.189-1.el8.x86_64 123/140 2024-04-17T22:28:15.647 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: libblkid-2.32.1-41.el8.x86_64 133/140 2024-04-17T22:28:15.762 INFO:teuthology.orchestra.run.smithi102.stdout: Cleanup : elfutils-libs-0.189-1.el8.x86_64 124/140 2024-04-17T22:28:15.808 INFO:teuthology.orchestra.run.smithi190.stdout: Cleanup : NetworkManager-team-1:1.40.16-4.el8.x86_64 134/140 2024-04-17T22:28:15.818 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: NetworkManager-1:1.40.16-4.el8.x86_64 135/140 2024-04-17T22:28:15.818 INFO:teuthology.orchestra.run.smithi190.stdout: Cleanup : NetworkManager-1:1.40.16-4.el8.x86_64 135/140 2024-04-17T22:28:15.968 INFO:teuthology.orchestra.run.smithi102.stdout: Cleanup : openssl-1:1.1.1k-9.el8.x86_64 125/140 2024-04-17T22:28:15.969 INFO:teuthology.orchestra.run.smithi102.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 126/140 2024-04-17T22:28:16.186 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 126/140 2024-04-17T22:28:16.186 INFO:teuthology.orchestra.run.smithi102.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 127/140 2024-04-17T22:28:16.233 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: NetworkManager-1:1.40.16-4.el8.x86_64 135/140 2024-04-17T22:28:16.233 INFO:teuthology.orchestra.run.smithi190.stdout: Cleanup : NetworkManager-libnm-1:1.40.16-4.el8.x86_64 136/140 2024-04-17T22:28:16.458 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 127/140 2024-04-17T22:28:16.470 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: NetworkManager-libnm-1:1.40.16-4.el8.x86_64 136/140 2024-04-17T22:28:16.470 INFO:teuthology.orchestra.run.smithi190.stdout: Cleanup : libuuid-2.32.1-41.el8.x86_64 137/140 2024-04-17T22:28:16.652 INFO:teuthology.orchestra.run.smithi102.stdout: Cleanup : iproute-6.2.0-1.el8.x86_64 128/140 2024-04-17T22:28:16.669 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: libuuid-2.32.1-41.el8.x86_64 137/140 2024-04-17T22:28:16.670 INFO:teuthology.orchestra.run.smithi190.stdout: Cleanup : libsmartcols-2.32.1-41.el8.x86_64 138/140 2024-04-17T22:28:16.822 INFO:teuthology.orchestra.run.smithi102.stdout: Cleanup : util-linux-user-2.32.1-41.el8.x86_64 129/140 2024-04-17T22:28:16.853 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: libsmartcols-2.32.1-41.el8.x86_64 138/140 2024-04-17T22:28:17.003 INFO:teuthology.orchestra.run.smithi190.stdout: Cleanup : elfutils-libelf-0.189-1.el8.x86_64 139/140 2024-04-17T22:28:17.003 INFO:teuthology.orchestra.run.smithi190.stdout: Cleanup : openssl-libs-1:1.1.1k-9.el8.x86_64 140/140 2024-04-17T22:28:17.044 INFO:teuthology.orchestra.run.smithi102.stdout: Cleanup : util-linux-2.32.1-41.el8.x86_64 130/140 2024-04-17T22:28:17.044 INFO:teuthology.orchestra.run.smithi102.stdout: Cleanup : libfdisk-2.32.1-41.el8.x86_64 131/140 2024-04-17T22:28:17.287 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: libfdisk-2.32.1-41.el8.x86_64 131/140 2024-04-17T22:28:17.287 INFO:teuthology.orchestra.run.smithi102.stdout: Cleanup : libmount-2.32.1-41.el8.x86_64 132/140 2024-04-17T22:28:17.478 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: libmount-2.32.1-41.el8.x86_64 132/140 2024-04-17T22:28:17.478 INFO:teuthology.orchestra.run.smithi102.stdout: Cleanup : libblkid-2.32.1-41.el8.x86_64 133/140 2024-04-17T22:28:17.729 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: libblkid-2.32.1-41.el8.x86_64 133/140 2024-04-17T22:28:17.869 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: openssl-libs-1:1.1.1k-9.el8.x86_64 140/140 2024-04-17T22:28:17.869 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-2:14.2.22-0.el8.x86_64 1/140 2024-04-17T22:28:17.869 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-base-2:14.2.22-0.el8.x86_64 2/140 2024-04-17T22:28:17.869 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-common-2:14.2.22-0.el8.x86_64 3/140 2024-04-17T22:28:17.869 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-fuse-2:14.2.22-0.el8.x86_64 4/140 2024-04-17T22:28:17.869 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-mds-2:14.2.22-0.el8.x86_64 5/140 2024-04-17T22:28:17.869 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-mgr-2:14.2.22-0.el8.x86_64 6/140 2024-04-17T22:28:17.869 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-mon-2:14.2.22-0.el8.x86_64 7/140 2024-04-17T22:28:17.869 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-osd-2:14.2.22-0.el8.x86_64 8/140 2024-04-17T22:28:17.870 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-radosgw-2:14.2.22-0.el8.x86_64 9/140 2024-04-17T22:28:17.870 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-selinux-2:14.2.22-0.el8.x86_64 10/140 2024-04-17T22:28:17.870 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-test-2:14.2.22-0.el8.x86_64 11/140 2024-04-17T22:28:17.870 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libcephfs-devel-2:14.2.22-0.el8.x86_64 12/140 2024-04-17T22:28:17.870 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libcephfs2-2:14.2.22-0.el8.x86_64 13/140 2024-04-17T22:28:17.870 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : librados-devel-2:14.2.22-0.el8.x86_64 14/140 2024-04-17T22:28:17.870 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libradosstriper1-2:14.2.22-0.el8.x86_64 15/140 2024-04-17T22:28:17.870 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : librgw2-2:14.2.22-0.el8.x86_64 16/140 2024-04-17T22:28:17.870 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-ceph-argparse-2:14.2.22-0.el8.x86_64 17/140 2024-04-17T22:28:17.870 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-cephfs-2:14.2.22-0.el8.x86_64 18/140 2024-04-17T22:28:17.870 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-rados-2:14.2.22-0.el8.x86_64 19/140 2024-04-17T22:28:17.870 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-rbd-2:14.2.22-0.el8.x86_64 20/140 2024-04-17T22:28:17.871 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-rgw-2:14.2.22-0.el8.x86_64 21/140 2024-04-17T22:28:17.871 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : rbd-fuse-2:14.2.22-0.el8.x86_64 22/140 2024-04-17T22:28:17.871 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : rbd-mirror-2:14.2.22-0.el8.x86_64 23/140 2024-04-17T22:28:17.871 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : rbd-nbd-2:14.2.22-0.el8.x86_64 24/140 2024-04-17T22:28:17.871 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-grafana-dashboards-2:14.2.22-0.el8.noarch 25/140 2024-04-17T22:28:17.871 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-mgr-dashboard-2:14.2.22-0.el8.noarch 26/140 2024-04-17T22:28:17.871 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-mgr-diskprediction-local-2:14.2.22-0.el8.no 27/140 2024-04-17T22:28:17.871 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-mgr-rook-2:14.2.22-0.el8.noarch 28/140 2024-04-17T22:28:17.871 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : bison-3.0.4-10.el8.x86_64 29/140 2024-04-17T22:28:17.871 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : flex-2.6.1-9.el8.x86_64 30/140 2024-04-17T22:28:17.871 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : jq-1.6-8.el8.x86_64 31/140 2024-04-17T22:28:17.871 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : librdkafka-1.6.1-1.el8.x86_64 32/140 2024-04-17T22:28:17.871 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 33/140 2024-04-17T22:28:17.872 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : oniguruma-6.8.2-3.el8.x86_64 34/140 2024-04-17T22:28:17.872 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : openblas-0.3.15-4.el8.x86_64 35/140 2024-04-17T22:28:17.872 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : openblas-threads-0.3.15-4.el8.x86_64 36/140 2024-04-17T22:28:17.872 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-mako-1.0.6-14.el8.noarch 37/140 2024-04-17T22:28:17.872 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 38/140 2024-04-17T22:28:17.872 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 39/140 2024-04-17T22:28:17.872 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 40/140 2024-04-17T22:28:17.872 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 41/140 2024-04-17T22:28:17.872 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 42/140 2024-04-17T22:28:17.872 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 43/140 2024-04-17T22:28:17.872 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 44/140 2024-04-17T22:28:17.872 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-pytz-2017.2-11.el8.noarch 45/140 2024-04-17T22:28:17.872 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-scipy-1.0.0-21.module_el8+762+77bd8591.x 46/140 2024-04-17T22:28:17.873 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 47/140 2024-04-17T22:28:17.873 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 48/140 2024-04-17T22:28:17.873 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 49/140 2024-04-17T22:28:17.873 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : elfutils-libelf-devel-0.190-2.el8.x86_64 50/140 2024-04-17T22:28:17.873 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : fuse-2.9.7-19.el8.x86_64 51/140 2024-04-17T22:28:17.873 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 52/140 2024-04-17T22:28:17.873 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : krb5-devel-1.18.2-22.el8.x86_64 53/140 2024-04-17T22:28:17.873 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libcom_err-devel-1.45.6-5.el8.x86_64 54/140 2024-04-17T22:28:17.873 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libgfortran-8.5.0-21.el8.x86_64 55/140 2024-04-17T22:28:17.873 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libquadmath-8.5.0-21.el8.x86_64 56/140 2024-04-17T22:28:17.873 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : librabbitmq-0.9.0-5.el8.x86_64 57/140 2024-04-17T22:28:17.873 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libselinux-devel-2.9-8.el8.x86_64 58/140 2024-04-17T22:28:17.873 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 59/140 2024-04-17T22:28:17.874 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libverto-devel-0.3.2-2.el8.x86_64 60/140 2024-04-17T22:28:17.874 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libzstd-devel-1.4.4-1.el8.x86_64 61/140 2024-04-17T22:28:17.874 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : openssl-devel-1:1.1.1k-12.el8.x86_64 62/140 2024-04-17T22:28:17.874 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : pcre2-devel-10.32-3.el8.x86_64 63/140 2024-04-17T22:28:17.874 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : pcre2-utf16-10.32-3.el8.x86_64 64/140 2024-04-17T22:28:17.874 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : pcre2-utf32-10.32-3.el8.x86_64 65/140 2024-04-17T22:28:17.874 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-cffi-1.11.5-6.el8.x86_64 66/140 2024-04-17T22:28:17.874 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-cryptography-3.2.1-7.el8.x86_64 67/140 2024-04-17T22:28:17.874 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 68/140 2024-04-17T22:28:17.874 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 69/140 2024-04-17T22:28:17.874 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 70/140 2024-04-17T22:28:17.874 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 71/140 2024-04-17T22:28:17.874 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : zlib-devel-1.2.11-21.el8.x86_64 72/140 2024-04-17T22:28:17.875 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 73/140 2024-04-17T22:28:17.875 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : leveldb-1.22-1.el8.x86_64 74/140 2024-04-17T22:28:17.875 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 75/140 2024-04-17T22:28:17.875 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 76/140 2024-04-17T22:28:17.875 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 77/140 2024-04-17T22:28:17.875 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 78/140 2024-04-17T22:28:17.875 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 79/140 2024-04-17T22:28:17.875 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 80/140 2024-04-17T22:28:17.875 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 81/140 2024-04-17T22:28:17.875 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 82/140 2024-04-17T22:28:17.875 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 83/140 2024-04-17T22:28:17.875 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 84/140 2024-04-17T22:28:17.875 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 85/140 2024-04-17T22:28:17.876 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 86/140 2024-04-17T22:28:17.876 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 87/140 2024-04-17T22:28:17.876 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 88/140 2024-04-17T22:28:17.876 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 89/140 2024-04-17T22:28:17.876 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-portend-2.6-1.el8.noarch 90/140 2024-04-17T22:28:17.876 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 91/140 2024-04-17T22:28:17.876 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 92/140 2024-04-17T22:28:17.876 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 93/140 2024-04-17T22:28:17.876 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 94/140 2024-04-17T22:28:17.876 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 95/140 2024-04-17T22:28:17.876 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 96/140 2024-04-17T22:28:17.877 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 97/140 2024-04-17T22:28:17.877 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 98/140 2024-04-17T22:28:17.877 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 99/140 2024-04-17T22:28:17.877 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 100/140 2024-04-17T22:28:17.877 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 101/140 2024-04-17T22:28:17.877 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 102/140 2024-04-17T22:28:17.877 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : librados2-2:14.2.22-0.el8.x86_64 103/140 2024-04-17T22:28:17.877 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 104/140 2024-04-17T22:28:17.877 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : librbd1-2:14.2.22-0.el8.x86_64 105/140 2024-04-17T22:28:17.877 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 106/140 2024-04-17T22:28:17.878 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : NetworkManager-1:1.40.16-15.el8.x86_64 107/140 2024-04-17T22:28:17.878 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : NetworkManager-1:1.40.16-4.el8.x86_64 108/140 2024-04-17T22:28:17.878 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : NetworkManager-libnm-1:1.40.16-15.el8.x86_64 109/140 2024-04-17T22:28:17.878 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : NetworkManager-libnm-1:1.40.16-4.el8.x86_64 110/140 2024-04-17T22:28:17.878 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : NetworkManager-team-1:1.40.16-15.el8.x86_64 111/140 2024-04-17T22:28:17.878 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : NetworkManager-team-1:1.40.16-4.el8.x86_64 112/140 2024-04-17T22:28:17.878 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : NetworkManager-tui-1:1.40.16-15.el8.x86_64 113/140 2024-04-17T22:28:17.878 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : NetworkManager-tui-1:1.40.16-4.el8.x86_64 114/140 2024-04-17T22:28:17.878 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el8.x86_64 115/140 2024-04-17T22:28:17.878 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : elfutils-debuginfod-client-0.189-1.el8.x86_64 116/140 2024-04-17T22:28:17.878 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : elfutils-libelf-0.190-2.el8.x86_64 117/140 2024-04-17T22:28:17.879 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : elfutils-libelf-0.189-1.el8.x86_64 118/140 2024-04-17T22:28:17.879 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : elfutils-libs-0.190-2.el8.x86_64 119/140 2024-04-17T22:28:17.879 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : elfutils-libs-0.189-1.el8.x86_64 120/140 2024-04-17T22:28:17.879 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : iproute-6.2.0-5.el8.x86_64 121/140 2024-04-17T22:28:17.879 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : iproute-6.2.0-1.el8.x86_64 122/140 2024-04-17T22:28:17.879 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libblkid-2.32.1-46.el8.x86_64 123/140 2024-04-17T22:28:17.879 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libblkid-2.32.1-41.el8.x86_64 124/140 2024-04-17T22:28:17.879 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libfdisk-2.32.1-46.el8.x86_64 125/140 2024-04-17T22:28:17.879 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libfdisk-2.32.1-41.el8.x86_64 126/140 2024-04-17T22:28:17.880 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libmount-2.32.1-46.el8.x86_64 127/140 2024-04-17T22:28:17.880 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libmount-2.32.1-41.el8.x86_64 128/140 2024-04-17T22:28:17.880 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libsmartcols-2.32.1-46.el8.x86_64 129/140 2024-04-17T22:28:17.880 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libsmartcols-2.32.1-41.el8.x86_64 130/140 2024-04-17T22:28:17.880 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libuuid-2.32.1-46.el8.x86_64 131/140 2024-04-17T22:28:17.880 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libuuid-2.32.1-41.el8.x86_64 132/140 2024-04-17T22:28:17.880 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : openssl-1:1.1.1k-12.el8.x86_64 133/140 2024-04-17T22:28:17.880 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : openssl-1:1.1.1k-9.el8.x86_64 134/140 2024-04-17T22:28:17.880 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : openssl-libs-1:1.1.1k-12.el8.x86_64 135/140 2024-04-17T22:28:17.880 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : openssl-libs-1:1.1.1k-9.el8.x86_64 136/140 2024-04-17T22:28:17.880 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : util-linux-2.32.1-46.el8.x86_64 137/140 2024-04-17T22:28:17.881 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : util-linux-2.32.1-41.el8.x86_64 138/140 2024-04-17T22:28:17.881 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : util-linux-user-2.32.1-46.el8.x86_64 139/140 2024-04-17T22:28:17.885 INFO:teuthology.orchestra.run.smithi102.stdout: Cleanup : NetworkManager-team-1:1.40.16-4.el8.x86_64 134/140 2024-04-17T22:28:17.895 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: NetworkManager-1:1.40.16-4.el8.x86_64 135/140 2024-04-17T22:28:17.895 INFO:teuthology.orchestra.run.smithi102.stdout: Cleanup : NetworkManager-1:1.40.16-4.el8.x86_64 135/140 2024-04-17T22:28:18.369 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: NetworkManager-1:1.40.16-4.el8.x86_64 135/140 2024-04-17T22:28:18.369 INFO:teuthology.orchestra.run.smithi102.stdout: Cleanup : NetworkManager-libnm-1:1.40.16-4.el8.x86_64 136/140 2024-04-17T22:28:18.577 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: NetworkManager-libnm-1:1.40.16-4.el8.x86_64 136/140 2024-04-17T22:28:18.577 INFO:teuthology.orchestra.run.smithi102.stdout: Cleanup : libuuid-2.32.1-41.el8.x86_64 137/140 2024-04-17T22:28:18.810 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: libuuid-2.32.1-41.el8.x86_64 137/140 2024-04-17T22:28:18.811 INFO:teuthology.orchestra.run.smithi102.stdout: Cleanup : libsmartcols-2.32.1-41.el8.x86_64 138/140 2024-04-17T22:28:18.994 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: libsmartcols-2.32.1-41.el8.x86_64 138/140 2024-04-17T22:28:19.185 INFO:teuthology.orchestra.run.smithi102.stdout: Cleanup : elfutils-libelf-0.189-1.el8.x86_64 139/140 2024-04-17T22:28:19.186 INFO:teuthology.orchestra.run.smithi102.stdout: Cleanup : openssl-libs-1:1.1.1k-9.el8.x86_64 140/140 2024-04-17T22:28:19.587 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : util-linux-user-2.32.1-41.el8.x86_64 140/140 2024-04-17T22:28:19.587 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-04-17T22:28:19.588 INFO:teuthology.orchestra.run.smithi190.stdout:Upgraded: 2024-04-17T22:28:19.588 INFO:teuthology.orchestra.run.smithi190.stdout: NetworkManager-1:1.40.16-15.el8.x86_64 2024-04-17T22:28:19.588 INFO:teuthology.orchestra.run.smithi190.stdout: NetworkManager-libnm-1:1.40.16-15.el8.x86_64 2024-04-17T22:28:19.588 INFO:teuthology.orchestra.run.smithi190.stdout: NetworkManager-team-1:1.40.16-15.el8.x86_64 2024-04-17T22:28:19.588 INFO:teuthology.orchestra.run.smithi190.stdout: NetworkManager-tui-1:1.40.16-15.el8.x86_64 2024-04-17T22:28:19.588 INFO:teuthology.orchestra.run.smithi190.stdout: elfutils-debuginfod-client-0.190-2.el8.x86_64 2024-04-17T22:28:19.588 INFO:teuthology.orchestra.run.smithi190.stdout: elfutils-libelf-0.190-2.el8.x86_64 2024-04-17T22:28:19.588 INFO:teuthology.orchestra.run.smithi190.stdout: elfutils-libs-0.190-2.el8.x86_64 2024-04-17T22:28:19.588 INFO:teuthology.orchestra.run.smithi190.stdout: iproute-6.2.0-5.el8.x86_64 2024-04-17T22:28:19.588 INFO:teuthology.orchestra.run.smithi190.stdout: libblkid-2.32.1-46.el8.x86_64 2024-04-17T22:28:19.588 INFO:teuthology.orchestra.run.smithi190.stdout: libfdisk-2.32.1-46.el8.x86_64 2024-04-17T22:28:19.588 INFO:teuthology.orchestra.run.smithi190.stdout: libmount-2.32.1-46.el8.x86_64 2024-04-17T22:28:19.588 INFO:teuthology.orchestra.run.smithi190.stdout: librados2-2:14.2.22-0.el8.x86_64 2024-04-17T22:28:19.589 INFO:teuthology.orchestra.run.smithi190.stdout: librbd1-2:14.2.22-0.el8.x86_64 2024-04-17T22:28:19.589 INFO:teuthology.orchestra.run.smithi190.stdout: libsmartcols-2.32.1-46.el8.x86_64 2024-04-17T22:28:19.589 INFO:teuthology.orchestra.run.smithi190.stdout: libuuid-2.32.1-46.el8.x86_64 2024-04-17T22:28:19.589 INFO:teuthology.orchestra.run.smithi190.stdout: openssl-1:1.1.1k-12.el8.x86_64 2024-04-17T22:28:19.589 INFO:teuthology.orchestra.run.smithi190.stdout: openssl-libs-1:1.1.1k-12.el8.x86_64 2024-04-17T22:28:19.589 INFO:teuthology.orchestra.run.smithi190.stdout: util-linux-2.32.1-46.el8.x86_64 2024-04-17T22:28:19.589 INFO:teuthology.orchestra.run.smithi190.stdout: util-linux-user-2.32.1-46.el8.x86_64 2024-04-17T22:28:19.589 INFO:teuthology.orchestra.run.smithi190.stdout:Installed: 2024-04-17T22:28:19.589 INFO:teuthology.orchestra.run.smithi190.stdout: bison-3.0.4-10.el8.x86_64 2024-04-17T22:28:19.589 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-2:14.2.22-0.el8.x86_64 2024-04-17T22:28:19.589 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-base-2:14.2.22-0.el8.x86_64 2024-04-17T22:28:19.589 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-common-2:14.2.22-0.el8.x86_64 2024-04-17T22:28:19.589 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-fuse-2:14.2.22-0.el8.x86_64 2024-04-17T22:28:19.589 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-grafana-dashboards-2:14.2.22-0.el8.noarch 2024-04-17T22:28:19.590 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mds-2:14.2.22-0.el8.x86_64 2024-04-17T22:28:19.590 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr-2:14.2.22-0.el8.x86_64 2024-04-17T22:28:19.590 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr-dashboard-2:14.2.22-0.el8.noarch 2024-04-17T22:28:19.590 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr-diskprediction-local-2:14.2.22-0.el8.noarch 2024-04-17T22:28:19.590 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr-rook-2:14.2.22-0.el8.noarch 2024-04-17T22:28:19.590 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mon-2:14.2.22-0.el8.x86_64 2024-04-17T22:28:19.590 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-osd-2:14.2.22-0.el8.x86_64 2024-04-17T22:28:19.590 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-radosgw-2:14.2.22-0.el8.x86_64 2024-04-17T22:28:19.590 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-selinux-2:14.2.22-0.el8.x86_64 2024-04-17T22:28:19.590 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-test-2:14.2.22-0.el8.x86_64 2024-04-17T22:28:19.590 INFO:teuthology.orchestra.run.smithi190.stdout: elfutils-libelf-devel-0.190-2.el8.x86_64 2024-04-17T22:28:19.590 INFO:teuthology.orchestra.run.smithi190.stdout: flex-2.6.1-9.el8.x86_64 2024-04-17T22:28:19.590 INFO:teuthology.orchestra.run.smithi190.stdout: fuse-2.9.7-19.el8.x86_64 2024-04-17T22:28:19.590 INFO:teuthology.orchestra.run.smithi190.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2024-04-17T22:28:19.591 INFO:teuthology.orchestra.run.smithi190.stdout: jq-1.6-8.el8.x86_64 2024-04-17T22:28:19.591 INFO:teuthology.orchestra.run.smithi190.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2024-04-17T22:28:19.591 INFO:teuthology.orchestra.run.smithi190.stdout: krb5-devel-1.18.2-22.el8.x86_64 2024-04-17T22:28:19.591 INFO:teuthology.orchestra.run.smithi190.stdout: leveldb-1.22-1.el8.x86_64 2024-04-17T22:28:19.591 INFO:teuthology.orchestra.run.smithi190.stdout: libcephfs-devel-2:14.2.22-0.el8.x86_64 2024-04-17T22:28:19.591 INFO:teuthology.orchestra.run.smithi190.stdout: libcephfs2-2:14.2.22-0.el8.x86_64 2024-04-17T22:28:19.591 INFO:teuthology.orchestra.run.smithi190.stdout: libcom_err-devel-1.45.6-5.el8.x86_64 2024-04-17T22:28:19.591 INFO:teuthology.orchestra.run.smithi190.stdout: libgfortran-8.5.0-21.el8.x86_64 2024-04-17T22:28:19.591 INFO:teuthology.orchestra.run.smithi190.stdout: liboath-2.6.2-3.el8.x86_64 2024-04-17T22:28:19.591 INFO:teuthology.orchestra.run.smithi190.stdout: libquadmath-8.5.0-21.el8.x86_64 2024-04-17T22:28:19.591 INFO:teuthology.orchestra.run.smithi190.stdout: librabbitmq-0.9.0-5.el8.x86_64 2024-04-17T22:28:19.591 INFO:teuthology.orchestra.run.smithi190.stdout: librados-devel-2:14.2.22-0.el8.x86_64 2024-04-17T22:28:19.591 INFO:teuthology.orchestra.run.smithi190.stdout: libradosstriper1-2:14.2.22-0.el8.x86_64 2024-04-17T22:28:19.591 INFO:teuthology.orchestra.run.smithi190.stdout: librdkafka-1.6.1-1.el8.x86_64 2024-04-17T22:28:19.592 INFO:teuthology.orchestra.run.smithi190.stdout: librgw2-2:14.2.22-0.el8.x86_64 2024-04-17T22:28:19.592 INFO:teuthology.orchestra.run.smithi190.stdout: libselinux-devel-2.9-8.el8.x86_64 2024-04-17T22:28:19.592 INFO:teuthology.orchestra.run.smithi190.stdout: libsepol-devel-2.9-3.el8.x86_64 2024-04-17T22:28:19.592 INFO:teuthology.orchestra.run.smithi190.stdout: libunwind-1.3.1-3.el8.x86_64 2024-04-17T22:28:19.592 INFO:teuthology.orchestra.run.smithi190.stdout: libverto-devel-0.3.2-2.el8.x86_64 2024-04-17T22:28:19.592 INFO:teuthology.orchestra.run.smithi190.stdout: libzstd-devel-1.4.4-1.el8.x86_64 2024-04-17T22:28:19.592 INFO:teuthology.orchestra.run.smithi190.stdout: lttng-ust-2.8.1-11.el8.x86_64 2024-04-17T22:28:19.592 INFO:teuthology.orchestra.run.smithi190.stdout: oniguruma-6.8.2-3.el8.x86_64 2024-04-17T22:28:19.592 INFO:teuthology.orchestra.run.smithi190.stdout: openblas-0.3.15-4.el8.x86_64 2024-04-17T22:28:19.592 INFO:teuthology.orchestra.run.smithi190.stdout: openblas-threads-0.3.15-4.el8.x86_64 2024-04-17T22:28:19.592 INFO:teuthology.orchestra.run.smithi190.stdout: openssl-devel-1:1.1.1k-12.el8.x86_64 2024-04-17T22:28:19.592 INFO:teuthology.orchestra.run.smithi190.stdout: pcre2-devel-10.32-3.el8.x86_64 2024-04-17T22:28:19.592 INFO:teuthology.orchestra.run.smithi190.stdout: pcre2-utf16-10.32-3.el8.x86_64 2024-04-17T22:28:19.592 INFO:teuthology.orchestra.run.smithi190.stdout: pcre2-utf32-10.32-3.el8.x86_64 2024-04-17T22:28:19.592 INFO:teuthology.orchestra.run.smithi190.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2024-04-17T22:28:19.593 INFO:teuthology.orchestra.run.smithi190.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2024-04-17T22:28:19.593 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cachetools-3.1.1-4.el8.noarch 2024-04-17T22:28:19.593 INFO:teuthology.orchestra.run.smithi190.stdout: python3-ceph-argparse-2:14.2.22-0.el8.x86_64 2024-04-17T22:28:19.593 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cephfs-2:14.2.22-0.el8.x86_64 2024-04-17T22:28:19.593 INFO:teuthology.orchestra.run.smithi190.stdout: python3-certifi-2018.10.15-7.el8.noarch 2024-04-17T22:28:19.593 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cffi-1.11.5-6.el8.x86_64 2024-04-17T22:28:19.593 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cheroot-8.5.2-1.el8.noarch 2024-04-17T22:28:19.593 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2024-04-17T22:28:19.593 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cryptography-3.2.1-7.el8.x86_64 2024-04-17T22:28:19.593 INFO:teuthology.orchestra.run.smithi190.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2024-04-17T22:28:19.593 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jaraco-6.2-6.el8.noarch 2024-04-17T22:28:19.593 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2024-04-17T22:28:19.593 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jwt-1.6.1-2.el8.noarch 2024-04-17T22:28:19.593 INFO:teuthology.orchestra.run.smithi190.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2024-04-17T22:28:19.593 INFO:teuthology.orchestra.run.smithi190.stdout: python3-logutils-0.3.5-11.el8.noarch 2024-04-17T22:28:19.594 INFO:teuthology.orchestra.run.smithi190.stdout: python3-mako-1.0.6-14.el8.noarch 2024-04-17T22:28:19.594 INFO:teuthology.orchestra.run.smithi190.stdout: python3-markupsafe-0.23-19.el8.x86_64 2024-04-17T22:28:19.594 INFO:teuthology.orchestra.run.smithi190.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2024-04-17T22:28:19.594 INFO:teuthology.orchestra.run.smithi190.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2024-04-17T22:28:19.594 INFO:teuthology.orchestra.run.smithi190.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2024-04-17T22:28:19.594 INFO:teuthology.orchestra.run.smithi190.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2024-04-17T22:28:19.594 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pecan-1.3.2-9.el8.noarch 2024-04-17T22:28:19.594 INFO:teuthology.orchestra.run.smithi190.stdout: python3-portend-2.6-1.el8.noarch 2024-04-17T22:28:19.594 INFO:teuthology.orchestra.run.smithi190.stdout: python3-prettytable-0.7.2-14.el8.noarch 2024-04-17T22:28:19.594 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2024-04-17T22:28:19.594 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2024-04-17T22:28:19.594 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2024-04-17T22:28:19.594 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pycparser-2.14-14.el8.noarch 2024-04-17T22:28:19.594 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pytz-2017.2-11.el8.noarch 2024-04-17T22:28:19.595 INFO:teuthology.orchestra.run.smithi190.stdout: python3-rados-2:14.2.22-0.el8.x86_64 2024-04-17T22:28:19.595 INFO:teuthology.orchestra.run.smithi190.stdout: python3-rbd-2:14.2.22-0.el8.x86_64 2024-04-17T22:28:19.595 INFO:teuthology.orchestra.run.smithi190.stdout: python3-repoze-lru-0.7-6.el8.noarch 2024-04-17T22:28:19.595 INFO:teuthology.orchestra.run.smithi190.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2024-04-17T22:28:19.595 INFO:teuthology.orchestra.run.smithi190.stdout: python3-rgw-2:14.2.22-0.el8.x86_64 2024-04-17T22:28:19.595 INFO:teuthology.orchestra.run.smithi190.stdout: python3-routes-2.4.1-12.el8.noarch 2024-04-17T22:28:19.595 INFO:teuthology.orchestra.run.smithi190.stdout: python3-rsa-4.9-2.el8.noarch 2024-04-17T22:28:19.595 INFO:teuthology.orchestra.run.smithi190.stdout: python3-scipy-1.0.0-21.module_el8+762+77bd8591.x86_64 2024-04-17T22:28:19.595 INFO:teuthology.orchestra.run.smithi190.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2024-04-17T22:28:19.595 INFO:teuthology.orchestra.run.smithi190.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2024-04-17T22:28:19.595 INFO:teuthology.orchestra.run.smithi190.stdout: python3-tempora-1.14.1-5.el8.noarch 2024-04-17T22:28:19.595 INFO:teuthology.orchestra.run.smithi190.stdout: python3-trustme-0.6.0-4.el8.noarch 2024-04-17T22:28:19.595 INFO:teuthology.orchestra.run.smithi190.stdout: python3-waitress-1.4.3-1.el8.noarch 2024-04-17T22:28:19.595 INFO:teuthology.orchestra.run.smithi190.stdout: python3-webob-1.8.5-1.el8.1.noarch 2024-04-17T22:28:19.596 INFO:teuthology.orchestra.run.smithi190.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2024-04-17T22:28:19.596 INFO:teuthology.orchestra.run.smithi190.stdout: python3-webtest-2.0.33-1.el8.noarch 2024-04-17T22:28:19.596 INFO:teuthology.orchestra.run.smithi190.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2024-04-17T22:28:19.596 INFO:teuthology.orchestra.run.smithi190.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2024-04-17T22:28:19.596 INFO:teuthology.orchestra.run.smithi190.stdout: rbd-fuse-2:14.2.22-0.el8.x86_64 2024-04-17T22:28:19.596 INFO:teuthology.orchestra.run.smithi190.stdout: rbd-mirror-2:14.2.22-0.el8.x86_64 2024-04-17T22:28:19.596 INFO:teuthology.orchestra.run.smithi190.stdout: rbd-nbd-2:14.2.22-0.el8.x86_64 2024-04-17T22:28:19.596 INFO:teuthology.orchestra.run.smithi190.stdout: socat-1.7.4.1-1.el8.x86_64 2024-04-17T22:28:19.596 INFO:teuthology.orchestra.run.smithi190.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2024-04-17T22:28:19.596 INFO:teuthology.orchestra.run.smithi190.stdout: zlib-devel-1.2.11-21.el8.x86_64 2024-04-17T22:28:19.597 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-04-17T22:28:19.597 INFO:teuthology.orchestra.run.smithi190.stdout:Complete! 2024-04-17T22:28:19.799 DEBUG:teuthology.parallel:result is None 2024-04-17T22:28:20.050 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: openssl-libs-1:1.1.1k-9.el8.x86_64 140/140 2024-04-17T22:28:20.051 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-2:14.2.22-0.el8.x86_64 1/140 2024-04-17T22:28:20.051 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-base-2:14.2.22-0.el8.x86_64 2/140 2024-04-17T22:28:20.051 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-common-2:14.2.22-0.el8.x86_64 3/140 2024-04-17T22:28:20.051 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-fuse-2:14.2.22-0.el8.x86_64 4/140 2024-04-17T22:28:20.051 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-mds-2:14.2.22-0.el8.x86_64 5/140 2024-04-17T22:28:20.051 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-mgr-2:14.2.22-0.el8.x86_64 6/140 2024-04-17T22:28:20.051 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-mon-2:14.2.22-0.el8.x86_64 7/140 2024-04-17T22:28:20.051 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-osd-2:14.2.22-0.el8.x86_64 8/140 2024-04-17T22:28:20.051 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-radosgw-2:14.2.22-0.el8.x86_64 9/140 2024-04-17T22:28:20.051 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-selinux-2:14.2.22-0.el8.x86_64 10/140 2024-04-17T22:28:20.051 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-test-2:14.2.22-0.el8.x86_64 11/140 2024-04-17T22:28:20.052 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libcephfs-devel-2:14.2.22-0.el8.x86_64 12/140 2024-04-17T22:28:20.052 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libcephfs2-2:14.2.22-0.el8.x86_64 13/140 2024-04-17T22:28:20.052 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : librados-devel-2:14.2.22-0.el8.x86_64 14/140 2024-04-17T22:28:20.052 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libradosstriper1-2:14.2.22-0.el8.x86_64 15/140 2024-04-17T22:28:20.052 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : librgw2-2:14.2.22-0.el8.x86_64 16/140 2024-04-17T22:28:20.052 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-ceph-argparse-2:14.2.22-0.el8.x86_64 17/140 2024-04-17T22:28:20.052 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-cephfs-2:14.2.22-0.el8.x86_64 18/140 2024-04-17T22:28:20.052 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-rados-2:14.2.22-0.el8.x86_64 19/140 2024-04-17T22:28:20.052 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-rbd-2:14.2.22-0.el8.x86_64 20/140 2024-04-17T22:28:20.052 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-rgw-2:14.2.22-0.el8.x86_64 21/140 2024-04-17T22:28:20.052 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : rbd-fuse-2:14.2.22-0.el8.x86_64 22/140 2024-04-17T22:28:20.052 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : rbd-mirror-2:14.2.22-0.el8.x86_64 23/140 2024-04-17T22:28:20.053 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : rbd-nbd-2:14.2.22-0.el8.x86_64 24/140 2024-04-17T22:28:20.053 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-grafana-dashboards-2:14.2.22-0.el8.noarch 25/140 2024-04-17T22:28:20.053 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-mgr-dashboard-2:14.2.22-0.el8.noarch 26/140 2024-04-17T22:28:20.053 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-mgr-diskprediction-local-2:14.2.22-0.el8.no 27/140 2024-04-17T22:28:20.053 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-mgr-rook-2:14.2.22-0.el8.noarch 28/140 2024-04-17T22:28:20.053 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : bison-3.0.4-10.el8.x86_64 29/140 2024-04-17T22:28:20.053 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : flex-2.6.1-9.el8.x86_64 30/140 2024-04-17T22:28:20.053 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : jq-1.6-8.el8.x86_64 31/140 2024-04-17T22:28:20.053 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : librdkafka-1.6.1-1.el8.x86_64 32/140 2024-04-17T22:28:20.053 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 33/140 2024-04-17T22:28:20.053 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : oniguruma-6.8.2-3.el8.x86_64 34/140 2024-04-17T22:28:20.053 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : openblas-0.3.15-4.el8.x86_64 35/140 2024-04-17T22:28:20.054 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : openblas-threads-0.3.15-4.el8.x86_64 36/140 2024-04-17T22:28:20.054 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-mako-1.0.6-14.el8.noarch 37/140 2024-04-17T22:28:20.054 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 38/140 2024-04-17T22:28:20.054 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 39/140 2024-04-17T22:28:20.054 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 40/140 2024-04-17T22:28:20.054 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 41/140 2024-04-17T22:28:20.054 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 42/140 2024-04-17T22:28:20.054 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 43/140 2024-04-17T22:28:20.054 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 44/140 2024-04-17T22:28:20.054 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-pytz-2017.2-11.el8.noarch 45/140 2024-04-17T22:28:20.054 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-scipy-1.0.0-21.module_el8+762+77bd8591.x 46/140 2024-04-17T22:28:20.055 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 47/140 2024-04-17T22:28:20.055 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 48/140 2024-04-17T22:28:20.055 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 49/140 2024-04-17T22:28:20.055 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : elfutils-libelf-devel-0.190-2.el8.x86_64 50/140 2024-04-17T22:28:20.055 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : fuse-2.9.7-19.el8.x86_64 51/140 2024-04-17T22:28:20.055 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 52/140 2024-04-17T22:28:20.055 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : krb5-devel-1.18.2-22.el8.x86_64 53/140 2024-04-17T22:28:20.055 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libcom_err-devel-1.45.6-5.el8.x86_64 54/140 2024-04-17T22:28:20.055 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libgfortran-8.5.0-21.el8.x86_64 55/140 2024-04-17T22:28:20.055 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libquadmath-8.5.0-21.el8.x86_64 56/140 2024-04-17T22:28:20.055 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : librabbitmq-0.9.0-5.el8.x86_64 57/140 2024-04-17T22:28:20.055 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libselinux-devel-2.9-8.el8.x86_64 58/140 2024-04-17T22:28:20.056 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 59/140 2024-04-17T22:28:20.056 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libverto-devel-0.3.2-2.el8.x86_64 60/140 2024-04-17T22:28:20.056 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libzstd-devel-1.4.4-1.el8.x86_64 61/140 2024-04-17T22:28:20.056 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : openssl-devel-1:1.1.1k-12.el8.x86_64 62/140 2024-04-17T22:28:20.056 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : pcre2-devel-10.32-3.el8.x86_64 63/140 2024-04-17T22:28:20.056 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : pcre2-utf16-10.32-3.el8.x86_64 64/140 2024-04-17T22:28:20.056 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : pcre2-utf32-10.32-3.el8.x86_64 65/140 2024-04-17T22:28:20.056 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-cffi-1.11.5-6.el8.x86_64 66/140 2024-04-17T22:28:20.056 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-cryptography-3.2.1-7.el8.x86_64 67/140 2024-04-17T22:28:20.056 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 68/140 2024-04-17T22:28:20.056 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 69/140 2024-04-17T22:28:20.056 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 70/140 2024-04-17T22:28:20.057 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 71/140 2024-04-17T22:28:20.057 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : zlib-devel-1.2.11-21.el8.x86_64 72/140 2024-04-17T22:28:20.057 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 73/140 2024-04-17T22:28:20.057 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : leveldb-1.22-1.el8.x86_64 74/140 2024-04-17T22:28:20.057 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 75/140 2024-04-17T22:28:20.057 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 76/140 2024-04-17T22:28:20.057 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 77/140 2024-04-17T22:28:20.057 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 78/140 2024-04-17T22:28:20.057 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 79/140 2024-04-17T22:28:20.058 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 80/140 2024-04-17T22:28:20.058 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 81/140 2024-04-17T22:28:20.058 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 82/140 2024-04-17T22:28:20.058 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 83/140 2024-04-17T22:28:20.058 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 84/140 2024-04-17T22:28:20.058 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 85/140 2024-04-17T22:28:20.058 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 86/140 2024-04-17T22:28:20.058 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 87/140 2024-04-17T22:28:20.058 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 88/140 2024-04-17T22:28:20.058 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 89/140 2024-04-17T22:28:20.058 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-portend-2.6-1.el8.noarch 90/140 2024-04-17T22:28:20.059 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 91/140 2024-04-17T22:28:20.059 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 92/140 2024-04-17T22:28:20.059 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 93/140 2024-04-17T22:28:20.059 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 94/140 2024-04-17T22:28:20.059 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 95/140 2024-04-17T22:28:20.059 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 96/140 2024-04-17T22:28:20.059 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 97/140 2024-04-17T22:28:20.059 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 98/140 2024-04-17T22:28:20.059 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 99/140 2024-04-17T22:28:20.059 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 100/140 2024-04-17T22:28:20.059 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 101/140 2024-04-17T22:28:20.059 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 102/140 2024-04-17T22:28:20.060 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : librados2-2:14.2.22-0.el8.x86_64 103/140 2024-04-17T22:28:20.060 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 104/140 2024-04-17T22:28:20.060 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : librbd1-2:14.2.22-0.el8.x86_64 105/140 2024-04-17T22:28:20.060 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 106/140 2024-04-17T22:28:20.060 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : NetworkManager-1:1.40.16-15.el8.x86_64 107/140 2024-04-17T22:28:20.060 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : NetworkManager-1:1.40.16-4.el8.x86_64 108/140 2024-04-17T22:28:20.060 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : NetworkManager-libnm-1:1.40.16-15.el8.x86_64 109/140 2024-04-17T22:28:20.060 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : NetworkManager-libnm-1:1.40.16-4.el8.x86_64 110/140 2024-04-17T22:28:20.060 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : NetworkManager-team-1:1.40.16-15.el8.x86_64 111/140 2024-04-17T22:28:20.060 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : NetworkManager-team-1:1.40.16-4.el8.x86_64 112/140 2024-04-17T22:28:20.060 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : NetworkManager-tui-1:1.40.16-15.el8.x86_64 113/140 2024-04-17T22:28:20.060 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : NetworkManager-tui-1:1.40.16-4.el8.x86_64 114/140 2024-04-17T22:28:20.060 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el8.x86_64 115/140 2024-04-17T22:28:20.061 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : elfutils-debuginfod-client-0.189-1.el8.x86_64 116/140 2024-04-17T22:28:20.061 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : elfutils-libelf-0.190-2.el8.x86_64 117/140 2024-04-17T22:28:20.061 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : elfutils-libelf-0.189-1.el8.x86_64 118/140 2024-04-17T22:28:20.061 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : elfutils-libs-0.190-2.el8.x86_64 119/140 2024-04-17T22:28:20.061 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : elfutils-libs-0.189-1.el8.x86_64 120/140 2024-04-17T22:28:20.061 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : iproute-6.2.0-5.el8.x86_64 121/140 2024-04-17T22:28:20.061 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : iproute-6.2.0-1.el8.x86_64 122/140 2024-04-17T22:28:20.061 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libblkid-2.32.1-46.el8.x86_64 123/140 2024-04-17T22:28:20.061 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libblkid-2.32.1-41.el8.x86_64 124/140 2024-04-17T22:28:20.061 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libfdisk-2.32.1-46.el8.x86_64 125/140 2024-04-17T22:28:20.061 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libfdisk-2.32.1-41.el8.x86_64 126/140 2024-04-17T22:28:20.061 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libmount-2.32.1-46.el8.x86_64 127/140 2024-04-17T22:28:20.061 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libmount-2.32.1-41.el8.x86_64 128/140 2024-04-17T22:28:20.062 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libsmartcols-2.32.1-46.el8.x86_64 129/140 2024-04-17T22:28:20.062 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libsmartcols-2.32.1-41.el8.x86_64 130/140 2024-04-17T22:28:20.062 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libuuid-2.32.1-46.el8.x86_64 131/140 2024-04-17T22:28:20.062 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libuuid-2.32.1-41.el8.x86_64 132/140 2024-04-17T22:28:20.062 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : openssl-1:1.1.1k-12.el8.x86_64 133/140 2024-04-17T22:28:20.062 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : openssl-1:1.1.1k-9.el8.x86_64 134/140 2024-04-17T22:28:20.062 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : openssl-libs-1:1.1.1k-12.el8.x86_64 135/140 2024-04-17T22:28:20.062 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : openssl-libs-1:1.1.1k-9.el8.x86_64 136/140 2024-04-17T22:28:20.062 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : util-linux-2.32.1-46.el8.x86_64 137/140 2024-04-17T22:28:20.062 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : util-linux-2.32.1-41.el8.x86_64 138/140 2024-04-17T22:28:20.062 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : util-linux-user-2.32.1-46.el8.x86_64 139/140 2024-04-17T22:28:21.811 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : util-linux-user-2.32.1-41.el8.x86_64 140/140 2024-04-17T22:28:21.811 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-17T22:28:21.811 INFO:teuthology.orchestra.run.smithi102.stdout:Upgraded: 2024-04-17T22:28:21.811 INFO:teuthology.orchestra.run.smithi102.stdout: NetworkManager-1:1.40.16-15.el8.x86_64 2024-04-17T22:28:21.811 INFO:teuthology.orchestra.run.smithi102.stdout: NetworkManager-libnm-1:1.40.16-15.el8.x86_64 2024-04-17T22:28:21.811 INFO:teuthology.orchestra.run.smithi102.stdout: NetworkManager-team-1:1.40.16-15.el8.x86_64 2024-04-17T22:28:21.811 INFO:teuthology.orchestra.run.smithi102.stdout: NetworkManager-tui-1:1.40.16-15.el8.x86_64 2024-04-17T22:28:21.811 INFO:teuthology.orchestra.run.smithi102.stdout: elfutils-debuginfod-client-0.190-2.el8.x86_64 2024-04-17T22:28:21.811 INFO:teuthology.orchestra.run.smithi102.stdout: elfutils-libelf-0.190-2.el8.x86_64 2024-04-17T22:28:21.811 INFO:teuthology.orchestra.run.smithi102.stdout: elfutils-libs-0.190-2.el8.x86_64 2024-04-17T22:28:21.811 INFO:teuthology.orchestra.run.smithi102.stdout: iproute-6.2.0-5.el8.x86_64 2024-04-17T22:28:21.811 INFO:teuthology.orchestra.run.smithi102.stdout: libblkid-2.32.1-46.el8.x86_64 2024-04-17T22:28:21.812 INFO:teuthology.orchestra.run.smithi102.stdout: libfdisk-2.32.1-46.el8.x86_64 2024-04-17T22:28:21.812 INFO:teuthology.orchestra.run.smithi102.stdout: libmount-2.32.1-46.el8.x86_64 2024-04-17T22:28:21.812 INFO:teuthology.orchestra.run.smithi102.stdout: librados2-2:14.2.22-0.el8.x86_64 2024-04-17T22:28:21.812 INFO:teuthology.orchestra.run.smithi102.stdout: librbd1-2:14.2.22-0.el8.x86_64 2024-04-17T22:28:21.812 INFO:teuthology.orchestra.run.smithi102.stdout: libsmartcols-2.32.1-46.el8.x86_64 2024-04-17T22:28:21.812 INFO:teuthology.orchestra.run.smithi102.stdout: libuuid-2.32.1-46.el8.x86_64 2024-04-17T22:28:21.812 INFO:teuthology.orchestra.run.smithi102.stdout: openssl-1:1.1.1k-12.el8.x86_64 2024-04-17T22:28:21.812 INFO:teuthology.orchestra.run.smithi102.stdout: openssl-libs-1:1.1.1k-12.el8.x86_64 2024-04-17T22:28:21.812 INFO:teuthology.orchestra.run.smithi102.stdout: util-linux-2.32.1-46.el8.x86_64 2024-04-17T22:28:21.812 INFO:teuthology.orchestra.run.smithi102.stdout: util-linux-user-2.32.1-46.el8.x86_64 2024-04-17T22:28:21.812 INFO:teuthology.orchestra.run.smithi102.stdout:Installed: 2024-04-17T22:28:21.812 INFO:teuthology.orchestra.run.smithi102.stdout: bison-3.0.4-10.el8.x86_64 2024-04-17T22:28:21.812 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-2:14.2.22-0.el8.x86_64 2024-04-17T22:28:21.812 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-base-2:14.2.22-0.el8.x86_64 2024-04-17T22:28:21.813 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-common-2:14.2.22-0.el8.x86_64 2024-04-17T22:28:21.813 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-fuse-2:14.2.22-0.el8.x86_64 2024-04-17T22:28:21.813 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-grafana-dashboards-2:14.2.22-0.el8.noarch 2024-04-17T22:28:21.813 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mds-2:14.2.22-0.el8.x86_64 2024-04-17T22:28:21.813 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mgr-2:14.2.22-0.el8.x86_64 2024-04-17T22:28:21.813 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mgr-dashboard-2:14.2.22-0.el8.noarch 2024-04-17T22:28:21.813 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mgr-diskprediction-local-2:14.2.22-0.el8.noarch 2024-04-17T22:28:21.813 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mgr-rook-2:14.2.22-0.el8.noarch 2024-04-17T22:28:21.813 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mon-2:14.2.22-0.el8.x86_64 2024-04-17T22:28:21.813 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-osd-2:14.2.22-0.el8.x86_64 2024-04-17T22:28:21.813 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-radosgw-2:14.2.22-0.el8.x86_64 2024-04-17T22:28:21.813 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-selinux-2:14.2.22-0.el8.x86_64 2024-04-17T22:28:21.813 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-test-2:14.2.22-0.el8.x86_64 2024-04-17T22:28:21.813 INFO:teuthology.orchestra.run.smithi102.stdout: elfutils-libelf-devel-0.190-2.el8.x86_64 2024-04-17T22:28:21.813 INFO:teuthology.orchestra.run.smithi102.stdout: flex-2.6.1-9.el8.x86_64 2024-04-17T22:28:21.814 INFO:teuthology.orchestra.run.smithi102.stdout: fuse-2.9.7-19.el8.x86_64 2024-04-17T22:28:21.814 INFO:teuthology.orchestra.run.smithi102.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2024-04-17T22:28:21.814 INFO:teuthology.orchestra.run.smithi102.stdout: jq-1.6-8.el8.x86_64 2024-04-17T22:28:21.814 INFO:teuthology.orchestra.run.smithi102.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2024-04-17T22:28:21.814 INFO:teuthology.orchestra.run.smithi102.stdout: krb5-devel-1.18.2-22.el8.x86_64 2024-04-17T22:28:21.814 INFO:teuthology.orchestra.run.smithi102.stdout: leveldb-1.22-1.el8.x86_64 2024-04-17T22:28:21.814 INFO:teuthology.orchestra.run.smithi102.stdout: libcephfs-devel-2:14.2.22-0.el8.x86_64 2024-04-17T22:28:21.814 INFO:teuthology.orchestra.run.smithi102.stdout: libcephfs2-2:14.2.22-0.el8.x86_64 2024-04-17T22:28:21.814 INFO:teuthology.orchestra.run.smithi102.stdout: libcom_err-devel-1.45.6-5.el8.x86_64 2024-04-17T22:28:21.814 INFO:teuthology.orchestra.run.smithi102.stdout: libgfortran-8.5.0-21.el8.x86_64 2024-04-17T22:28:21.814 INFO:teuthology.orchestra.run.smithi102.stdout: liboath-2.6.2-3.el8.x86_64 2024-04-17T22:28:21.814 INFO:teuthology.orchestra.run.smithi102.stdout: libquadmath-8.5.0-21.el8.x86_64 2024-04-17T22:28:21.814 INFO:teuthology.orchestra.run.smithi102.stdout: librabbitmq-0.9.0-5.el8.x86_64 2024-04-17T22:28:21.814 INFO:teuthology.orchestra.run.smithi102.stdout: librados-devel-2:14.2.22-0.el8.x86_64 2024-04-17T22:28:21.815 INFO:teuthology.orchestra.run.smithi102.stdout: libradosstriper1-2:14.2.22-0.el8.x86_64 2024-04-17T22:28:21.815 INFO:teuthology.orchestra.run.smithi102.stdout: librdkafka-1.6.1-1.el8.x86_64 2024-04-17T22:28:21.815 INFO:teuthology.orchestra.run.smithi102.stdout: librgw2-2:14.2.22-0.el8.x86_64 2024-04-17T22:28:21.815 INFO:teuthology.orchestra.run.smithi102.stdout: libselinux-devel-2.9-8.el8.x86_64 2024-04-17T22:28:21.815 INFO:teuthology.orchestra.run.smithi102.stdout: libsepol-devel-2.9-3.el8.x86_64 2024-04-17T22:28:21.815 INFO:teuthology.orchestra.run.smithi102.stdout: libunwind-1.3.1-3.el8.x86_64 2024-04-17T22:28:21.815 INFO:teuthology.orchestra.run.smithi102.stdout: libverto-devel-0.3.2-2.el8.x86_64 2024-04-17T22:28:21.815 INFO:teuthology.orchestra.run.smithi102.stdout: libzstd-devel-1.4.4-1.el8.x86_64 2024-04-17T22:28:21.815 INFO:teuthology.orchestra.run.smithi102.stdout: lttng-ust-2.8.1-11.el8.x86_64 2024-04-17T22:28:21.815 INFO:teuthology.orchestra.run.smithi102.stdout: oniguruma-6.8.2-3.el8.x86_64 2024-04-17T22:28:21.815 INFO:teuthology.orchestra.run.smithi102.stdout: openblas-0.3.15-4.el8.x86_64 2024-04-17T22:28:21.815 INFO:teuthology.orchestra.run.smithi102.stdout: openblas-threads-0.3.15-4.el8.x86_64 2024-04-17T22:28:21.815 INFO:teuthology.orchestra.run.smithi102.stdout: openssl-devel-1:1.1.1k-12.el8.x86_64 2024-04-17T22:28:21.815 INFO:teuthology.orchestra.run.smithi102.stdout: pcre2-devel-10.32-3.el8.x86_64 2024-04-17T22:28:21.816 INFO:teuthology.orchestra.run.smithi102.stdout: pcre2-utf16-10.32-3.el8.x86_64 2024-04-17T22:28:21.816 INFO:teuthology.orchestra.run.smithi102.stdout: pcre2-utf32-10.32-3.el8.x86_64 2024-04-17T22:28:21.816 INFO:teuthology.orchestra.run.smithi102.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2024-04-17T22:28:21.816 INFO:teuthology.orchestra.run.smithi102.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2024-04-17T22:28:21.816 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cachetools-3.1.1-4.el8.noarch 2024-04-17T22:28:21.816 INFO:teuthology.orchestra.run.smithi102.stdout: python3-ceph-argparse-2:14.2.22-0.el8.x86_64 2024-04-17T22:28:21.816 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cephfs-2:14.2.22-0.el8.x86_64 2024-04-17T22:28:21.816 INFO:teuthology.orchestra.run.smithi102.stdout: python3-certifi-2018.10.15-7.el8.noarch 2024-04-17T22:28:21.816 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cffi-1.11.5-6.el8.x86_64 2024-04-17T22:28:21.816 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cheroot-8.5.2-1.el8.noarch 2024-04-17T22:28:21.816 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2024-04-17T22:28:21.816 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cryptography-3.2.1-7.el8.x86_64 2024-04-17T22:28:21.817 INFO:teuthology.orchestra.run.smithi102.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2024-04-17T22:28:21.817 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jaraco-6.2-6.el8.noarch 2024-04-17T22:28:21.817 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2024-04-17T22:28:21.817 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jwt-1.6.1-2.el8.noarch 2024-04-17T22:28:21.817 INFO:teuthology.orchestra.run.smithi102.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2024-04-17T22:28:21.817 INFO:teuthology.orchestra.run.smithi102.stdout: python3-logutils-0.3.5-11.el8.noarch 2024-04-17T22:28:21.817 INFO:teuthology.orchestra.run.smithi102.stdout: python3-mako-1.0.6-14.el8.noarch 2024-04-17T22:28:21.817 INFO:teuthology.orchestra.run.smithi102.stdout: python3-markupsafe-0.23-19.el8.x86_64 2024-04-17T22:28:21.817 INFO:teuthology.orchestra.run.smithi102.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2024-04-17T22:28:21.817 INFO:teuthology.orchestra.run.smithi102.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2024-04-17T22:28:21.817 INFO:teuthology.orchestra.run.smithi102.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2024-04-17T22:28:21.817 INFO:teuthology.orchestra.run.smithi102.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2024-04-17T22:28:21.817 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pecan-1.3.2-9.el8.noarch 2024-04-17T22:28:21.818 INFO:teuthology.orchestra.run.smithi102.stdout: python3-portend-2.6-1.el8.noarch 2024-04-17T22:28:21.818 INFO:teuthology.orchestra.run.smithi102.stdout: python3-prettytable-0.7.2-14.el8.noarch 2024-04-17T22:28:21.818 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2024-04-17T22:28:21.818 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2024-04-17T22:28:21.818 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2024-04-17T22:28:21.818 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pycparser-2.14-14.el8.noarch 2024-04-17T22:28:21.818 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pytz-2017.2-11.el8.noarch 2024-04-17T22:28:21.818 INFO:teuthology.orchestra.run.smithi102.stdout: python3-rados-2:14.2.22-0.el8.x86_64 2024-04-17T22:28:21.818 INFO:teuthology.orchestra.run.smithi102.stdout: python3-rbd-2:14.2.22-0.el8.x86_64 2024-04-17T22:28:21.818 INFO:teuthology.orchestra.run.smithi102.stdout: python3-repoze-lru-0.7-6.el8.noarch 2024-04-17T22:28:21.818 INFO:teuthology.orchestra.run.smithi102.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2024-04-17T22:28:21.818 INFO:teuthology.orchestra.run.smithi102.stdout: python3-rgw-2:14.2.22-0.el8.x86_64 2024-04-17T22:28:21.818 INFO:teuthology.orchestra.run.smithi102.stdout: python3-routes-2.4.1-12.el8.noarch 2024-04-17T22:28:21.818 INFO:teuthology.orchestra.run.smithi102.stdout: python3-rsa-4.9-2.el8.noarch 2024-04-17T22:28:21.819 INFO:teuthology.orchestra.run.smithi102.stdout: python3-scipy-1.0.0-21.module_el8+762+77bd8591.x86_64 2024-04-17T22:28:21.819 INFO:teuthology.orchestra.run.smithi102.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2024-04-17T22:28:21.819 INFO:teuthology.orchestra.run.smithi102.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2024-04-17T22:28:21.819 INFO:teuthology.orchestra.run.smithi102.stdout: python3-tempora-1.14.1-5.el8.noarch 2024-04-17T22:28:21.819 INFO:teuthology.orchestra.run.smithi102.stdout: python3-trustme-0.6.0-4.el8.noarch 2024-04-17T22:28:21.819 INFO:teuthology.orchestra.run.smithi102.stdout: python3-waitress-1.4.3-1.el8.noarch 2024-04-17T22:28:21.819 INFO:teuthology.orchestra.run.smithi102.stdout: python3-webob-1.8.5-1.el8.1.noarch 2024-04-17T22:28:21.819 INFO:teuthology.orchestra.run.smithi102.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2024-04-17T22:28:21.819 INFO:teuthology.orchestra.run.smithi102.stdout: python3-webtest-2.0.33-1.el8.noarch 2024-04-17T22:28:21.819 INFO:teuthology.orchestra.run.smithi102.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2024-04-17T22:28:21.819 INFO:teuthology.orchestra.run.smithi102.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2024-04-17T22:28:21.819 INFO:teuthology.orchestra.run.smithi102.stdout: rbd-fuse-2:14.2.22-0.el8.x86_64 2024-04-17T22:28:21.819 INFO:teuthology.orchestra.run.smithi102.stdout: rbd-mirror-2:14.2.22-0.el8.x86_64 2024-04-17T22:28:21.819 INFO:teuthology.orchestra.run.smithi102.stdout: rbd-nbd-2:14.2.22-0.el8.x86_64 2024-04-17T22:28:21.820 INFO:teuthology.orchestra.run.smithi102.stdout: socat-1.7.4.1-1.el8.x86_64 2024-04-17T22:28:21.820 INFO:teuthology.orchestra.run.smithi102.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2024-04-17T22:28:21.820 INFO:teuthology.orchestra.run.smithi102.stdout: zlib-devel-1.2.11-21.el8.x86_64 2024-04-17T22:28:21.820 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-17T22:28:21.820 INFO:teuthology.orchestra.run.smithi102.stdout:Complete! 2024-04-17T22:28:22.035 DEBUG:teuthology.parallel:result is None 2024-04-17T22:28:22.035 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-17T22:28:22.035 INFO:teuthology.packaging:ref: None 2024-04-17T22:28:22.035 INFO:teuthology.packaging:tag: None 2024-04-17T22:28:22.035 INFO:teuthology.packaging:branch: nautilus 2024-04-17T22:28:22.036 INFO:teuthology.packaging:sha1: 41c500dd60c93c8d83053b72cffc3dd1a952a336 2024-04-17T22:28:22.036 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=nautilus 2024-04-17T22:28:22.282 DEBUG:teuthology.orchestra.run.smithi102:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-17T22:28:22.307 INFO:teuthology.orchestra.run.smithi102.stdout:14.2.22-0.el8 2024-04-17T22:28:22.307 INFO:teuthology.packaging:The installed version of ceph is 14.2.22-0.el8 2024-04-17T22:28:22.307 INFO:teuthology.task.install:The correct ceph version 14.2.22-0 is installed. 2024-04-17T22:28:22.309 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-17T22:28:22.309 INFO:teuthology.packaging:ref: None 2024-04-17T22:28:22.309 INFO:teuthology.packaging:tag: None 2024-04-17T22:28:22.309 INFO:teuthology.packaging:branch: nautilus 2024-04-17T22:28:22.309 INFO:teuthology.packaging:sha1: 41c500dd60c93c8d83053b72cffc3dd1a952a336 2024-04-17T22:28:22.310 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=nautilus 2024-04-17T22:28:22.475 DEBUG:teuthology.orchestra.run.smithi190:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-17T22:28:22.503 INFO:teuthology.orchestra.run.smithi190.stdout:14.2.22-0.el8 2024-04-17T22:28:22.503 INFO:teuthology.packaging:The installed version of ceph is 14.2.22-0.el8 2024-04-17T22:28:22.504 INFO:teuthology.task.install:The correct ceph version 14.2.22-0 is installed. 2024-04-17T22:28:22.505 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-17T22:28:22.506 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-04-17T22:28:22.506 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-17T22:28:22.535 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-04-17T22:28:22.535 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-17T22:28:22.578 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-17T22:28:22.578 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-04-17T22:28:22.578 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd of=/usr/bin/daemon-helper 2024-04-17T22:28:22.604 DEBUG:teuthology.orchestra.run.smithi102:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-17T22:28:22.671 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-04-17T22:28:22.672 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd of=/usr/bin/daemon-helper 2024-04-17T22:28:22.700 DEBUG:teuthology.orchestra.run.smithi190:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-17T22:28:22.769 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-17T22:28:22.770 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-04-17T22:28:22.770 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-17T22:28:22.797 DEBUG:teuthology.orchestra.run.smithi102:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-17T22:28:22.863 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-04-17T22:28:22.864 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-17T22:28:22.892 DEBUG:teuthology.orchestra.run.smithi190:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-17T22:28:22.959 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-17T22:28:22.959 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-04-17T22:28:22.959 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd of=/usr/bin/stdin-killer 2024-04-17T22:28:22.985 DEBUG:teuthology.orchestra.run.smithi102:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-17T22:28:23.051 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-04-17T22:28:23.051 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd of=/usr/bin/stdin-killer 2024-04-17T22:28:23.077 DEBUG:teuthology.orchestra.run.smithi190:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-17T22:28:23.144 INFO:teuthology.run_tasks:Running task print... 2024-04-17T22:28:23.153 INFO:teuthology.task.print:**** done installing nautilus 2024-04-17T22:28:23.154 INFO:teuthology.run_tasks:Running task ceph... 2024-04-17T22:28:23.265 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-04-17T22:28:23.265 DEBUG:teuthology.orchestra.run.smithi102:> sudo chmod 777 /var/log/ceph 2024-04-17T22:28:23.267 DEBUG:teuthology.orchestra.run.smithi190:> sudo chmod 777 /var/log/ceph 2024-04-17T22:28:23.293 INFO:tasks.ceph:Disabling ceph logrotate... 2024-04-17T22:28:23.293 DEBUG:teuthology.orchestra.run.smithi102:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-17T22:28:23.333 DEBUG:teuthology.orchestra.run.smithi190:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-17T22:28:23.360 INFO:tasks.ceph:Creating extra log directories... 2024-04-17T22:28:23.360 DEBUG:teuthology.orchestra.run.smithi102:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-17T22:28:23.399 DEBUG:teuthology.orchestra.run.smithi190:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-17T22:28:23.436 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-04-17T22:28:23.436 INFO:tasks.ceph:config {'conf': {'global': {'mon warn on pool no app': False, 'ms bind msgr2': False, 'mon pg warn min per osd': 0}, 'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['overall HEALTH_', '\\(FS_', '\\(MDS_', '\\(OSD_', '\\(MON_DOWN\\)', '\\(CACHE_POOL_', '\\(POOL_', '\\(MGR_DOWN\\)', '\\(PG_', '\\(SMALLER_PGP_NUM\\)', 'Monitor daemon marked osd', 'Behind on trimming', 'Manager daemon', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-04-17T22:28:23.437 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/teuthology-2024-04-17_21:24:02-fs-squid-distro-default-smithi/7661154', 'branch': 'squid', 'description': 'fs/upgrade/upgraded_client/{bluestore-bitmap branch/nautilus centos_8.stream clusters/{1-mds-1-client-micro} conf/{client mds mgr mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} tasks/{0-install 1-mount/mount/fuse 2-clients/fuse-upgrade 3-workload/new_ops/{newops}}}', 'email': 'ceph-qa@ceph.com', 'first_in_suite': False, 'job_id': '7661154', 'kernel': {'kdb': 1, 'sha1': 'distro'}, 'ktype': 'distro', 'last_in_suite': False, 'machine_type': 'smithi', 'name': 'teuthology-2024-04-17_21:24:02-fs-squid-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 10}}], 'os_type': 'centos', 'os_version': '8.stream', 'overrides': {'admin_socket': {'branch': 'squid'}, 'ceph': {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'mon pg warn min per osd': 0}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN'], 'sha1': '41c500dd60c93c8d83053b72cffc3dd1a952a336'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '41c500dd60c93c8d83053b72cffc3dd1a952a336'}}, 'selinux': {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'squid', 'sha1': '1a6a3e742fd81c6d70be70552499bff767652c32'}}, 'owner': 'scheduled_teuthology@teuthology', 'priority': 100, 'repo': 'https://git.ceph.com/ceph.git', 'roles': [['mon.a', 'mon.b', 'mon.c', 'mgr.x', 'mds.a', 'osd.0', 'osd.1', 'osd.2', 'osd.3'], ['client.0']], 'seed': 827, 'sha1': '41c500dd60c93c8d83053b72cffc3dd1a952a336', 'sleep_before_teardown': 0, 'subset': '175/512', 'suite': 'fs', 'suite_branch': 'squid', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph_1a6a3e742fd81c6d70be70552499bff767652c32/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph.git', 'suite_sha1': '1a6a3e742fd81c6d70be70552499bff767652c32', 'targets': {'smithi102.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLRJ9qjqTzlTgwiH95uehu4ogBsUfPt8b9MSsgdDu7C2GB9dG25LSJ7HKVK0llq9OfhNJNysQyHz9rcmeUiReok=', 'smithi190.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHYg7CPsZA5CTSHbhqLlD7ulb3ycZN3tg4NvwWBAkpq9tcby13wkhr0sn0M+peU5FOfF7oYZaVTx0DYXHXSUxtk='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'branch': 'nautilus', 'exclude_packages': ['cephadm', 'ceph-mgr-cephadm', 'ceph-immutable-object-cache', 'python3-rados', 'python3-rgw', 'python3-rbd', 'python3-cephfs', 'ceph-volume'], 'extra_packages': ['python-rados', 'python-rgw', 'python-rbd', 'python-cephfs'], 'extra_system_packages': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux'], 'flavor': 'default', 'sha1': '41c500dd60c93c8d83053b72cffc3dd1a952a336'}}, {'print': '**** done installing nautilus'}, {'ceph': {'conf': {'global': {'mon warn on pool no app': False, 'ms bind msgr2': False, 'mon pg warn min per osd': 0}, 'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'log-ignorelist': ['overall HEALTH_', '\\(FS_', '\\(MDS_', '\\(OSD_', '\\(MON_DOWN\\)', '\\(CACHE_POOL_', '\\(POOL_', '\\(MGR_DOWN\\)', '\\(PG_', '\\(SMALLER_PGP_NUM\\)', 'Monitor daemon marked osd', 'Behind on trimming', 'Manager daemon', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN'], 'flavor': 'default', 'fs': 'xfs', 'sha1': '41c500dd60c93c8d83053b72cffc3dd1a952a336', 'cluster': 'ceph'}}, {'exec': {'osd.0': ['ceph osd set-require-min-compat-client nautilus']}}, {'print': '**** done ceph'}, {'ceph-fuse': None}, {'ceph-fuse': {'client.0': {'mounted': False}}}, {'print': '**** done unmount client.0'}, {'install.upgrade': {'client.0': {'branch': 'pacific'}}}, {'print': '**** done install.upgrade on client.0'}, {'ceph.healthy': None}, {'ceph-fuse': {'client.0': None}}, {'print': '**** done remount client.0'}, {'cephfs_test_runner': {'modules': ['tasks.cephfs.test_newops']}}], 'teuthology': {'branch': 'nautilus', 'fragments_dropped': [], 'meta': {}, 'postmerge': ['local function is_kupstream() return false end', 'local function is_kdistro() return false end', 'local function is_fuse() return true end', "local function syntax_version() return '' end", 'if not is_fuse() then reject() end']}, 'teuthology_branch': 'main', 'teuthology_sha1': '9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17', 'timestamp': '2024-04-17_21:24:02', 'tube': 'smithi', 'user': 'teuthology', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-04-17T22:28:23.437 DEBUG:teuthology.orchestra.run.smithi102:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-17T22:28:23.464 DEBUG:teuthology.orchestra.run.smithi190:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-17T22:28:23.485 DEBUG:teuthology.orchestra.run.smithi102:> sudo install -d -m0777 -- /var/run/ceph 2024-04-17T22:28:23.521 DEBUG:teuthology.orchestra.run.smithi190:> sudo install -d -m0777 -- /var/run/ceph 2024-04-17T22:28:23.551 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-04-17T22:28:23.551 DEBUG:teuthology.orchestra.run.smithi102:> dd if=/scratch_devs of=/dev/stdout 2024-04-17T22:28:23.604 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-17T22:28:23.604 DEBUG:teuthology.orchestra.run.smithi102:> stat /dev/vg_nvme/lv_1 2024-04-17T22:28:23.662 INFO:teuthology.orchestra.run.smithi102.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-17T22:28:23.662 INFO:teuthology.orchestra.run.smithi102.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-17T22:28:23.662 INFO:teuthology.orchestra.run.smithi102.stdout:Device: 6h/6d Inode: 168195 Links: 1 2024-04-17T22:28:23.662 INFO:teuthology.orchestra.run.smithi102.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-17T22:28:23.663 INFO:teuthology.orchestra.run.smithi102.stdout:Context: system_u:object_r:device_t:s0 2024-04-17T22:28:23.663 INFO:teuthology.orchestra.run.smithi102.stdout:Access: 2024-04-17 22:28:19.689769734 +0000 2024-04-17T22:28:23.663 INFO:teuthology.orchestra.run.smithi102.stdout:Modify: 2024-04-17 22:25:59.109951822 +0000 2024-04-17T22:28:23.663 INFO:teuthology.orchestra.run.smithi102.stdout:Change: 2024-04-17 22:25:59.109951822 +0000 2024-04-17T22:28:23.663 INFO:teuthology.orchestra.run.smithi102.stdout: Birth: - 2024-04-17T22:28:23.663 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-17T22:28:23.728 INFO:teuthology.orchestra.run.smithi102.stderr:1+0 records in 2024-04-17T22:28:23.728 INFO:teuthology.orchestra.run.smithi102.stderr:1+0 records out 2024-04-17T22:28:23.728 INFO:teuthology.orchestra.run.smithi102.stderr:512 bytes copied, 0.00019502 s, 2.6 MB/s 2024-04-17T22:28:23.729 DEBUG:teuthology.orchestra.run.smithi102:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-17T22:28:23.788 DEBUG:teuthology.orchestra.run.smithi102:> stat /dev/vg_nvme/lv_2 2024-04-17T22:28:23.847 INFO:teuthology.orchestra.run.smithi102.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-17T22:28:23.847 INFO:teuthology.orchestra.run.smithi102.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-17T22:28:23.847 INFO:teuthology.orchestra.run.smithi102.stdout:Device: 6h/6d Inode: 167625 Links: 1 2024-04-17T22:28:23.847 INFO:teuthology.orchestra.run.smithi102.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-17T22:28:23.847 INFO:teuthology.orchestra.run.smithi102.stdout:Context: system_u:object_r:device_t:s0 2024-04-17T22:28:23.847 INFO:teuthology.orchestra.run.smithi102.stdout:Access: 2024-04-17 22:28:19.689769734 +0000 2024-04-17T22:28:23.847 INFO:teuthology.orchestra.run.smithi102.stdout:Modify: 2024-04-17 22:25:59.432945396 +0000 2024-04-17T22:28:23.848 INFO:teuthology.orchestra.run.smithi102.stdout:Change: 2024-04-17 22:25:59.432945396 +0000 2024-04-17T22:28:23.848 INFO:teuthology.orchestra.run.smithi102.stdout: Birth: - 2024-04-17T22:28:23.848 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-17T22:28:23.912 INFO:teuthology.orchestra.run.smithi102.stderr:1+0 records in 2024-04-17T22:28:23.913 INFO:teuthology.orchestra.run.smithi102.stderr:1+0 records out 2024-04-17T22:28:23.913 INFO:teuthology.orchestra.run.smithi102.stderr:512 bytes copied, 0.000219771 s, 2.3 MB/s 2024-04-17T22:28:23.914 DEBUG:teuthology.orchestra.run.smithi102:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-17T22:28:23.972 DEBUG:teuthology.orchestra.run.smithi102:> stat /dev/vg_nvme/lv_3 2024-04-17T22:28:24.029 INFO:teuthology.orchestra.run.smithi102.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-17T22:28:24.030 INFO:teuthology.orchestra.run.smithi102.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-17T22:28:24.030 INFO:teuthology.orchestra.run.smithi102.stdout:Device: 6h/6d Inode: 169297 Links: 1 2024-04-17T22:28:24.030 INFO:teuthology.orchestra.run.smithi102.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-17T22:28:24.030 INFO:teuthology.orchestra.run.smithi102.stdout:Context: system_u:object_r:device_t:s0 2024-04-17T22:28:24.030 INFO:teuthology.orchestra.run.smithi102.stdout:Access: 2024-04-17 22:28:19.689769734 +0000 2024-04-17T22:28:24.030 INFO:teuthology.orchestra.run.smithi102.stdout:Modify: 2024-04-17 22:25:59.786938353 +0000 2024-04-17T22:28:24.030 INFO:teuthology.orchestra.run.smithi102.stdout:Change: 2024-04-17 22:25:59.786938353 +0000 2024-04-17T22:28:24.030 INFO:teuthology.orchestra.run.smithi102.stdout: Birth: - 2024-04-17T22:28:24.030 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-17T22:28:24.094 INFO:teuthology.orchestra.run.smithi102.stderr:1+0 records in 2024-04-17T22:28:24.094 INFO:teuthology.orchestra.run.smithi102.stderr:1+0 records out 2024-04-17T22:28:24.094 INFO:teuthology.orchestra.run.smithi102.stderr:512 bytes copied, 0.000176163 s, 2.9 MB/s 2024-04-17T22:28:24.095 DEBUG:teuthology.orchestra.run.smithi102:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-17T22:28:24.153 DEBUG:teuthology.orchestra.run.smithi102:> stat /dev/vg_nvme/lv_4 2024-04-17T22:28:24.211 INFO:teuthology.orchestra.run.smithi102.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-17T22:28:24.211 INFO:teuthology.orchestra.run.smithi102.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-17T22:28:24.211 INFO:teuthology.orchestra.run.smithi102.stdout:Device: 6h/6d Inode: 168363 Links: 1 2024-04-17T22:28:24.211 INFO:teuthology.orchestra.run.smithi102.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-17T22:28:24.211 INFO:teuthology.orchestra.run.smithi102.stdout:Context: system_u:object_r:device_t:s0 2024-04-17T22:28:24.211 INFO:teuthology.orchestra.run.smithi102.stdout:Access: 2024-04-17 22:28:19.689769734 +0000 2024-04-17T22:28:24.211 INFO:teuthology.orchestra.run.smithi102.stdout:Modify: 2024-04-17 22:26:00.121931688 +0000 2024-04-17T22:28:24.211 INFO:teuthology.orchestra.run.smithi102.stdout:Change: 2024-04-17 22:26:00.121931688 +0000 2024-04-17T22:28:24.211 INFO:teuthology.orchestra.run.smithi102.stdout: Birth: - 2024-04-17T22:28:24.212 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-17T22:28:24.276 INFO:teuthology.orchestra.run.smithi102.stderr:1+0 records in 2024-04-17T22:28:24.276 INFO:teuthology.orchestra.run.smithi102.stderr:1+0 records out 2024-04-17T22:28:24.276 INFO:teuthology.orchestra.run.smithi102.stderr:512 bytes copied, 0.0002241 s, 2.3 MB/s 2024-04-17T22:28:24.277 DEBUG:teuthology.orchestra.run.smithi102:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-17T22:28:24.335 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-17T22:28:24.335 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi102.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}} 2024-04-17T22:28:24.335 INFO:tasks.ceph:Generating config... 2024-04-17T22:28:24.337 INFO:tasks.ceph:[global] mon warn on pool no app = False 2024-04-17T22:28:24.337 INFO:tasks.ceph:[global] ms bind msgr2 = False 2024-04-17T22:28:24.337 INFO:tasks.ceph:[global] mon pg warn min per osd = 0 2024-04-17T22:28:24.337 INFO:tasks.ceph:[client] client mount timeout = 600 2024-04-17T22:28:24.337 INFO:tasks.ceph:[client] debug client = 20 2024-04-17T22:28:24.337 INFO:tasks.ceph:[client] debug ms = 1 2024-04-17T22:28:24.337 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-04-17T22:28:24.337 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-04-17T22:28:24.337 INFO:tasks.ceph:[mds] debug mds = 20 2024-04-17T22:28:24.337 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-04-17T22:28:24.337 INFO:tasks.ceph:[mds] debug ms = 1 2024-04-17T22:28:24.337 INFO:tasks.ceph:[mds] mds debug frag = True 2024-04-17T22:28:24.337 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-04-17T22:28:24.337 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-04-17T22:28:24.338 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-04-17T22:28:24.338 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-04-17T22:28:24.338 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-04-17T22:28:24.338 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-04-17T22:28:24.338 INFO:tasks.ceph:[mgr] debug client = 20 2024-04-17T22:28:24.338 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-04-17T22:28:24.338 INFO:tasks.ceph:[mgr] debug ms = 1 2024-04-17T22:28:24.338 INFO:tasks.ceph:[mon] debug mon = 20 2024-04-17T22:28:24.338 INFO:tasks.ceph:[mon] debug ms = 1 2024-04-17T22:28:24.338 INFO:tasks.ceph:[mon] debug paxos = 20 2024-04-17T22:28:24.338 INFO:tasks.ceph:[mon] mon down mkfs grace = 300 2024-04-17T22:28:24.338 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-04-17T22:28:24.338 INFO:tasks.ceph:[osd] bdev async discard = True 2024-04-17T22:28:24.338 INFO:tasks.ceph:[osd] bdev enable discard = True 2024-04-17T22:28:24.338 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2024-04-17T22:28:24.339 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-04-17T22:28:24.339 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-04-17T22:28:24.339 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-04-17T22:28:24.339 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-04-17T22:28:24.339 INFO:tasks.ceph:[osd] debug ms = 1 2024-04-17T22:28:24.339 INFO:tasks.ceph:[osd] debug osd = 20 2024-04-17T22:28:24.339 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-04-17T22:28:24.339 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-04-17T22:28:24.339 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-04-17T22:28:24.339 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-04-17T22:28:24.339 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-04-17T22:28:24.339 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-04-17T22:28:24.339 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-04-17T22:28:24.339 INFO:tasks.ceph:Setting up mon.a... 2024-04-17T22:28:24.340 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-04-17T22:28:24.732 INFO:teuthology.orchestra.run.smithi102.stdout:creating /etc/ceph/ceph.keyring 2024-04-17T22:28:24.735 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-04-17T22:28:24.799 DEBUG:teuthology.orchestra.run.smithi102:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-17T22:28:24.825 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.102'), ('mon.b', '[v2:172.21.15.102:3301,v1:172.21.15.102:6790]'), ('mon.c', '[v2:172.21.15.102:3302,v1:172.21.15.102:6791]')] 2024-04-17T22:28:24.825 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.102,[v2:172.21.15.102:3301,v1:172.21.15.102:6790],[v2:172.21.15.102:3302,v1:172.21.15.102:6791]', 'mon warn on pool no app': False, 'ms bind msgr2': False, 'mon pg warn min per osd': 0}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug client': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-04-17T22:28:24.826 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-04-17T22:28:24.826 DEBUG:teuthology.orchestra.run.smithi102:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-04-17T22:28:24.884 DEBUG:teuthology.orchestra.run.smithi102:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.102 --addv b '[v2:172.21.15.102:3301,v1:172.21.15.102:6790]' --addv c '[v2:172.21.15.102:3302,v1:172.21.15.102:6791]' --print /home/ubuntu/cephtest/ceph.monmap 2024-04-17T22:28:24.954 INFO:teuthology.orchestra.run.smithi102.stderr:warning: line 129: 'mon_down_mkfs_grace' in section 'mon' redefined 2024-04-17T22:28:24.990 INFO:teuthology.orchestra.run.smithi102.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-04-17T22:28:24.990 INFO:teuthology.orchestra.run.smithi102.stdout:monmaptool: generated fsid cb63e789-091d-49b1-b120-55f9db526871 2024-04-17T22:28:24.990 INFO:teuthology.orchestra.run.smithi102.stdout:epoch 0 2024-04-17T22:28:24.991 INFO:teuthology.orchestra.run.smithi102.stdout:fsid cb63e789-091d-49b1-b120-55f9db526871 2024-04-17T22:28:24.991 INFO:teuthology.orchestra.run.smithi102.stdout:last_changed 2024-04-17 22:28:25.005304 2024-04-17T22:28:24.991 INFO:teuthology.orchestra.run.smithi102.stdout:created 2024-04-17 22:28:25.005304 2024-04-17T22:28:24.991 INFO:teuthology.orchestra.run.smithi102.stdout:min_mon_release 0 (unknown) 2024-04-17T22:28:24.991 INFO:teuthology.orchestra.run.smithi102.stdout:0: [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] mon.a 2024-04-17T22:28:24.991 INFO:teuthology.orchestra.run.smithi102.stdout:1: [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] mon.b 2024-04-17T22:28:24.991 INFO:teuthology.orchestra.run.smithi102.stdout:2: [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] mon.c 2024-04-17T22:28:24.991 INFO:teuthology.orchestra.run.smithi102.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-04-17T22:28:24.992 DEBUG:teuthology.orchestra.run.smithi102:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-04-17T22:28:25.050 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID cb63e789-091d-49b1-b120-55f9db526871... 2024-04-17T22:28:25.051 DEBUG:teuthology.orchestra.run.smithi102:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-17T22:28:25.093 DEBUG:teuthology.orchestra.run.smithi190:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-17T22:28:25.134 INFO:teuthology.orchestra.run.smithi102.stdout:[global] 2024-04-17T22:28:25.134 INFO:teuthology.orchestra.run.smithi102.stdout: chdir = "" 2024-04-17T22:28:25.134 INFO:teuthology.orchestra.run.smithi102.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-17T22:28:25.134 INFO:teuthology.orchestra.run.smithi102.stdout: auth supported = cephx 2024-04-17T22:28:25.134 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-17T22:28:25.134 INFO:teuthology.orchestra.run.smithi102.stdout: filestore xattr use omap = true 2024-04-17T22:28:25.134 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-17T22:28:25.134 INFO:teuthology.orchestra.run.smithi102.stdout: mon clock drift allowed = 1.000 2024-04-17T22:28:25.134 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-17T22:28:25.134 INFO:teuthology.orchestra.run.smithi102.stdout: osd crush chooseleaf type = 0 2024-04-17T22:28:25.134 INFO:teuthology.orchestra.run.smithi102.stdout: auth debug = true 2024-04-17T22:28:25.134 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-17T22:28:25.135 INFO:teuthology.orchestra.run.smithi102.stdout: ms die on old message = true 2024-04-17T22:28:25.135 INFO:teuthology.orchestra.run.smithi102.stdout: ms die on bug = true 2024-04-17T22:28:25.135 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-17T22:28:25.135 INFO:teuthology.orchestra.run.smithi102.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-17T22:28:25.135 INFO:teuthology.orchestra.run.smithi102.stdout: mon pg warn max object skew = 0 2024-04-17T22:28:25.135 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-17T22:28:25.135 INFO:teuthology.orchestra.run.smithi102.stdout: # disable pg_autoscaler by default for new pools 2024-04-17T22:28:25.135 INFO:teuthology.orchestra.run.smithi102.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-17T22:28:25.135 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-17T22:28:25.135 INFO:teuthology.orchestra.run.smithi102.stdout: osd pool default size = 2 2024-04-17T22:28:25.135 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-17T22:28:25.135 INFO:teuthology.orchestra.run.smithi102.stdout: mon osd allow primary affinity = true 2024-04-17T22:28:25.135 INFO:teuthology.orchestra.run.smithi102.stdout: mon osd allow pg remap = true 2024-04-17T22:28:25.135 INFO:teuthology.orchestra.run.smithi102.stdout: mon warn on legacy crush tunables = false 2024-04-17T22:28:25.136 INFO:teuthology.orchestra.run.smithi102.stdout: mon warn on crush straw calc version zero = false 2024-04-17T22:28:25.136 INFO:teuthology.orchestra.run.smithi102.stdout: mon warn on no sortbitwise = false 2024-04-17T22:28:25.136 INFO:teuthology.orchestra.run.smithi102.stdout: mon warn on osd down out interval zero = false 2024-04-17T22:28:25.136 INFO:teuthology.orchestra.run.smithi102.stdout: mon warn on too few osds = false 2024-04-17T22:28:25.136 INFO:teuthology.orchestra.run.smithi102.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-17T22:28:25.136 INFO:teuthology.orchestra.run.smithi102.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-17T22:28:25.136 INFO:teuthology.orchestra.run.smithi102.stdout: mon_allow_pool_size_one = true 2024-04-17T22:28:25.136 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-17T22:28:25.136 INFO:teuthology.orchestra.run.smithi102.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-17T22:28:25.136 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-17T22:28:25.136 INFO:teuthology.orchestra.run.smithi102.stdout: osd default data pool replay window = 5 2024-04-17T22:28:25.136 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-17T22:28:25.136 INFO:teuthology.orchestra.run.smithi102.stdout: mon allow pool delete = true 2024-04-17T22:28:25.136 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-17T22:28:25.136 INFO:teuthology.orchestra.run.smithi102.stdout: mon cluster log file level = debug 2024-04-17T22:28:25.137 INFO:teuthology.orchestra.run.smithi102.stdout: debug asserts on shutdown = true 2024-04-17T22:28:25.137 INFO:teuthology.orchestra.run.smithi102.stdout: mon health detail to clog = false 2024-04-17T22:28:25.137 INFO:teuthology.orchestra.run.smithi102.stdout: mon host = "172.21.15.102,[v2:172.21.15.102:3301,v1:172.21.15.102:6790],[v2:172.21.15.102:3302,v1:172.21.15.102:6791]" 2024-04-17T22:28:25.137 INFO:teuthology.orchestra.run.smithi102.stdout: mon warn on pool no app = False 2024-04-17T22:28:25.137 INFO:teuthology.orchestra.run.smithi102.stdout: ms bind msgr2 = False 2024-04-17T22:28:25.137 INFO:teuthology.orchestra.run.smithi102.stdout: mon pg warn min per osd = 0 2024-04-17T22:28:25.137 INFO:teuthology.orchestra.run.smithi102.stdout: fsid = cb63e789-091d-49b1-b120-55f9db526871 2024-04-17T22:28:25.137 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-17T22:28:25.137 INFO:teuthology.orchestra.run.smithi102.stdout:[osd] 2024-04-17T22:28:25.137 INFO:teuthology.orchestra.run.smithi102.stdout: osd journal size = 100 2024-04-17T22:28:25.137 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-17T22:28:25.137 INFO:teuthology.orchestra.run.smithi102.stdout: osd scrub load threshold = 5.0 2024-04-17T22:28:25.137 INFO:teuthology.orchestra.run.smithi102.stdout: osd scrub max interval = 600 2024-04-17T22:28:25.137 INFO:teuthology.orchestra.run.smithi102.stdout: osd mclock profile = high_recovery_ops 2024-04-17T22:28:25.138 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-17T22:28:25.138 INFO:teuthology.orchestra.run.smithi102.stdout: osd recover clone overlap = true 2024-04-17T22:28:25.138 INFO:teuthology.orchestra.run.smithi102.stdout: osd recovery max chunk = 1048576 2024-04-17T22:28:25.138 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-17T22:28:25.138 INFO:teuthology.orchestra.run.smithi102.stdout: osd debug shutdown = true 2024-04-17T22:28:25.138 INFO:teuthology.orchestra.run.smithi102.stdout: osd debug op order = true 2024-04-17T22:28:25.138 INFO:teuthology.orchestra.run.smithi102.stdout: osd debug verify stray on activate = true 2024-04-17T22:28:25.138 INFO:teuthology.orchestra.run.smithi102.stdout: osd debug trim objects = true 2024-04-17T22:28:25.138 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-17T22:28:25.138 INFO:teuthology.orchestra.run.smithi102.stdout: osd open classes on start = true 2024-04-17T22:28:25.138 INFO:teuthology.orchestra.run.smithi102.stdout: osd debug pg log writeout = true 2024-04-17T22:28:25.138 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-17T22:28:25.138 INFO:teuthology.orchestra.run.smithi102.stdout: osd deep scrub update digest min age = 30 2024-04-17T22:28:25.138 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-17T22:28:25.139 INFO:teuthology.orchestra.run.smithi102.stdout: osd map max advance = 10 2024-04-17T22:28:25.139 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-17T22:28:25.139 INFO:teuthology.orchestra.run.smithi102.stdout: journal zero on create = true 2024-04-17T22:28:25.139 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-17T22:28:25.139 INFO:teuthology.orchestra.run.smithi102.stdout: filestore ondisk finisher threads = 3 2024-04-17T22:28:25.139 INFO:teuthology.orchestra.run.smithi102.stdout: filestore apply finisher threads = 3 2024-04-17T22:28:25.139 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-17T22:28:25.139 INFO:teuthology.orchestra.run.smithi102.stdout: bdev debug aio = true 2024-04-17T22:28:25.139 INFO:teuthology.orchestra.run.smithi102.stdout: osd debug misdirected ops = true 2024-04-17T22:28:25.139 INFO:teuthology.orchestra.run.smithi102.stdout: bdev async discard = True 2024-04-17T22:28:25.139 INFO:teuthology.orchestra.run.smithi102.stdout: bdev enable discard = True 2024-04-17T22:28:25.139 INFO:teuthology.orchestra.run.smithi102.stdout: bluestore allocator = bitmap 2024-04-17T22:28:25.139 INFO:teuthology.orchestra.run.smithi102.stdout: bluestore block size = 96636764160 2024-04-17T22:28:25.139 INFO:teuthology.orchestra.run.smithi102.stdout: bluestore fsck on mount = True 2024-04-17T22:28:25.140 INFO:teuthology.orchestra.run.smithi102.stdout: debug bluefs = 1/20 2024-04-17T22:28:25.140 INFO:teuthology.orchestra.run.smithi102.stdout: debug bluestore = 1/20 2024-04-17T22:28:25.140 INFO:teuthology.orchestra.run.smithi102.stdout: debug ms = 1 2024-04-17T22:28:25.140 INFO:teuthology.orchestra.run.smithi102.stdout: debug osd = 20 2024-04-17T22:28:25.140 INFO:teuthology.orchestra.run.smithi102.stdout: debug rocksdb = 4/10 2024-04-17T22:28:25.140 INFO:teuthology.orchestra.run.smithi102.stdout: mon osd backfillfull_ratio = 0.85 2024-04-17T22:28:25.140 INFO:teuthology.orchestra.run.smithi102.stdout: mon osd full ratio = 0.9 2024-04-17T22:28:25.140 INFO:teuthology.orchestra.run.smithi102.stdout: mon osd nearfull ratio = 0.8 2024-04-17T22:28:25.140 INFO:teuthology.orchestra.run.smithi102.stdout: osd failsafe full ratio = 0.95 2024-04-17T22:28:25.140 INFO:teuthology.orchestra.run.smithi102.stdout: osd objectstore = bluestore 2024-04-17T22:28:25.140 INFO:teuthology.orchestra.run.smithi102.stdout: osd op complaint time = 180 2024-04-17T22:28:25.140 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-17T22:28:25.140 INFO:teuthology.orchestra.run.smithi102.stdout:[mgr] 2024-04-17T22:28:25.140 INFO:teuthology.orchestra.run.smithi102.stdout: debug ms = 1 2024-04-17T22:28:25.141 INFO:teuthology.orchestra.run.smithi102.stdout: debug mgr = 20 2024-04-17T22:28:25.141 INFO:teuthology.orchestra.run.smithi102.stdout: debug mon = 20 2024-04-17T22:28:25.141 INFO:teuthology.orchestra.run.smithi102.stdout: debug auth = 20 2024-04-17T22:28:25.141 INFO:teuthology.orchestra.run.smithi102.stdout: mon reweight min pgs per osd = 4 2024-04-17T22:28:25.141 INFO:teuthology.orchestra.run.smithi102.stdout: mon reweight min bytes per osd = 10 2024-04-17T22:28:25.141 INFO:teuthology.orchestra.run.smithi102.stdout: mgr/telemetry/nag = false 2024-04-17T22:28:25.141 INFO:teuthology.orchestra.run.smithi102.stdout: debug client = 20 2024-04-17T22:28:25.141 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-17T22:28:25.141 INFO:teuthology.orchestra.run.smithi102.stdout:[mon] 2024-04-17T22:28:25.141 INFO:teuthology.orchestra.run.smithi102.stdout: debug ms = 1 2024-04-17T22:28:25.141 INFO:teuthology.orchestra.run.smithi102.stdout: debug mon = 20 2024-04-17T22:28:25.141 INFO:teuthology.orchestra.run.smithi102.stdout: debug paxos = 20 2024-04-17T22:28:25.141 INFO:teuthology.orchestra.run.smithi102.stdout: debug auth = 20 2024-04-17T22:28:25.141 INFO:teuthology.orchestra.run.smithi102.stdout: mon data avail warn = 5 2024-04-17T22:28:25.142 INFO:teuthology.orchestra.run.smithi102.stdout: mon mgr mkfs grace = 240 2024-04-17T22:28:25.142 INFO:teuthology.orchestra.run.smithi102.stdout: mon reweight min pgs per osd = 4 2024-04-17T22:28:25.142 INFO:teuthology.orchestra.run.smithi102.stdout: mon osd reporter subtree level = osd 2024-04-17T22:28:25.142 INFO:teuthology.orchestra.run.smithi102.stdout: mon osd prime pg temp = true 2024-04-17T22:28:25.142 INFO:teuthology.orchestra.run.smithi102.stdout: mon reweight min bytes per osd = 10 2024-04-17T22:28:25.142 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-17T22:28:25.142 INFO:teuthology.orchestra.run.smithi102.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-17T22:28:25.142 INFO:teuthology.orchestra.run.smithi102.stdout: auth mon ticket ttl = 660 # 11m 2024-04-17T22:28:25.142 INFO:teuthology.orchestra.run.smithi102.stdout: auth service ticket ttl = 240 # 4m 2024-04-17T22:28:25.142 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-17T22:28:25.142 INFO:teuthology.orchestra.run.smithi102.stdout: # don't complain about insecure global_id in the test suite 2024-04-17T22:28:25.142 INFO:teuthology.orchestra.run.smithi102.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-17T22:28:25.142 INFO:teuthology.orchestra.run.smithi102.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-17T22:28:25.142 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-17T22:28:25.143 INFO:teuthology.orchestra.run.smithi102.stdout: # 1m isn't quite enough 2024-04-17T22:28:25.143 INFO:teuthology.orchestra.run.smithi102.stdout: mon_down_mkfs_grace = 2m 2024-04-17T22:28:25.143 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-17T22:28:25.143 INFO:teuthology.orchestra.run.smithi102.stdout: mon_warn_on_filestore_osds = false 2024-04-17T22:28:25.143 INFO:teuthology.orchestra.run.smithi102.stdout: mon down mkfs grace = 300 2024-04-17T22:28:25.143 INFO:teuthology.orchestra.run.smithi102.stdout: mon op complaint time = 120 2024-04-17T22:28:25.143 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-17T22:28:25.143 INFO:teuthology.orchestra.run.smithi102.stdout:[client] 2024-04-17T22:28:25.143 INFO:teuthology.orchestra.run.smithi102.stdout: rgw cache enabled = true 2024-04-17T22:28:25.143 INFO:teuthology.orchestra.run.smithi102.stdout: rgw enable ops log = true 2024-04-17T22:28:25.143 INFO:teuthology.orchestra.run.smithi102.stdout: rgw enable usage log = true 2024-04-17T22:28:25.143 INFO:teuthology.orchestra.run.smithi102.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-17T22:28:25.143 INFO:teuthology.orchestra.run.smithi102.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-17T22:28:25.143 INFO:teuthology.orchestra.run.smithi102.stdout: client mount timeout = 600 2024-04-17T22:28:25.144 INFO:teuthology.orchestra.run.smithi102.stdout: debug client = 20 2024-04-17T22:28:25.144 INFO:teuthology.orchestra.run.smithi102.stdout: debug ms = 1 2024-04-17T22:28:25.144 INFO:teuthology.orchestra.run.smithi102.stdout: rados mon op timeout = 900 2024-04-17T22:28:25.144 INFO:teuthology.orchestra.run.smithi102.stdout: rados osd op timeout = 900 2024-04-17T22:28:25.144 INFO:teuthology.orchestra.run.smithi102.stdout:[mon.a] 2024-04-17T22:28:25.144 INFO:teuthology.orchestra.run.smithi102.stdout:[mon.b] 2024-04-17T22:28:25.144 INFO:teuthology.orchestra.run.smithi102.stdout:[mon.c] 2024-04-17T22:28:25.144 INFO:teuthology.orchestra.run.smithi102.stdout:[mds.a] 2024-04-17T22:28:25.144 INFO:teuthology.orchestra.run.smithi102.stdout:[mds] 2024-04-17T22:28:25.144 INFO:teuthology.orchestra.run.smithi102.stdout: debug mds = 20 2024-04-17T22:28:25.144 INFO:teuthology.orchestra.run.smithi102.stdout: debug mds balancer = 20 2024-04-17T22:28:25.144 INFO:teuthology.orchestra.run.smithi102.stdout: debug ms = 1 2024-04-17T22:28:25.144 INFO:teuthology.orchestra.run.smithi102.stdout: mds debug frag = True 2024-04-17T22:28:25.144 INFO:teuthology.orchestra.run.smithi102.stdout: mds debug scatterstat = True 2024-04-17T22:28:25.145 INFO:teuthology.orchestra.run.smithi102.stdout: mds op complaint time = 180 2024-04-17T22:28:25.145 INFO:teuthology.orchestra.run.smithi102.stdout: mds verify scatter = True 2024-04-17T22:28:25.145 INFO:teuthology.orchestra.run.smithi102.stdout: osd op complaint time = 180 2024-04-17T22:28:25.145 INFO:teuthology.orchestra.run.smithi102.stdout: rados mon op timeout = 900 2024-04-17T22:28:25.145 INFO:teuthology.orchestra.run.smithi102.stdout: rados osd op timeout = 900 2024-04-17T22:28:25.145 INFO:teuthology.orchestra.run.smithi190.stdout:[global] 2024-04-17T22:28:25.146 INFO:teuthology.orchestra.run.smithi190.stdout: chdir = "" 2024-04-17T22:28:25.146 INFO:teuthology.orchestra.run.smithi190.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-17T22:28:25.146 INFO:teuthology.orchestra.run.smithi190.stdout: auth supported = cephx 2024-04-17T22:28:25.146 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-04-17T22:28:25.146 INFO:teuthology.orchestra.run.smithi190.stdout: filestore xattr use omap = true 2024-04-17T22:28:25.146 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-04-17T22:28:25.146 INFO:teuthology.orchestra.run.smithi190.stdout: mon clock drift allowed = 1.000 2024-04-17T22:28:25.146 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-04-17T22:28:25.146 INFO:teuthology.orchestra.run.smithi190.stdout: osd crush chooseleaf type = 0 2024-04-17T22:28:25.146 INFO:teuthology.orchestra.run.smithi190.stdout: auth debug = true 2024-04-17T22:28:25.146 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-04-17T22:28:25.146 INFO:teuthology.orchestra.run.smithi190.stdout: ms die on old message = true 2024-04-17T22:28:25.146 INFO:teuthology.orchestra.run.smithi190.stdout: ms die on bug = true 2024-04-17T22:28:25.147 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-04-17T22:28:25.147 INFO:teuthology.orchestra.run.smithi190.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-17T22:28:25.147 INFO:teuthology.orchestra.run.smithi190.stdout: mon pg warn max object skew = 0 2024-04-17T22:28:25.147 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-04-17T22:28:25.147 INFO:teuthology.orchestra.run.smithi190.stdout: # disable pg_autoscaler by default for new pools 2024-04-17T22:28:25.147 INFO:teuthology.orchestra.run.smithi190.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-17T22:28:25.147 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-04-17T22:28:25.147 INFO:teuthology.orchestra.run.smithi190.stdout: osd pool default size = 2 2024-04-17T22:28:25.147 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-04-17T22:28:25.147 INFO:teuthology.orchestra.run.smithi190.stdout: mon osd allow primary affinity = true 2024-04-17T22:28:25.147 INFO:teuthology.orchestra.run.smithi190.stdout: mon osd allow pg remap = true 2024-04-17T22:28:25.147 INFO:teuthology.orchestra.run.smithi190.stdout: mon warn on legacy crush tunables = false 2024-04-17T22:28:25.147 INFO:teuthology.orchestra.run.smithi190.stdout: mon warn on crush straw calc version zero = false 2024-04-17T22:28:25.147 INFO:teuthology.orchestra.run.smithi190.stdout: mon warn on no sortbitwise = false 2024-04-17T22:28:25.148 INFO:teuthology.orchestra.run.smithi190.stdout: mon warn on osd down out interval zero = false 2024-04-17T22:28:25.148 INFO:teuthology.orchestra.run.smithi190.stdout: mon warn on too few osds = false 2024-04-17T22:28:25.148 INFO:teuthology.orchestra.run.smithi190.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-17T22:28:25.148 INFO:teuthology.orchestra.run.smithi190.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-17T22:28:25.148 INFO:teuthology.orchestra.run.smithi190.stdout: mon_allow_pool_size_one = true 2024-04-17T22:28:25.148 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-04-17T22:28:25.148 INFO:teuthology.orchestra.run.smithi190.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-17T22:28:25.148 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-04-17T22:28:25.148 INFO:teuthology.orchestra.run.smithi190.stdout: osd default data pool replay window = 5 2024-04-17T22:28:25.148 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-04-17T22:28:25.148 INFO:teuthology.orchestra.run.smithi190.stdout: mon allow pool delete = true 2024-04-17T22:28:25.148 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-04-17T22:28:25.148 INFO:teuthology.orchestra.run.smithi190.stdout: mon cluster log file level = debug 2024-04-17T22:28:25.148 INFO:teuthology.orchestra.run.smithi190.stdout: debug asserts on shutdown = true 2024-04-17T22:28:25.149 INFO:teuthology.orchestra.run.smithi190.stdout: mon health detail to clog = false 2024-04-17T22:28:25.149 INFO:teuthology.orchestra.run.smithi190.stdout: mon host = "172.21.15.102,[v2:172.21.15.102:3301,v1:172.21.15.102:6790],[v2:172.21.15.102:3302,v1:172.21.15.102:6791]" 2024-04-17T22:28:25.149 INFO:teuthology.orchestra.run.smithi190.stdout: mon warn on pool no app = False 2024-04-17T22:28:25.149 INFO:teuthology.orchestra.run.smithi190.stdout: ms bind msgr2 = False 2024-04-17T22:28:25.149 INFO:teuthology.orchestra.run.smithi190.stdout: mon pg warn min per osd = 0 2024-04-17T22:28:25.149 INFO:teuthology.orchestra.run.smithi190.stdout: fsid = cb63e789-091d-49b1-b120-55f9db526871 2024-04-17T22:28:25.149 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-04-17T22:28:25.149 INFO:teuthology.orchestra.run.smithi190.stdout:[osd] 2024-04-17T22:28:25.149 INFO:teuthology.orchestra.run.smithi190.stdout: osd journal size = 100 2024-04-17T22:28:25.149 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-04-17T22:28:25.149 INFO:teuthology.orchestra.run.smithi190.stdout: osd scrub load threshold = 5.0 2024-04-17T22:28:25.149 INFO:teuthology.orchestra.run.smithi190.stdout: osd scrub max interval = 600 2024-04-17T22:28:25.149 INFO:teuthology.orchestra.run.smithi190.stdout: osd mclock profile = high_recovery_ops 2024-04-17T22:28:25.149 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-04-17T22:28:25.150 INFO:teuthology.orchestra.run.smithi190.stdout: osd recover clone overlap = true 2024-04-17T22:28:25.150 INFO:teuthology.orchestra.run.smithi190.stdout: osd recovery max chunk = 1048576 2024-04-17T22:28:25.150 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-04-17T22:28:25.150 INFO:teuthology.orchestra.run.smithi190.stdout: osd debug shutdown = true 2024-04-17T22:28:25.150 INFO:teuthology.orchestra.run.smithi190.stdout: osd debug op order = true 2024-04-17T22:28:25.150 INFO:teuthology.orchestra.run.smithi190.stdout: osd debug verify stray on activate = true 2024-04-17T22:28:25.150 INFO:teuthology.orchestra.run.smithi190.stdout: osd debug trim objects = true 2024-04-17T22:28:25.150 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-04-17T22:28:25.150 INFO:teuthology.orchestra.run.smithi190.stdout: osd open classes on start = true 2024-04-17T22:28:25.150 INFO:teuthology.orchestra.run.smithi190.stdout: osd debug pg log writeout = true 2024-04-17T22:28:25.150 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-04-17T22:28:25.150 INFO:teuthology.orchestra.run.smithi190.stdout: osd deep scrub update digest min age = 30 2024-04-17T22:28:25.150 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-04-17T22:28:25.151 INFO:teuthology.orchestra.run.smithi190.stdout: osd map max advance = 10 2024-04-17T22:28:25.151 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-04-17T22:28:25.151 INFO:teuthology.orchestra.run.smithi190.stdout: journal zero on create = true 2024-04-17T22:28:25.151 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-04-17T22:28:25.151 INFO:teuthology.orchestra.run.smithi190.stdout: filestore ondisk finisher threads = 3 2024-04-17T22:28:25.151 INFO:teuthology.orchestra.run.smithi190.stdout: filestore apply finisher threads = 3 2024-04-17T22:28:25.151 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-04-17T22:28:25.151 INFO:teuthology.orchestra.run.smithi190.stdout: bdev debug aio = true 2024-04-17T22:28:25.151 INFO:teuthology.orchestra.run.smithi190.stdout: osd debug misdirected ops = true 2024-04-17T22:28:25.151 INFO:teuthology.orchestra.run.smithi190.stdout: bdev async discard = True 2024-04-17T22:28:25.151 INFO:teuthology.orchestra.run.smithi190.stdout: bdev enable discard = True 2024-04-17T22:28:25.151 INFO:teuthology.orchestra.run.smithi190.stdout: bluestore allocator = bitmap 2024-04-17T22:28:25.151 INFO:teuthology.orchestra.run.smithi190.stdout: bluestore block size = 96636764160 2024-04-17T22:28:25.151 INFO:teuthology.orchestra.run.smithi190.stdout: bluestore fsck on mount = True 2024-04-17T22:28:25.152 INFO:teuthology.orchestra.run.smithi190.stdout: debug bluefs = 1/20 2024-04-17T22:28:25.152 INFO:teuthology.orchestra.run.smithi190.stdout: debug bluestore = 1/20 2024-04-17T22:28:25.152 INFO:teuthology.orchestra.run.smithi190.stdout: debug ms = 1 2024-04-17T22:28:25.152 INFO:teuthology.orchestra.run.smithi190.stdout: debug osd = 20 2024-04-17T22:28:25.152 INFO:teuthology.orchestra.run.smithi190.stdout: debug rocksdb = 4/10 2024-04-17T22:28:25.152 INFO:teuthology.orchestra.run.smithi190.stdout: mon osd backfillfull_ratio = 0.85 2024-04-17T22:28:25.152 INFO:teuthology.orchestra.run.smithi190.stdout: mon osd full ratio = 0.9 2024-04-17T22:28:25.152 INFO:teuthology.orchestra.run.smithi190.stdout: mon osd nearfull ratio = 0.8 2024-04-17T22:28:25.152 INFO:teuthology.orchestra.run.smithi190.stdout: osd failsafe full ratio = 0.95 2024-04-17T22:28:25.152 INFO:teuthology.orchestra.run.smithi190.stdout: osd objectstore = bluestore 2024-04-17T22:28:25.152 INFO:teuthology.orchestra.run.smithi190.stdout: osd op complaint time = 180 2024-04-17T22:28:25.152 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-04-17T22:28:25.152 INFO:teuthology.orchestra.run.smithi190.stdout:[mgr] 2024-04-17T22:28:25.152 INFO:teuthology.orchestra.run.smithi190.stdout: debug ms = 1 2024-04-17T22:28:25.152 INFO:teuthology.orchestra.run.smithi190.stdout: debug mgr = 20 2024-04-17T22:28:25.153 INFO:teuthology.orchestra.run.smithi190.stdout: debug mon = 20 2024-04-17T22:28:25.153 INFO:teuthology.orchestra.run.smithi190.stdout: debug auth = 20 2024-04-17T22:28:25.153 INFO:teuthology.orchestra.run.smithi190.stdout: mon reweight min pgs per osd = 4 2024-04-17T22:28:25.153 INFO:teuthology.orchestra.run.smithi190.stdout: mon reweight min bytes per osd = 10 2024-04-17T22:28:25.153 INFO:teuthology.orchestra.run.smithi190.stdout: mgr/telemetry/nag = false 2024-04-17T22:28:25.153 INFO:teuthology.orchestra.run.smithi190.stdout: debug client = 20 2024-04-17T22:28:25.153 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-04-17T22:28:25.153 INFO:teuthology.orchestra.run.smithi190.stdout:[mon] 2024-04-17T22:28:25.153 INFO:teuthology.orchestra.run.smithi190.stdout: debug ms = 1 2024-04-17T22:28:25.153 INFO:teuthology.orchestra.run.smithi190.stdout: debug mon = 20 2024-04-17T22:28:25.153 INFO:teuthology.orchestra.run.smithi190.stdout: debug paxos = 20 2024-04-17T22:28:25.153 INFO:teuthology.orchestra.run.smithi190.stdout: debug auth = 20 2024-04-17T22:28:25.153 INFO:teuthology.orchestra.run.smithi190.stdout: mon data avail warn = 5 2024-04-17T22:28:25.153 INFO:teuthology.orchestra.run.smithi190.stdout: mon mgr mkfs grace = 240 2024-04-17T22:28:25.153 INFO:teuthology.orchestra.run.smithi190.stdout: mon reweight min pgs per osd = 4 2024-04-17T22:28:25.154 INFO:teuthology.orchestra.run.smithi190.stdout: mon osd reporter subtree level = osd 2024-04-17T22:28:25.154 INFO:teuthology.orchestra.run.smithi190.stdout: mon osd prime pg temp = true 2024-04-17T22:28:25.154 INFO:teuthology.orchestra.run.smithi190.stdout: mon reweight min bytes per osd = 10 2024-04-17T22:28:25.154 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-04-17T22:28:25.154 INFO:teuthology.orchestra.run.smithi190.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-17T22:28:25.154 INFO:teuthology.orchestra.run.smithi190.stdout: auth mon ticket ttl = 660 # 11m 2024-04-17T22:28:25.154 INFO:teuthology.orchestra.run.smithi190.stdout: auth service ticket ttl = 240 # 4m 2024-04-17T22:28:25.154 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-04-17T22:28:25.154 INFO:teuthology.orchestra.run.smithi190.stdout: # don't complain about insecure global_id in the test suite 2024-04-17T22:28:25.154 INFO:teuthology.orchestra.run.smithi190.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-17T22:28:25.155 INFO:teuthology.orchestra.run.smithi190.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-17T22:28:25.155 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-04-17T22:28:25.155 INFO:teuthology.orchestra.run.smithi190.stdout: # 1m isn't quite enough 2024-04-17T22:28:25.155 INFO:teuthology.orchestra.run.smithi190.stdout: mon_down_mkfs_grace = 2m 2024-04-17T22:28:25.155 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-04-17T22:28:25.155 INFO:teuthology.orchestra.run.smithi190.stdout: mon_warn_on_filestore_osds = false 2024-04-17T22:28:25.155 INFO:teuthology.orchestra.run.smithi190.stdout: mon down mkfs grace = 300 2024-04-17T22:28:25.155 INFO:teuthology.orchestra.run.smithi190.stdout: mon op complaint time = 120 2024-04-17T22:28:25.155 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-04-17T22:28:25.155 INFO:teuthology.orchestra.run.smithi190.stdout:[client] 2024-04-17T22:28:25.155 INFO:teuthology.orchestra.run.smithi190.stdout: rgw cache enabled = true 2024-04-17T22:28:25.155 INFO:teuthology.orchestra.run.smithi190.stdout: rgw enable ops log = true 2024-04-17T22:28:25.155 INFO:teuthology.orchestra.run.smithi190.stdout: rgw enable usage log = true 2024-04-17T22:28:25.155 INFO:teuthology.orchestra.run.smithi190.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-17T22:28:25.155 INFO:teuthology.orchestra.run.smithi190.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-17T22:28:25.156 INFO:teuthology.orchestra.run.smithi190.stdout: client mount timeout = 600 2024-04-17T22:28:25.156 INFO:teuthology.orchestra.run.smithi190.stdout: debug client = 20 2024-04-17T22:28:25.156 INFO:teuthology.orchestra.run.smithi190.stdout: debug ms = 1 2024-04-17T22:28:25.156 INFO:teuthology.orchestra.run.smithi190.stdout: rados mon op timeout = 900 2024-04-17T22:28:25.156 INFO:teuthology.orchestra.run.smithi190.stdout: rados osd op timeout = 900 2024-04-17T22:28:25.156 INFO:teuthology.orchestra.run.smithi190.stdout:[mon.a] 2024-04-17T22:28:25.156 INFO:teuthology.orchestra.run.smithi190.stdout:[mon.b] 2024-04-17T22:28:25.156 INFO:teuthology.orchestra.run.smithi190.stdout:[mon.c] 2024-04-17T22:28:25.156 INFO:teuthology.orchestra.run.smithi190.stdout:[mds.a] 2024-04-17T22:28:25.156 INFO:teuthology.orchestra.run.smithi190.stdout:[mds] 2024-04-17T22:28:25.156 INFO:teuthology.orchestra.run.smithi190.stdout: debug mds = 20 2024-04-17T22:28:25.156 INFO:teuthology.orchestra.run.smithi190.stdout: debug mds balancer = 20 2024-04-17T22:28:25.156 INFO:teuthology.orchestra.run.smithi190.stdout: debug ms = 1 2024-04-17T22:28:25.156 INFO:teuthology.orchestra.run.smithi190.stdout: mds debug frag = True 2024-04-17T22:28:25.156 INFO:teuthology.orchestra.run.smithi190.stdout: mds debug scatterstat = True 2024-04-17T22:28:25.157 INFO:teuthology.orchestra.run.smithi190.stdout: mds op complaint time = 180 2024-04-17T22:28:25.157 INFO:teuthology.orchestra.run.smithi190.stdout: mds verify scatter = True 2024-04-17T22:28:25.157 INFO:teuthology.orchestra.run.smithi190.stdout: osd op complaint time = 180 2024-04-17T22:28:25.157 INFO:teuthology.orchestra.run.smithi190.stdout: rados mon op timeout = 900 2024-04-17T22:28:25.157 INFO:teuthology.orchestra.run.smithi190.stdout: rados osd op timeout = 900 2024-04-17T22:28:25.158 INFO:tasks.ceph:Creating admin key on mon.a... 2024-04-17T22:28:25.158 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-04-17T22:28:25.271 INFO:tasks.ceph:Copying monmap to all nodes... 2024-04-17T22:28:25.271 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-04-17T22:28:25.271 DEBUG:teuthology.orchestra.run.smithi102:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-04-17T22:28:25.289 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-04-17T22:28:25.289 DEBUG:teuthology.orchestra.run.smithi102:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-04-17T22:28:25.346 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi102.front.sepia.ceph.com 2024-04-17T22:28:25.347 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-04-17T22:28:25.347 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-17T22:28:25.347 DEBUG:teuthology.orchestra.run.smithi102:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-17T22:28:25.422 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-04-17T22:28:25.422 DEBUG:teuthology.orchestra.run.smithi102:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-17T22:28:25.479 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi190.front.sepia.ceph.com 2024-04-17T22:28:25.479 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-04-17T22:28:25.479 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-17T22:28:25.479 DEBUG:teuthology.orchestra.run.smithi190:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-17T22:28:25.521 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-04-17T22:28:25.521 DEBUG:teuthology.orchestra.run.smithi190:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-17T22:28:25.580 INFO:tasks.ceph:Setting up mon nodes... 2024-04-17T22:28:25.580 INFO:tasks.ceph:Setting up mgr nodes... 2024-04-17T22:28:25.580 DEBUG:teuthology.orchestra.run.smithi102:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-04-17T22:28:25.652 INFO:teuthology.orchestra.run.smithi102.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-04-17T22:28:25.654 INFO:tasks.ceph:Setting up mds nodes... 2024-04-17T22:28:25.654 DEBUG:teuthology.orchestra.run.smithi102:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-04-17T22:28:25.726 INFO:teuthology.orchestra.run.smithi102.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-04-17T22:28:25.728 DEBUG:teuthology.orchestra.run.smithi102:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-04-17T22:28:25.755 INFO:tasks.ceph_client:Setting up client nodes... 2024-04-17T22:28:25.755 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-17T22:28:26.018 INFO:teuthology.orchestra.run.smithi190.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-04-17T22:28:26.033 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-04-17T22:28:26.033 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi102.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}} 2024-04-17T22:28:26.034 DEBUG:teuthology.orchestra.run.smithi102:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-04-17T22:28:26.060 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-17T22:28:26.061 INFO:tasks.ceph:role: osd.0 2024-04-17T22:28:26.061 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi102.front.sepia.ceph.com 2024-04-17T22:28:26.061 DEBUG:teuthology.orchestra.run.smithi102:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-17T22:28:26.134 INFO:teuthology.orchestra.run.smithi102.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-17T22:28:26.134 INFO:teuthology.orchestra.run.smithi102.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-17T22:28:26.135 INFO:teuthology.orchestra.run.smithi102.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-17T22:28:26.135 INFO:teuthology.orchestra.run.smithi102.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-17T22:28:26.135 INFO:teuthology.orchestra.run.smithi102.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-17T22:28:26.135 INFO:teuthology.orchestra.run.smithi102.stdout: = sunit=0 swidth=0 blks 2024-04-17T22:28:26.135 INFO:teuthology.orchestra.run.smithi102.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-17T22:28:26.135 INFO:teuthology.orchestra.run.smithi102.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-17T22:28:26.135 INFO:teuthology.orchestra.run.smithi102.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-17T22:28:26.135 INFO:teuthology.orchestra.run.smithi102.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-17T22:28:26.562 INFO:teuthology.orchestra.run.smithi102.stdout:Discarding blocks...Done. 2024-04-17T22:28:26.564 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi102.front.sepia.ceph.com -o noatime 2024-04-17T22:28:26.564 DEBUG:teuthology.orchestra.run.smithi102:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-04-17T22:28:26.740 DEBUG:teuthology.orchestra.run.smithi102:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-04-17T22:28:26.774 DEBUG:teuthology.orchestra.run.smithi102:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-04-17T22:28:26.842 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-17T22:28:26.842 INFO:tasks.ceph:role: osd.1 2024-04-17T22:28:26.842 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi102.front.sepia.ceph.com 2024-04-17T22:28:26.842 DEBUG:teuthology.orchestra.run.smithi102:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-17T22:28:26.916 INFO:teuthology.orchestra.run.smithi102.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-17T22:28:26.916 INFO:teuthology.orchestra.run.smithi102.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-17T22:28:26.916 INFO:teuthology.orchestra.run.smithi102.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-17T22:28:26.916 INFO:teuthology.orchestra.run.smithi102.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-17T22:28:26.916 INFO:teuthology.orchestra.run.smithi102.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-17T22:28:26.916 INFO:teuthology.orchestra.run.smithi102.stdout: = sunit=0 swidth=0 blks 2024-04-17T22:28:26.916 INFO:teuthology.orchestra.run.smithi102.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-17T22:28:26.916 INFO:teuthology.orchestra.run.smithi102.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-17T22:28:26.917 INFO:teuthology.orchestra.run.smithi102.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-17T22:28:26.917 INFO:teuthology.orchestra.run.smithi102.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-17T22:28:27.340 INFO:teuthology.orchestra.run.smithi102.stdout:Discarding blocks...Done. 2024-04-17T22:28:27.342 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi102.front.sepia.ceph.com -o noatime 2024-04-17T22:28:27.342 DEBUG:teuthology.orchestra.run.smithi102:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-04-17T22:28:27.446 DEBUG:teuthology.orchestra.run.smithi102:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-04-17T22:28:27.477 DEBUG:teuthology.orchestra.run.smithi102:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-04-17T22:28:27.544 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-17T22:28:27.544 INFO:tasks.ceph:role: osd.2 2024-04-17T22:28:27.544 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi102.front.sepia.ceph.com 2024-04-17T22:28:27.544 DEBUG:teuthology.orchestra.run.smithi102:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-17T22:28:27.618 INFO:teuthology.orchestra.run.smithi102.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-17T22:28:27.619 INFO:teuthology.orchestra.run.smithi102.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-17T22:28:27.619 INFO:teuthology.orchestra.run.smithi102.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-17T22:28:27.619 INFO:teuthology.orchestra.run.smithi102.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-17T22:28:27.619 INFO:teuthology.orchestra.run.smithi102.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-17T22:28:27.619 INFO:teuthology.orchestra.run.smithi102.stdout: = sunit=0 swidth=0 blks 2024-04-17T22:28:27.619 INFO:teuthology.orchestra.run.smithi102.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-17T22:28:27.619 INFO:teuthology.orchestra.run.smithi102.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-17T22:28:27.619 INFO:teuthology.orchestra.run.smithi102.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-17T22:28:27.619 INFO:teuthology.orchestra.run.smithi102.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-17T22:28:28.068 INFO:teuthology.orchestra.run.smithi102.stdout:Discarding blocks...Done. 2024-04-17T22:28:28.070 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi102.front.sepia.ceph.com -o noatime 2024-04-17T22:28:28.070 DEBUG:teuthology.orchestra.run.smithi102:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-04-17T22:28:28.224 DEBUG:teuthology.orchestra.run.smithi102:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-04-17T22:28:28.257 DEBUG:teuthology.orchestra.run.smithi102:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-04-17T22:28:28.324 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-17T22:28:28.324 INFO:tasks.ceph:role: osd.3 2024-04-17T22:28:28.324 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi102.front.sepia.ceph.com 2024-04-17T22:28:28.325 DEBUG:teuthology.orchestra.run.smithi102:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-17T22:28:28.398 INFO:teuthology.orchestra.run.smithi102.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-17T22:28:28.398 INFO:teuthology.orchestra.run.smithi102.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-17T22:28:28.399 INFO:teuthology.orchestra.run.smithi102.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-17T22:28:28.399 INFO:teuthology.orchestra.run.smithi102.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-17T22:28:28.399 INFO:teuthology.orchestra.run.smithi102.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-17T22:28:28.399 INFO:teuthology.orchestra.run.smithi102.stdout: = sunit=0 swidth=0 blks 2024-04-17T22:28:28.399 INFO:teuthology.orchestra.run.smithi102.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-17T22:28:28.399 INFO:teuthology.orchestra.run.smithi102.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-17T22:28:28.399 INFO:teuthology.orchestra.run.smithi102.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-17T22:28:28.399 INFO:teuthology.orchestra.run.smithi102.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-17T22:28:28.839 INFO:teuthology.orchestra.run.smithi102.stdout:Discarding blocks...Done. 2024-04-17T22:28:28.841 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi102.front.sepia.ceph.com -o noatime 2024-04-17T22:28:28.841 DEBUG:teuthology.orchestra.run.smithi102:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-04-17T22:28:29.005 DEBUG:teuthology.orchestra.run.smithi102:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-04-17T22:28:29.038 DEBUG:teuthology.orchestra.run.smithi102:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-17T22:28:29.126 INFO:teuthology.orchestra.run.smithi102.stderr:warning: line 130: 'mon_down_mkfs_grace' in section 'mon' redefined 2024-04-17T22:28:29.153 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:29.166 7fc9fd073dc0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-04-17T22:28:29.153 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:29.166 7fc9fd073dc0 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-04-17T22:28:29.153 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:29.166 7fc9fd073dc0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-17T22:28:29.154 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:29.166 7fc9fd073dc0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-17T22:28:29.154 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:29.166 7fc9fd073dc0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-17T22:28:29.154 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:29.167 7fc9fd073dc0 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-04-17T22:28:31.979 DEBUG:teuthology.orchestra.run.smithi102:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-04-17T22:28:32.011 DEBUG:teuthology.orchestra.run.smithi102:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-17T22:28:32.097 INFO:teuthology.orchestra.run.smithi102.stderr:warning: line 130: 'mon_down_mkfs_grace' in section 'mon' redefined 2024-04-17T22:28:32.126 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:32.139 7fba5ffc2dc0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-04-17T22:28:32.126 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:32.139 7fba5ffc2dc0 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-04-17T22:28:32.127 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:32.139 7fba5ffc2dc0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-17T22:28:32.127 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:32.139 7fba5ffc2dc0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-17T22:28:32.127 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:32.139 7fba5ffc2dc0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-17T22:28:32.127 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:32.140 7fba5ffc2dc0 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-04-17T22:28:34.951 DEBUG:teuthology.orchestra.run.smithi102:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-04-17T22:28:34.984 DEBUG:teuthology.orchestra.run.smithi102:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-17T22:28:35.071 INFO:teuthology.orchestra.run.smithi102.stderr:warning: line 130: 'mon_down_mkfs_grace' in section 'mon' redefined 2024-04-17T22:28:35.099 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:35.111 7f207ce1edc0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-04-17T22:28:35.099 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:35.112 7f207ce1edc0 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-04-17T22:28:35.099 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:35.112 7f207ce1edc0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-17T22:28:35.099 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:35.112 7f207ce1edc0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-17T22:28:35.099 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:35.112 7f207ce1edc0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-17T22:28:35.099 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:35.112 7f207ce1edc0 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-04-17T22:28:37.925 DEBUG:teuthology.orchestra.run.smithi102:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-04-17T22:28:37.957 DEBUG:teuthology.orchestra.run.smithi102:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-17T22:28:38.044 INFO:teuthology.orchestra.run.smithi102.stderr:warning: line 130: 'mon_down_mkfs_grace' in section 'mon' redefined 2024-04-17T22:28:38.072 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:38.085 7f05113a2dc0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-04-17T22:28:38.072 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:38.086 7f05113a2dc0 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-04-17T22:28:38.073 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:38.086 7f05113a2dc0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-17T22:28:38.073 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:38.086 7f05113a2dc0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-17T22:28:38.073 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:38.086 7f05113a2dc0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-17T22:28:38.073 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:38.086 7f05113a2dc0 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-04-17T22:28:40.906 DEBUG:teuthology.orchestra.run.smithi102:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-04-17T22:28:40.939 INFO:tasks.ceph:Reading keys from all nodes... 2024-04-17T22:28:40.939 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-04-17T22:28:40.940 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-04-17T22:28:41.012 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-04-17T22:28:41.012 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-04-17T22:28:41.085 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-04-17T22:28:41.085 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-04-17T22:28:41.156 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-04-17T22:28:41.156 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-04-17T22:28:41.226 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-04-17T22:28:41.226 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-04-17T22:28:41.295 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-04-17T22:28:41.295 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-04-17T22:28:41.363 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-04-17T22:28:41.364 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-04-17T22:28:41.385 INFO:tasks.ceph:Adding keys to all mons... 2024-04-17T22:28:41.385 DEBUG:teuthology.orchestra.run.smithi102:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-17T22:28:41.429 INFO:teuthology.orchestra.run.smithi102.stdout:[mgr.x] 2024-04-17T22:28:41.429 INFO:teuthology.orchestra.run.smithi102.stdout: key = AQAJTSBmVVS9JxAAVo4zURUUQ/1CNk/ZkIe0FQ== 2024-04-17T22:28:41.429 INFO:teuthology.orchestra.run.smithi102.stdout:[mds.a] 2024-04-17T22:28:41.429 INFO:teuthology.orchestra.run.smithi102.stdout: key = AQAJTSBmD30qLBAAIPhihrDWdFgNV27UuM4C5Q== 2024-04-17T22:28:41.430 INFO:teuthology.orchestra.run.smithi102.stdout:[osd.0] 2024-04-17T22:28:41.430 INFO:teuthology.orchestra.run.smithi102.stdout: key = AQANTSBmc2wAChAARaR//sfymHYNQZngBEXzlg== 2024-04-17T22:28:41.430 INFO:teuthology.orchestra.run.smithi102.stdout:[osd.1] 2024-04-17T22:28:41.430 INFO:teuthology.orchestra.run.smithi102.stdout: key = AQAQTSBmnIpmCBAAMw5q2mXkJxKuMhjiEq26xQ== 2024-04-17T22:28:41.430 INFO:teuthology.orchestra.run.smithi102.stdout:[osd.2] 2024-04-17T22:28:41.430 INFO:teuthology.orchestra.run.smithi102.stdout: key = AQATTSBmnAq/BhAANhjLDl3tH9XeRhOlnJsgjw== 2024-04-17T22:28:41.430 INFO:teuthology.orchestra.run.smithi102.stdout:[osd.3] 2024-04-17T22:28:41.430 INFO:teuthology.orchestra.run.smithi102.stdout: key = AQAWTSBmWq4vBRAATEjnXjqBqfsyiX7MaBNkYw== 2024-04-17T22:28:41.430 INFO:teuthology.orchestra.run.smithi102.stdout:[client.0] 2024-04-17T22:28:41.430 INFO:teuthology.orchestra.run.smithi102.stdout: key = AQAKTSBmLWP4ARAAV7xuVZUCRfvs7Zs54H89Qw== 2024-04-17T22:28:41.431 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-17T22:28:41.542 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-17T22:28:41.616 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-17T22:28:41.686 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-17T22:28:41.756 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-17T22:28:41.826 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-17T22:28:41.896 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-17T22:28:41.967 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-04-17T22:28:41.968 DEBUG:teuthology.orchestra.run.smithi102:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-04-17T22:28:41.995 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-17T22:28:42.074 INFO:teuthology.orchestra.run.smithi102.stderr:warning: line 130: 'mon_down_mkfs_grace' in section 'mon' redefined 2024-04-17T22:28:42.110 DEBUG:teuthology.orchestra.run.smithi102:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-04-17T22:28:42.176 DEBUG:teuthology.orchestra.run.smithi102:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-04-17T22:28:42.243 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-17T22:28:42.323 INFO:teuthology.orchestra.run.smithi102.stderr:warning: line 130: 'mon_down_mkfs_grace' in section 'mon' redefined 2024-04-17T22:28:42.360 DEBUG:teuthology.orchestra.run.smithi102:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-04-17T22:28:42.427 DEBUG:teuthology.orchestra.run.smithi102:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-04-17T22:28:42.494 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-17T22:28:42.574 INFO:teuthology.orchestra.run.smithi102.stderr:warning: line 130: 'mon_down_mkfs_grace' in section 'mon' redefined 2024-04-17T22:28:42.675 DEBUG:teuthology.orchestra.run.smithi102:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-04-17T22:28:42.742 DEBUG:teuthology.orchestra.run.smithi102:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-17T22:28:42.799 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-04-17T22:28:42.799 INFO:tasks.ceph.mon.a:Restarting daemon 2024-04-17T22:28:42.800 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-04-17T22:28:42.842 INFO:tasks.ceph.mon.a:Started 2024-04-17T22:28:42.842 INFO:tasks.ceph.mon.b:Restarting daemon 2024-04-17T22:28:42.842 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-04-17T22:28:42.843 INFO:tasks.ceph.mon.b:Started 2024-04-17T22:28:42.844 INFO:tasks.ceph.mon.c:Restarting daemon 2024-04-17T22:28:42.844 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-04-17T22:28:42.845 INFO:tasks.ceph.mon.c:Started 2024-04-17T22:28:42.845 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-04-17T22:28:42.846 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-04-17T22:28:42.846 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-04-17T22:28:42.848 INFO:tasks.ceph.mgr.x:Started 2024-04-17T22:28:42.848 INFO:tasks.ceph:Setting crush tunables to default 2024-04-17T22:28:42.848 DEBUG:teuthology.orchestra.run.smithi102:> sudo ceph --cluster ceph osd crush tunables default 2024-04-17T22:28:42.919 INFO:tasks.ceph.mon.a.smithi102.stderr:warning: line 130: 'mon_down_mkfs_grace' in section 'mon' redefined 2024-04-17T22:28:42.919 INFO:tasks.ceph.mon.b.smithi102.stderr:warning: line 130: 'mon_down_mkfs_grace' in section 'mon' redefined 2024-04-17T22:28:42.941 INFO:tasks.ceph.mgr.x.smithi102.stderr:warning: line 130: 'mon_down_mkfs_grace' in section 'mon' redefined 2024-04-17T22:28:42.946 INFO:tasks.ceph.mon.c.smithi102.stderr:warning: line 130: 'mon_down_mkfs_grace' in section 'mon' redefined 2024-04-17T22:28:42.995 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:43.007 7f736af3c700 1 Processor -- start 2024-04-17T22:28:42.996 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:43.007 7f736af3c700 1 -- start start 2024-04-17T22:28:42.996 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:43.007 7f736af3c700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f73640f5750 0x7f736410c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:28:42.996 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:43.007 7f736af3c700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f736410c720 0x7f73641a0be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:28:42.996 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:43.007 7f736af3c700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f736410cb70 0x7f73641a1210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:28:42.996 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:43.007 7f736af3c700 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73640f93c0 con 0x7f736410cb70 2024-04-17T22:28:42.996 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:43.007 7f736af3c700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73641a1840 con 0x7f73640f5750 2024-04-17T22:28:42.996 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:43.007 7f736af3c700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f73641a1a70 con 0x7f736410c720 2024-04-17T22:28:42.996 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:43.007 7f73637fe700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f73640f5750 0x7f736410c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:28:42.996 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:43.008 7f7362ffd700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f736410c720 0x7f73641a0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:28:42.996 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:43.008 7f73637fe700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f73640f5750 0x7f736410c0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:35604/0 (socket says 172.21.15.102:35604) 2024-04-17T22:28:42.996 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:43.008 7f73637fe700 1 -- 172.21.15.102:0/2170063074 learned_addr learned my addr 172.21.15.102:0/2170063074 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:28:42.996 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:43.008 7f7363fff700 1 --2- 172.21.15.102:0/2170063074 >> v2:172.21.15.102:3300/0 conn(0x7f736410cb70 0x7f73641a1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:28:42.996 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:43.008 7f73637fe700 1 -- 172.21.15.102:0/2170063074 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f73640f5750 msgr2=0x7f736410c0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-17T22:28:42.996 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:43.008 7f73637fe700 1 -- 172.21.15.102:0/2170063074 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f73640f5750 msgr2=0x7f736410c0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-17T22:28:42.997 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:43.008 7f7362ffd700 1 -- 172.21.15.102:0/2170063074 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f736410c720 msgr2=0x7f73641a0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-17T22:28:42.997 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:43.008 7f73637fe700 1 --2- 172.21.15.102:0/2170063074 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f73640f5750 0x7f736410c0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-04-17T22:28:42.997 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:43.008 7f7362ffd700 1 -- 172.21.15.102:0/2170063074 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f736410c720 msgr2=0x7f73641a0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-17T22:28:42.997 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:43.008 7f7362ffd700 1 --2- 172.21.15.102:0/2170063074 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f736410c720 0x7f73641a0be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-04-17T22:28:42.997 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:43.008 7f73637fe700 1 --2- 172.21.15.102:0/2170063074 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f73640f5750 0x7f736410c0f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2024-04-17T22:28:42.997 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:43.008 7f7363fff700 1 -- 172.21.15.102:0/2170063074 >> v2:172.21.15.102:3300/0 conn(0x7f736410cb70 msgr2=0x7f73641a1210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-17T22:28:42.997 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:43.008 7f7362ffd700 1 --2- 172.21.15.102:0/2170063074 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f736410c720 0x7f73641a0be0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2024-04-17T22:28:42.997 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:43.008 7f7363fff700 1 -- 172.21.15.102:0/2170063074 >> v2:172.21.15.102:3300/0 conn(0x7f736410cb70 msgr2=0x7f73641a1210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-17T22:28:42.997 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:43.008 7f7363fff700 1 --2- 172.21.15.102:0/2170063074 >> v2:172.21.15.102:3300/0 conn(0x7f736410cb70 0x7f73641a1210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-04-17T22:28:42.997 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:43.008 7f7363fff700 1 --2- 172.21.15.102:0/2170063074 >> v2:172.21.15.102:3300/0 conn(0x7f736410cb70 0x7f73641a1210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2024-04-17T22:28:43.196 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:43.209 7f73637fe700 1 --2- 172.21.15.102:0/2170063074 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f73640f5750 0x7f736410c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:28:43.196 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:43.209 7f7362ffd700 1 --2- 172.21.15.102:0/2170063074 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f736410c720 0x7f73641a0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:28:43.196 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:43.209 7f7363fff700 1 --2- 172.21.15.102:0/2170063074 >> v2:172.21.15.102:3300/0 conn(0x7f736410cb70 0x7f73641a1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:28:43.196 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:43.209 7f73637fe700 1 -- 172.21.15.102:0/2170063074 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f73640f5750 msgr2=0x7f736410c0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-17T22:28:43.196 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:43.209 7f73637fe700 1 -- 172.21.15.102:0/2170063074 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f73640f5750 msgr2=0x7f736410c0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-17T22:28:43.196 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:43.209 7f7362ffd700 1 -- 172.21.15.102:0/2170063074 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f736410c720 msgr2=0x7f73641a0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-17T22:28:43.196 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:43.209 7f7363fff700 1 -- 172.21.15.102:0/2170063074 >> v2:172.21.15.102:3300/0 conn(0x7f736410cb70 msgr2=0x7f73641a1210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-17T22:28:43.197 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:43.209 7f7362ffd700 1 -- 172.21.15.102:0/2170063074 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f736410c720 msgr2=0x7f73641a0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-17T22:28:43.197 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:43.209 7f7363fff700 1 -- 172.21.15.102:0/2170063074 >> v2:172.21.15.102:3300/0 conn(0x7f736410cb70 msgr2=0x7f73641a1210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-17T22:28:43.197 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:43.209 7f73637fe700 1 --2- 172.21.15.102:0/2170063074 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f73640f5750 0x7f736410c0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-04-17T22:28:43.197 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:43.209 7f7362ffd700 1 --2- 172.21.15.102:0/2170063074 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f736410c720 0x7f73641a0be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-04-17T22:28:43.197 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:43.209 7f7363fff700 1 --2- 172.21.15.102:0/2170063074 >> v2:172.21.15.102:3300/0 conn(0x7f736410cb70 0x7f73641a1210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-04-17T22:28:43.197 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:43.209 7f7362ffd700 1 --2- 172.21.15.102:0/2170063074 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f736410c720 0x7f73641a0be0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.400000 2024-04-17T22:28:43.197 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:43.209 7f7363fff700 1 --2- 172.21.15.102:0/2170063074 >> v2:172.21.15.102:3300/0 conn(0x7f736410cb70 0x7f73641a1210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.400000 2024-04-17T22:28:43.197 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:43.209 7f73637fe700 1 --2- 172.21.15.102:0/2170063074 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f73640f5750 0x7f736410c0f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.400000 2024-04-17T22:28:43.597 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:43.610 7f7362ffd700 1 --2- 172.21.15.102:0/2170063074 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f736410c720 0x7f73641a0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:28:43.597 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:43.610 7f73637fe700 1 --2- 172.21.15.102:0/2170063074 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f73640f5750 0x7f736410c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:28:43.597 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:43.610 7f7363fff700 1 --2- 172.21.15.102:0/2170063074 >> v2:172.21.15.102:3300/0 conn(0x7f736410cb70 0x7f73641a1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:28:43.597 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:43.610 7f7362ffd700 1 -- 172.21.15.102:0/2170063074 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f736410c720 msgr2=0x7f73641a0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-17T22:28:43.597 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:43.610 7f7362ffd700 1 -- 172.21.15.102:0/2170063074 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f736410c720 msgr2=0x7f73641a0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-17T22:28:43.598 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:43.610 7f73637fe700 1 -- 172.21.15.102:0/2170063074 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f73640f5750 msgr2=0x7f736410c0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-17T22:28:43.598 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:43.610 7f73637fe700 1 -- 172.21.15.102:0/2170063074 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f73640f5750 msgr2=0x7f736410c0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-17T22:28:43.598 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:43.610 7f7363fff700 1 -- 172.21.15.102:0/2170063074 >> v2:172.21.15.102:3300/0 conn(0x7f736410cb70 msgr2=0x7f73641a1210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-17T22:28:43.598 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:43.610 7f7363fff700 1 -- 172.21.15.102:0/2170063074 >> v2:172.21.15.102:3300/0 conn(0x7f736410cb70 msgr2=0x7f73641a1210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-17T22:28:43.598 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:43.610 7f7362ffd700 1 --2- 172.21.15.102:0/2170063074 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f736410c720 0x7f73641a0be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-04-17T22:28:43.598 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:43.610 7f73637fe700 1 --2- 172.21.15.102:0/2170063074 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f73640f5750 0x7f736410c0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-04-17T22:28:43.598 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:43.610 7f7363fff700 1 --2- 172.21.15.102:0/2170063074 >> v2:172.21.15.102:3300/0 conn(0x7f736410cb70 0x7f73641a1210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-04-17T22:28:43.598 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:43.610 7f73637fe700 1 --2- 172.21.15.102:0/2170063074 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f73640f5750 0x7f736410c0f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.800000 2024-04-17T22:28:43.598 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:43.610 7f7363fff700 1 --2- 172.21.15.102:0/2170063074 >> v2:172.21.15.102:3300/0 conn(0x7f736410cb70 0x7f73641a1210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.800000 2024-04-17T22:28:43.598 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:43.610 7f7362ffd700 1 --2- 172.21.15.102:0/2170063074 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f736410c720 0x7f73641a0be0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.800000 2024-04-17T22:28:44.398 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:44.412 7f7362ffd700 1 --2- 172.21.15.102:0/2170063074 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f736410c720 0x7f73641a0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:28:44.398 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:44.412 7f7363fff700 1 --2- 172.21.15.102:0/2170063074 >> v2:172.21.15.102:3300/0 conn(0x7f736410cb70 0x7f73641a1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:28:44.398 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:44.412 7f73637fe700 1 --2- 172.21.15.102:0/2170063074 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f73640f5750 0x7f736410c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:28:44.399 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:44.412 7f7362ffd700 1 -- 172.21.15.102:0/2170063074 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f736410c720 msgr2=0x7f73641a0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-17T22:28:44.399 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:44.412 7f73637fe700 1 -- 172.21.15.102:0/2170063074 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f73640f5750 msgr2=0x7f736410c0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-17T22:28:44.399 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:44.412 7f7363fff700 1 -- 172.21.15.102:0/2170063074 >> v2:172.21.15.102:3300/0 conn(0x7f736410cb70 msgr2=0x7f73641a1210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-17T22:28:44.399 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:44.412 7f73637fe700 1 -- 172.21.15.102:0/2170063074 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f73640f5750 msgr2=0x7f736410c0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-17T22:28:44.399 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:44.412 7f7362ffd700 1 -- 172.21.15.102:0/2170063074 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f736410c720 msgr2=0x7f73641a0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-17T22:28:44.399 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:44.412 7f7363fff700 1 -- 172.21.15.102:0/2170063074 >> v2:172.21.15.102:3300/0 conn(0x7f736410cb70 msgr2=0x7f73641a1210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-17T22:28:44.399 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:44.412 7f73637fe700 1 --2- 172.21.15.102:0/2170063074 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f73640f5750 0x7f736410c0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-04-17T22:28:44.399 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:44.412 7f7362ffd700 1 --2- 172.21.15.102:0/2170063074 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f736410c720 0x7f73641a0be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-04-17T22:28:44.399 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:44.412 7f7363fff700 1 --2- 172.21.15.102:0/2170063074 >> v2:172.21.15.102:3300/0 conn(0x7f736410cb70 0x7f73641a1210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-04-17T22:28:44.399 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:44.412 7f73637fe700 1 --2- 172.21.15.102:0/2170063074 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f73640f5750 0x7f736410c0f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 1.600000 2024-04-17T22:28:44.399 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:44.412 7f7363fff700 1 --2- 172.21.15.102:0/2170063074 >> v2:172.21.15.102:3300/0 conn(0x7f736410cb70 0x7f73641a1210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 1.600000 2024-04-17T22:28:44.399 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:44.412 7f7362ffd700 1 --2- 172.21.15.102:0/2170063074 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f736410c720 0x7f73641a0be0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 1.600000 2024-04-17T22:28:45.995 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:46.008 7f73617fa700 1 -- 172.21.15.102:0/2170063074 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f736410c720 msgr2=0x7f73641a0be0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-17T22:28:45.996 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:46.008 7f73617fa700 1 --2- 172.21.15.102:0/2170063074 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f736410c720 0x7f73641a0be0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:28:45.996 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:46.008 7f73617fa700 1 -- 172.21.15.102:0/2170063074 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f73640f5750 msgr2=0x7f736410c0f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-17T22:28:45.996 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:46.008 7f73617fa700 1 --2- 172.21.15.102:0/2170063074 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f73640f5750 0x7f736410c0f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:28:45.996 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:46.008 7f73617fa700 1 -- 172.21.15.102:0/2170063074 >> v2:172.21.15.102:3300/0 conn(0x7f736410cb70 msgr2=0x7f73641a1210 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-17T22:28:45.996 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:46.008 7f73617fa700 1 --2- 172.21.15.102:0/2170063074 >> v2:172.21.15.102:3300/0 conn(0x7f736410cb70 0x7f73641a1210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:28:45.997 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:46.008 7f73617fa700 1 --2- 172.21.15.102:0/2170063074 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f7350003270 0x7f73500056f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:28:45.997 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:46.008 7f73617fa700 1 --2- 172.21.15.102:0/2170063074 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f7350005d40 0x7f7350008200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:28:45.997 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:46.008 7f73617fa700 1 -- 172.21.15.102:0/2170063074 --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f735000c7d0 con 0x7f73500088d0 2024-04-17T22:28:45.997 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:46.008 7f73617fa700 1 -- 172.21.15.102:0/2170063074 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73640f93c0 con 0x7f7350005d40 2024-04-17T22:28:45.997 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:46.008 7f73617fa700 1 -- 172.21.15.102:0/2170063074 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f73641a1840 con 0x7f7350003270 2024-04-17T22:28:45.997 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:46.008 7f73637fe700 1 --2- 172.21.15.102:0/2170063074 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f7350003270 0x7f73500056f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:28:45.997 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:46.008 7f7362ffd700 1 --2- 172.21.15.102:0/2170063074 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f7350005d40 0x7f7350008200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:28:45.997 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:46.008 7f73637fe700 1 -- 172.21.15.102:0/2170063074 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f7350003270 msgr2=0x7f73500056f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-17T22:28:45.997 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:46.008 7f73637fe700 1 -- 172.21.15.102:0/2170063074 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f7350003270 msgr2=0x7f73500056f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-17T22:28:45.997 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:46.008 7f73637fe700 1 --2- 172.21.15.102:0/2170063074 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f7350003270 0x7f73500056f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-04-17T22:28:45.997 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:46.008 7f7362ffd700 1 -- 172.21.15.102:0/2170063074 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f7350005d40 msgr2=0x7f7350008200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-17T22:28:45.997 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:46.008 7f7362ffd700 1 -- 172.21.15.102:0/2170063074 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f7350005d40 msgr2=0x7f7350008200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-17T22:28:45.997 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:46.008 7f7362ffd700 1 --2- 172.21.15.102:0/2170063074 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f7350005d40 0x7f7350008200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-04-17T22:28:45.997 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:46.008 7f73637fe700 1 --2- 172.21.15.102:0/2170063074 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f7350003270 0x7f73500056f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2024-04-17T22:28:45.998 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:46.008 7f7362ffd700 1 --2- 172.21.15.102:0/2170063074 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f7350005d40 0x7f7350008200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2024-04-17T22:28:46.196 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:46.209 7f73637fe700 1 --2- 172.21.15.102:0/2170063074 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f7350003270 0x7f73500056f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:28:46.196 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:46.209 7f7362ffd700 1 --2- 172.21.15.102:0/2170063074 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f7350005d40 0x7f7350008200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:28:46.196 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:46.209 7f73637fe700 1 -- 172.21.15.102:0/2170063074 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f7350003270 msgr2=0x7f73500056f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-17T22:28:46.196 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:46.209 7f73637fe700 1 -- 172.21.15.102:0/2170063074 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f7350003270 msgr2=0x7f73500056f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-17T22:28:46.196 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:46.209 7f73637fe700 1 --2- 172.21.15.102:0/2170063074 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f7350003270 0x7f73500056f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-04-17T22:28:46.196 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:46.209 7f7362ffd700 1 -- 172.21.15.102:0/2170063074 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f7350005d40 msgr2=0x7f7350008200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-17T22:28:46.196 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:46.209 7f73637fe700 1 --2- 172.21.15.102:0/2170063074 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f7350003270 0x7f73500056f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.400000 2024-04-17T22:28:46.197 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:46.209 7f7362ffd700 1 -- 172.21.15.102:0/2170063074 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f7350005d40 msgr2=0x7f7350008200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-17T22:28:46.197 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:46.209 7f7362ffd700 1 --2- 172.21.15.102:0/2170063074 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f7350005d40 0x7f7350008200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-04-17T22:28:46.197 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:46.209 7f7362ffd700 1 --2- 172.21.15.102:0/2170063074 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f7350005d40 0x7f7350008200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.400000 2024-04-17T22:28:46.597 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:46.610 7f73637fe700 1 --2- 172.21.15.102:0/2170063074 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f7350003270 0x7f73500056f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:28:46.598 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:46.610 7f7362ffd700 1 --2- 172.21.15.102:0/2170063074 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f7350005d40 0x7f7350008200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:28:46.598 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:46.610 7f73637fe700 1 -- 172.21.15.102:0/2170063074 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f7350003270 msgr2=0x7f73500056f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-17T22:28:46.598 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:46.610 7f73637fe700 1 -- 172.21.15.102:0/2170063074 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f7350003270 msgr2=0x7f73500056f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-17T22:28:46.598 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:46.610 7f73637fe700 1 --2- 172.21.15.102:0/2170063074 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f7350003270 0x7f73500056f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-04-17T22:28:46.598 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:46.610 7f7362ffd700 1 -- 172.21.15.102:0/2170063074 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f7350005d40 msgr2=0x7f7350008200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-17T22:28:46.598 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:46.610 7f7362ffd700 1 -- 172.21.15.102:0/2170063074 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f7350005d40 msgr2=0x7f7350008200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-17T22:28:46.599 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:46.610 7f7362ffd700 1 --2- 172.21.15.102:0/2170063074 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f7350005d40 0x7f7350008200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-04-17T22:28:46.599 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:46.610 7f73637fe700 1 --2- 172.21.15.102:0/2170063074 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f7350003270 0x7f73500056f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.800000 2024-04-17T22:28:46.599 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:46.610 7f7362ffd700 1 --2- 172.21.15.102:0/2170063074 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f7350005d40 0x7f7350008200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.800000 2024-04-17T22:28:47.398 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:47.412 7f73637fe700 1 --2- 172.21.15.102:0/2170063074 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f7350003270 0x7f73500056f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:28:47.398 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:47.412 7f7362ffd700 1 --2- 172.21.15.102:0/2170063074 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f7350005d40 0x7f7350008200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:28:47.398 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:47.412 7f73637fe700 1 -- 172.21.15.102:0/2170063074 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f7350003270 msgr2=0x7f73500056f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-17T22:28:47.398 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:47.412 7f73637fe700 1 -- 172.21.15.102:0/2170063074 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f7350003270 msgr2=0x7f73500056f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-17T22:28:47.398 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:47.412 7f73637fe700 1 --2- 172.21.15.102:0/2170063074 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f7350003270 0x7f73500056f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-04-17T22:28:47.398 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:47.412 7f7362ffd700 1 -- 172.21.15.102:0/2170063074 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f7350005d40 msgr2=0x7f7350008200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-17T22:28:47.398 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:47.412 7f7362ffd700 1 -- 172.21.15.102:0/2170063074 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f7350005d40 msgr2=0x7f7350008200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-17T22:28:47.399 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:47.412 7f7362ffd700 1 --2- 172.21.15.102:0/2170063074 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f7350005d40 0x7f7350008200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-04-17T22:28:47.399 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:47.412 7f73637fe700 1 --2- 172.21.15.102:0/2170063074 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f7350003270 0x7f73500056f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 1.600000 2024-04-17T22:28:47.399 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:47.412 7f7362ffd700 1 --2- 172.21.15.102:0/2170063074 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f7350005d40 0x7f7350008200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 1.600000 2024-04-17T22:28:48.205 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:48.219 7f73627fc700 1 -- 172.21.15.102:0/2170063074 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1804211604 0 0) 0x7f7354006d80 con 0x7f73500088d0 2024-04-17T22:28:48.205 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:48.219 7f73627fc700 1 -- 172.21.15.102:0/2170063074 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7344003270 con 0x7f73500088d0 2024-04-17T22:28:48.208 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:48.219 7f73627fc700 1 -- 172.21.15.102:0/2170063074 <== mon.0 v1:172.21.15.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3679996247 0 0) 0x7f735400d300 con 0x7f73500088d0 2024-04-17T22:28:48.208 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:48.219 7f73627fc700 1 -- 172.21.15.102:0/2170063074 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f7350003270 msgr2=0x7f73500056f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-17T22:28:48.208 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:48.219 7f73627fc700 1 --2- 172.21.15.102:0/2170063074 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f7350003270 0x7f73500056f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:28:48.208 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:48.219 7f73627fc700 1 -- 172.21.15.102:0/2170063074 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f7350005d40 msgr2=0x7f7350008200 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-17T22:28:48.208 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:48.220 7f73627fc700 1 --2- 172.21.15.102:0/2170063074 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f7350005d40 0x7f7350008200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:28:48.208 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:48.220 7f73627fc700 1 -- 172.21.15.102:0/2170063074 --> v1:172.21.15.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73641a82f0 con 0x7f73500088d0 2024-04-17T22:28:48.208 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:48.220 7f73627fc700 1 -- 172.21.15.102:0/2170063074 <== mon.0 v1:172.21.15.102:6789/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (unknown 1481520889 0 0) 0x7f735400d720 con 0x7f73500088d0 2024-04-17T22:28:48.208 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:48.220 7f73627fc700 1 -- 172.21.15.102:0/2170063074 >> v1:172.21.15.102:6789/0 conn(0x7f73500088d0 legacy=0x7f735000adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:28:48.208 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:48.220 7f73627fc700 1 --2- 172.21.15.102:0/2170063074 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f7344003920 0x7f7344003d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:28:48.208 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:48.220 7f73627fc700 1 --2- 172.21.15.102:0/2170063074 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f73440045d0 0x7f7344004a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:28:48.209 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:48.220 7f73627fc700 1 --2- 172.21.15.102:0/2170063074 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f7344009810 0x7f734400bc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:28:48.209 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:48.220 7f73627fc700 1 -- 172.21.15.102:0/2170063074 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73640f93c0 con 0x7f73440045d0 2024-04-17T22:28:48.209 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:48.220 7f73627fc700 1 -- 172.21.15.102:0/2170063074 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73641a1840 con 0x7f7344003920 2024-04-17T22:28:48.209 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:48.220 7f73627fc700 1 -- 172.21.15.102:0/2170063074 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f73440106b0 con 0x7f7344009810 2024-04-17T22:28:48.209 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:48.220 7f7362ffd700 1 --2- 172.21.15.102:0/2170063074 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f73440045d0 0x7f7344004a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:28:48.209 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:48.220 7f7363fff700 1 --2- 172.21.15.102:0/2170063074 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f7344009810 0x7f734400bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:28:48.209 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:48.220 7f73637fe700 1 --2- 172.21.15.102:0/2170063074 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f7344003920 0x7f7344003d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:28:48.209 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:48.220 7f7362ffd700 1 -- 172.21.15.102:0/2170063074 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f7344009810 msgr2=0x7f734400bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:28:48.209 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:48.220 7f7362ffd700 1 --2- 172.21.15.102:0/2170063074 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f7344009810 0x7f734400bc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:28:48.209 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:48.220 7f7362ffd700 1 -- 172.21.15.102:0/2170063074 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f7344003920 msgr2=0x7f7344003d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:28:48.209 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:48.220 7f7362ffd700 1 --2- 172.21.15.102:0/2170063074 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f7344003920 0x7f7344003d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:28:48.209 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:48.220 7f7362ffd700 1 -- 172.21.15.102:0/2170063074 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7344010860 con 0x7f73440045d0 2024-04-17T22:28:48.209 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:48.220 7f7363fff700 1 --2- 172.21.15.102:0/2170063074 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f7344009810 0x7f734400bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:28:48.210 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:48.221 7f7362ffd700 1 --2- 172.21.15.102:0/2170063074 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f73440045d0 0x7f7344004a20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0x7f735800a000 tx=0x7f7358006f50).ready entity=mon.0 client_cookie=a9678d8a755f847e server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:28:48.210 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:48.221 7f73627fc700 1 -- 172.21.15.102:0/2170063074 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f7344010860 con 0x7f73440045d0 2024-04-17T22:28:48.210 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:48.221 7f73627fc700 1 -- 172.21.15.102:0/2170063074 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7358017b80 con 0x7f73440045d0 2024-04-17T22:28:48.210 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:48.221 7f73627fc700 1 -- 172.21.15.102:0/2170063074 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f735801f070 con 0x7f73440045d0 2024-04-17T22:28:48.210 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:48.221 7f736af3c700 1 -- 172.21.15.102:0/2170063074 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f73440045d0 msgr2=0x7f7344004a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:28:48.210 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:48.221 7f736af3c700 1 --2- 172.21.15.102:0/2170063074 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f73440045d0 0x7f7344004a20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0x7f735800a000 tx=0x7f7358006f50).stop 2024-04-17T22:28:48.210 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:48.221 7f7363fff700 1 -- 172.21.15.102:0/2170063074 reap_dead start 2024-04-17T22:28:48.210 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:48.222 7f736af3c700 1 -- 172.21.15.102:0/2170063074 shutdown_connections 2024-04-17T22:28:48.210 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:48.222 7f736af3c700 1 -- 172.21.15.102:0/2170063074 >> 172.21.15.102:0/2170063074 conn(0x7f73641138a0 msgr2=0x7f73640f4f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:28:48.210 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:48.222 7f736af3c700 1 -- 172.21.15.102:0/2170063074 shutdown_connections 2024-04-17T22:28:48.210 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:48.222 7f736af3c700 1 -- 172.21.15.102:0/2170063074 wait complete. 2024-04-17T22:28:48.231 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:48.245 7f736af3c700 1 Processor -- start 2024-04-17T22:28:48.231 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:48.246 7f736af3c700 1 -- start start 2024-04-17T22:28:48.232 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:48.246 7f736af3c700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f73640d4b40 0x7f73640d2ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:28:48.232 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:48.246 7f736af3c700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f73640d3100 0x7f73641b0570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:28:48.232 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:48.246 7f736af3c700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f73640d3550 0x7f73641b0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:28:48.232 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:48.246 7f736af3c700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73640d4540 con 0x7f73640d3550 2024-04-17T22:28:48.232 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:48.246 7f736af3c700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73640ce860 con 0x7f73640d4b40 2024-04-17T22:28:48.232 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:48.246 7f736af3c700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f73640cedf0 con 0x7f73640d3100 2024-04-17T22:28:48.232 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:48.246 7f7363fff700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f73640d3550 0x7f73641b0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:28:48.232 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:48.246 7f7363fff700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f73640d3550 0x7f73641b0ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:39774/0 (socket says 172.21.15.102:39774) 2024-04-17T22:28:48.232 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:48.246 7f7363fff700 1 -- 172.21.15.102:0/1984440211 learned_addr learned my addr 172.21.15.102:0/1984440211 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:28:48.233 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:48.246 7f7362ffd700 1 --2- 172.21.15.102:0/1984440211 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f73640d3100 0x7f73641b0570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:28:48.233 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:48.246 7f73637fe700 1 --2- 172.21.15.102:0/1984440211 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f73640d4b40 0x7f73640d2ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:28:48.234 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:48.247 7f7363fff700 1 -- 172.21.15.102:0/1984440211 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f73640d3100 msgr2=0x7f73641b0570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:28:48.234 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:48.247 7f7363fff700 1 --2- 172.21.15.102:0/1984440211 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f73640d3100 0x7f73641b0570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:28:48.234 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:48.247 7f7363fff700 1 -- 172.21.15.102:0/1984440211 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f73640d4b40 msgr2=0x7f73640d2ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:28:48.234 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:48.247 7f7363fff700 1 --2- 172.21.15.102:0/1984440211 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f73640d4b40 0x7f73640d2ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:28:48.234 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:48.247 7f7363fff700 1 -- 172.21.15.102:0/1984440211 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73640d5da0 con 0x7f73640d3550 2024-04-17T22:28:48.234 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:48.247 7f7363fff700 1 --2- 172.21.15.102:0/1984440211 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f73640d3550 0x7f73641b0ba0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 rx=0x7f73640c31d0 tx=0x7f7354006830).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:28:48.234 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:48.247 7f7360ff9700 1 -- 172.21.15.102:0/1984440211 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f73640d5da0 con 0x7f73640d3550 2024-04-17T22:28:48.234 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:48.247 7f7360ff9700 1 -- 172.21.15.102:0/1984440211 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7354016b30 con 0x7f73640d3550 2024-04-17T22:28:48.235 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:48.247 7f736af3c700 1 -- 172.21.15.102:0/1984440211 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73641b11d0 con 0x7f73640d3550 2024-04-17T22:28:48.235 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:48.247 7f7360ff9700 1 -- 172.21.15.102:0/1984440211 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f735400f040 con 0x7f73640d3550 2024-04-17T22:28:48.235 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:48.247 7f736af3c700 1 -- 172.21.15.102:0/1984440211 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73641b79a0 con 0x7f73640d3550 2024-04-17T22:28:48.235 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:48.247 7f7360ff9700 1 -- 172.21.15.102:0/1984440211 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 1) v1 ==== 190+0+0 (secure 0 0 0) 0x7f7354017440 con 0x7f73640d3550 2024-04-17T22:28:48.235 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:48.248 7f7360ff9700 1 -- 172.21.15.102:0/1984440211 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f7354017be0 con 0x7f73640d3550 2024-04-17T22:28:48.235 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:48.248 7f736af3c700 1 -- 172.21.15.102:0/1984440211 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73640f0800 con 0x7f73640d3550 2024-04-17T22:28:48.236 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:48.251 7f7360ff9700 1 -- 172.21.15.102:0/1984440211 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+66062 (secure 0 0 0) 0x7f7354016730 con 0x7f73640d3550 2024-04-17T22:28:48.302 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:48.316 7f736af3c700 1 -- 172.21.15.102:0/1984440211 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f73640abc60 con 0x7f73640d3550 2024-04-17T22:28:48.307 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:48.321 7f7360ff9700 1 -- 172.21.15.102:0/1984440211 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7f7364114420 con 0x7f73640d3550 2024-04-17T22:28:48.308 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:48.322 7f736af3c700 1 -- 172.21.15.102:0/1984440211 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f73640d3550 msgr2=0x7f73641b0ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:28:48.308 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:48.322 7f736af3c700 1 --2- 172.21.15.102:0/1984440211 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f73640d3550 0x7f73641b0ba0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 rx=0x7f73640c31d0 tx=0x7f7354006830).stop 2024-04-17T22:28:48.308 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:48.322 7f736af3c700 1 -- 172.21.15.102:0/1984440211 shutdown_connections 2024-04-17T22:28:48.309 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:48.322 7f736af3c700 1 --2- 172.21.15.102:0/1984440211 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f73640d3550 0x7f73641b0ba0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:28:48.309 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:48.322 7f736af3c700 1 --2- 172.21.15.102:0/1984440211 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f73640d4b40 0x7f73640d2ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:28:48.309 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:48.322 7f736af3c700 1 --2- 172.21.15.102:0/1984440211 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f73640d3100 0x7f73641b0570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:28:48.309 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:48.322 7f736af3c700 1 -- 172.21.15.102:0/1984440211 >> 172.21.15.102:0/1984440211 conn(0x7f73640c59d0 msgr2=0x7f73640bd900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:28:48.309 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:48.322 7f736af3c700 1 -- 172.21.15.102:0/1984440211 shutdown_connections 2024-04-17T22:28:48.309 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:48.322 7f736af3c700 1 -- 172.21.15.102:0/1984440211 wait complete. 2024-04-17T22:28:48.309 INFO:teuthology.orchestra.run.smithi102.stderr:adjusted tunables profile to default 2024-04-17T22:28:48.319 INFO:tasks.ceph:check_enable_crimson: False 2024-04-17T22:28:48.319 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-04-17T22:28:48.320 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-04-17T22:28:48.320 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-04-17T22:28:48.348 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-04-17T22:28:48.348 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-04-17T22:28:48.415 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-04-17T22:28:48.415 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-04-17T22:28:48.481 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-04-17T22:28:48.481 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-04-17T22:28:48.547 DEBUG:teuthology.orchestra.run.smithi102:> sudo ceph --cluster ceph osd new 3351ef85-ade4-4b90-a8f4-213f4e00ba01 0 2024-04-17T22:28:48.714 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:48.727 7f18338b6700 1 Processor -- start 2024-04-17T22:28:48.715 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:48.727 7f18338b6700 1 -- start start 2024-04-17T22:28:48.715 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:48.727 7f18338b6700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f182c0f5840 0x7f182c10b010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:28:48.715 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:48.727 7f18338b6700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f182c10ba90 0x7f182c1a02f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:28:48.715 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:48.727 7f18338b6700 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f182c103b40 con 0x7f182c10b640 2024-04-17T22:28:48.715 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:48.727 7f18338b6700 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f182c10bee0 con 0x7f182c10ba90 2024-04-17T22:28:48.715 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:48.727 7f18338b6700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f182c0fa370 con 0x7f182c0f5840 2024-04-17T22:28:48.715 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:48.727 7f1830eb2700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f182c10ba90 0x7f182c1a02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:28:48.715 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:48.727 7f182bfff700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f182c0f5840 0x7f182c10b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:28:48.715 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:48.727 7f1830eb2700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f182c10ba90 0x7f182c1a02f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:39778/0 (socket says 172.21.15.102:39778) 2024-04-17T22:28:48.715 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:48.727 7f1830eb2700 1 -- 172.21.15.102:0/3225295797 learned_addr learned my addr 172.21.15.102:0/3225295797 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:28:48.715 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:48.728 7f182bfff700 1 -- 172.21.15.102:0/3225295797 >> v2:172.21.15.102:3300/0 conn(0x7f182c10ba90 msgr2=0x7f182c1a02f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:28:48.715 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:48.728 7f182bfff700 1 --2- 172.21.15.102:0/3225295797 >> v2:172.21.15.102:3300/0 conn(0x7f182c10ba90 0x7f182c1a02f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:28:48.716 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:48.728 7f182bfff700 1 -- 172.21.15.102:0/3225295797 >> v1:172.21.15.102:6789/0 conn(0x7f182c10b640 legacy=0x7f182c19faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:28:48.716 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:48.728 7f182bfff700 1 -- 172.21.15.102:0/3225295797 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f182c1a0920 con 0x7f182c0f5840 2024-04-17T22:28:48.716 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:48.728 7f1830eb2700 1 --2- 172.21.15.102:0/3225295797 >> v2:172.21.15.102:3300/0 conn(0x7f182c10ba90 0x7f182c1a02f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:28:48.716 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:48.728 7f182bfff700 1 --2- 172.21.15.102:0/3225295797 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f182c0f5840 0x7f182c10b010 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0x7f181c002f70 tx=0x7f181c00dd30).ready entity=mon.1 client_cookie=a3782cc73fa97b64 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:28:48.716 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:48.728 7f182b7fe700 1 -- 172.21.15.102:0/3225295797 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f182c1a0920 con 0x7f182c0f5840 2024-04-17T22:28:48.716 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:48.728 7f182b7fe700 1 -- 172.21.15.102:0/3225295797 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f181c00f7d0 con 0x7f182c0f5840 2024-04-17T22:28:48.716 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:48.728 7f182b7fe700 1 -- 172.21.15.102:0/3225295797 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f181c013070 con 0x7f182c0f5840 2024-04-17T22:28:48.716 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:48.728 7f18338b6700 1 -- 172.21.15.102:0/3225295797 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f182c0f5840 msgr2=0x7f182c10b010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:28:48.716 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:48.728 7f18338b6700 1 --2- 172.21.15.102:0/3225295797 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f182c0f5840 0x7f182c10b010 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0x7f181c002f70 tx=0x7f181c00dd30).stop 2024-04-17T22:28:48.716 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:48.729 7f18338b6700 1 -- 172.21.15.102:0/3225295797 shutdown_connections 2024-04-17T22:28:48.716 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:48.729 7f18338b6700 1 --2- 172.21.15.102:0/3225295797 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f182c0f5840 0x7f182c10b010 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:28:48.716 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:48.729 7f18338b6700 1 --2- 172.21.15.102:0/3225295797 >> v2:172.21.15.102:3300/0 conn(0x7f182c10ba90 0x7f182c1a02f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:28:48.716 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:48.729 7f18338b6700 1 -- 172.21.15.102:0/3225295797 >> 172.21.15.102:0/3225295797 conn(0x7f182c1138a0 msgr2=0x7f182c0f4f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:28:48.717 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:48.729 7f18338b6700 1 -- 172.21.15.102:0/3225295797 shutdown_connections 2024-04-17T22:28:48.717 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:48.729 7f18338b6700 1 -- 172.21.15.102:0/3225295797 wait complete. 2024-04-17T22:28:48.739 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:48.752 7f18338b6700 1 Processor -- start 2024-04-17T22:28:48.740 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:48.752 7f18338b6700 1 -- start start 2024-04-17T22:28:48.740 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:48.752 7f18338b6700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f182c0d72e0 0x7f182c0ce1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:28:48.740 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:48.752 7f18338b6700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f182c0ce810 0x7f182c1b0100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:28:48.740 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:48.752 7f18338b6700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f182c0cec60 0x7f182c1b0730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:28:48.740 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:48.752 7f18338b6700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f182c0d6c20 con 0x7f182c0cec60 2024-04-17T22:28:48.740 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:48.752 7f18338b6700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f182c1b0d60 con 0x7f182c0ce810 2024-04-17T22:28:48.740 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:48.752 7f18338b6700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f182c1b0f90 con 0x7f182c0d72e0 2024-04-17T22:28:48.740 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:48.752 7f1830eb2700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f182c0cec60 0x7f182c1b0730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:28:48.740 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:48.752 7f182bfff700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f182c0d72e0 0x7f182c0ce1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:28:48.740 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:48.752 7f1823fff700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f182c0ce810 0x7f182c1b0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:28:48.740 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:48.752 7f1830eb2700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f182c0cec60 0x7f182c1b0730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:39784/0 (socket says 172.21.15.102:39784) 2024-04-17T22:28:48.740 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:48.752 7f1830eb2700 1 -- 172.21.15.102:0/823613157 learned_addr learned my addr 172.21.15.102:0/823613157 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:28:48.741 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:48.753 7f182bfff700 1 -- 172.21.15.102:0/823613157 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f182c0ce810 msgr2=0x7f182c1b0100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:28:48.741 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:48.753 7f182bfff700 1 --2- 172.21.15.102:0/823613157 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f182c0ce810 0x7f182c1b0100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:28:48.741 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:48.753 7f182bfff700 1 -- 172.21.15.102:0/823613157 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f182c0cec60 msgr2=0x7f182c1b0730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:28:48.741 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:48.753 7f182bfff700 1 --2- 172.21.15.102:0/823613157 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f182c0cec60 0x7f182c1b0730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:28:48.741 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:48.753 7f182bfff700 1 -- 172.21.15.102:0/823613157 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f182c1b77f0 con 0x7f182c0d72e0 2024-04-17T22:28:48.741 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:48.753 7f1823fff700 1 --2- 172.21.15.102:0/823613157 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f182c0ce810 0x7f182c1b0100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:28:48.741 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:48.753 7f1830eb2700 1 --2- 172.21.15.102:0/823613157 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f182c0cec60 0x7f182c1b0730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:28:48.741 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:48.753 7f182bfff700 1 --2- 172.21.15.102:0/823613157 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f182c0d72e0 0x7f182c0ce1e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0x7f181c002f70 tx=0x7f181c0068c0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:28:48.741 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:48.753 7f1829ffb700 1 -- 172.21.15.102:0/823613157 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f182c1b77f0 con 0x7f182c0d72e0 2024-04-17T22:28:48.741 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:48.753 7f1829ffb700 1 -- 172.21.15.102:0/823613157 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f181c017c40 con 0x7f182c0d72e0 2024-04-17T22:28:48.741 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:48.753 7f1829ffb700 1 -- 172.21.15.102:0/823613157 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f181c00b640 con 0x7f182c0d72e0 2024-04-17T22:28:48.741 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:48.753 7f18338b6700 1 -- 172.21.15.102:0/823613157 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f182c1b79f0 con 0x7f182c0d72e0 2024-04-17T22:28:48.741 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:48.753 7f18338b6700 1 -- 172.21.15.102:0/823613157 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f182c1b7bf0 con 0x7f182c0d72e0 2024-04-17T22:28:48.741 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:48.753 7f1829ffb700 1 -- 172.21.15.102:0/823613157 <== mon.2 v2:172.21.15.102:3302/0 4 ==== mgrmap(e 1) v1 ==== 190+0+0 (secure 0 0 0) 0x7f181c02b060 con 0x7f182c0d72e0 2024-04-17T22:28:48.742 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:48.753 7f1829ffb700 1 -- 172.21.15.102:0/823613157 <== mon.2 v2:172.21.15.102:3302/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 721+0+0 (secure 0 0 0) 0x7f181c00bb90 con 0x7f182c0d72e0 2024-04-17T22:28:48.742 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:48.754 7f18338b6700 1 -- 172.21.15.102:0/823613157 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f182c0dd1e0 con 0x7f182c0d72e0 2024-04-17T22:28:48.742 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:48.756 7f1829ffb700 1 -- 172.21.15.102:0/823613157 <== mon.2 v2:172.21.15.102:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+66062 (secure 0 0 0) 0x7f181c017840 con 0x7f182c0d72e0 2024-04-17T22:28:48.813 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:48.827 7f18338b6700 1 -- 172.21.15.102:0/823613157 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "osd new", "uuid": "3351ef85-ade4-4b90-a8f4-213f4e00ba01", "id": 0} v 0) v1 -- 0x7f182c0f0800 con 0x7f182c0d72e0 2024-04-17T22:28:48.823 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:48.838 7f1829ffb700 1 -- 172.21.15.102:0/823613157 <== mon.2 v2:172.21.15.102:3302/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "3351ef85-ade4-4b90-a8f4-213f4e00ba01", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7f181c00d0d0 con 0x7f182c0d72e0 2024-04-17T22:28:48.823 INFO:teuthology.orchestra.run.smithi102.stdout:0 2024-04-17T22:28:48.824 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:48.838 7f18338b6700 1 -- 172.21.15.102:0/823613157 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f182c0d72e0 msgr2=0x7f182c0ce1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:28:48.825 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:48.838 7f18338b6700 1 --2- 172.21.15.102:0/823613157 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f182c0d72e0 0x7f182c0ce1e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0x7f181c002f70 tx=0x7f181c0068c0).stop 2024-04-17T22:28:48.825 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:48.839 7f18338b6700 1 -- 172.21.15.102:0/823613157 shutdown_connections 2024-04-17T22:28:48.825 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:48.839 7f18338b6700 1 --2- 172.21.15.102:0/823613157 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f182c0cec60 0x7f182c1b0730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:28:48.825 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:48.839 7f18338b6700 1 --2- 172.21.15.102:0/823613157 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f182c0d72e0 0x7f182c0ce1e0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:28:48.825 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:48.839 7f18338b6700 1 --2- 172.21.15.102:0/823613157 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f182c0ce810 0x7f182c1b0100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:28:48.825 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:48.839 7f18338b6700 1 -- 172.21.15.102:0/823613157 >> 172.21.15.102:0/823613157 conn(0x7f182c0c52c0 msgr2=0x7f182c1a42d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:28:48.825 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:48.839 7f18338b6700 1 -- 172.21.15.102:0/823613157 shutdown_connections 2024-04-17T22:28:48.825 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:48.839 7f18338b6700 1 -- 172.21.15.102:0/823613157 wait complete. 2024-04-17T22:28:48.835 DEBUG:teuthology.orchestra.run.smithi102:> sudo ceph --cluster ceph osd new 04f67eac-f36b-48f5-982d-4cfc776596d0 1 2024-04-17T22:28:48.962 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:48.975 7f967f799700 1 Processor -- start 2024-04-17T22:28:48.962 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:48.975 7f967f799700 1 -- start start 2024-04-17T22:28:48.962 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:48.975 7f967f799700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f96781a41f0 0x7f96781a6650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:28:48.962 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:48.975 7f967f799700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f96781a6c80 0x7f96781a90e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:28:48.962 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:48.975 7f967f799700 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9678111fb0 con 0x7f96780fb170 2024-04-17T22:28:48.962 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:48.975 7f967f799700 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96780f23e0 con 0x7f96781a6c80 2024-04-17T22:28:48.962 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:48.975 7f967f799700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f96781067f0 con 0x7f96781a41f0 2024-04-17T22:28:48.963 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:48.975 7f966ffff700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f96781a41f0 0x7f96781a6650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:28:48.963 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:48.975 7f967cd95700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f96781a6c80 0x7f96781a90e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:28:48.963 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:48.975 7f966ffff700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f96781a41f0 0x7f96781a6650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:50402/0 (socket says 172.21.15.102:50402) 2024-04-17T22:28:48.963 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:48.975 7f966ffff700 1 -- 172.21.15.102:0/1890907698 learned_addr learned my addr 172.21.15.102:0/1890907698 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:28:48.963 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:48.976 7f967cd95700 1 -- 172.21.15.102:0/1890907698 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f96781a41f0 msgr2=0x7f96781a6650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:28:48.963 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:48.976 7f967cd95700 1 --2- 172.21.15.102:0/1890907698 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f96781a41f0 0x7f96781a6650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:28:48.963 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:48.976 7f967cd95700 1 -- 172.21.15.102:0/1890907698 >> v1:172.21.15.102:6789/0 conn(0x7f96780fb170 legacy=0x7f96780ff810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:28:48.963 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:48.976 7f96777fe700 1 -- 172.21.15.102:0/1890907698 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2233846392 0 0) 0x7f9668001110 con 0x7f96780fb170 2024-04-17T22:28:48.963 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:48.976 7f967cd95700 1 -- 172.21.15.102:0/1890907698 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96781aa720 con 0x7f96781a6c80 2024-04-17T22:28:48.963 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:48.976 7f966ffff700 1 --2- 172.21.15.102:0/1890907698 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f96781a41f0 0x7f96781a6650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:28:48.963 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:48.976 7f967cd95700 1 --2- 172.21.15.102:0/1890907698 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f96781a6c80 0x7f96781a90e0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 rx=0x7f96700032f0 tx=0x7f9670009d10).ready entity=mon.0 client_cookie=b566be238301e806 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:28:48.963 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:48.976 7f96777fe700 1 -- 172.21.15.102:0/1890907698 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f96781aa720 con 0x7f96781a6c80 2024-04-17T22:28:48.963 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:48.976 7f96777fe700 1 -- 172.21.15.102:0/1890907698 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f967000b800 con 0x7f96781a6c80 2024-04-17T22:28:48.963 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:48.976 7f96777fe700 1 -- 172.21.15.102:0/1890907698 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f967000b9e0 con 0x7f96781a6c80 2024-04-17T22:28:48.964 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:48.976 7f967f799700 1 -- 172.21.15.102:0/1890907698 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f96781a6c80 msgr2=0x7f96781a90e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:28:48.964 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:48.976 7f967f799700 1 --2- 172.21.15.102:0/1890907698 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f96781a6c80 0x7f96781a90e0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 rx=0x7f96700032f0 tx=0x7f9670009d10).stop 2024-04-17T22:28:48.964 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:48.977 7f967f799700 1 -- 172.21.15.102:0/1890907698 shutdown_connections 2024-04-17T22:28:48.964 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:48.977 7f967f799700 1 --2- 172.21.15.102:0/1890907698 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f96781a6c80 0x7f96781a90e0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:28:48.965 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:48.977 7f967f799700 1 --2- 172.21.15.102:0/1890907698 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f96781a41f0 0x7f96781a6650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:28:48.965 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:48.977 7f967f799700 1 -- 172.21.15.102:0/1890907698 >> 172.21.15.102:0/1890907698 conn(0x7f9678114950 msgr2=0x7f9678114da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:28:48.965 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:48.977 7f967f799700 1 -- 172.21.15.102:0/1890907698 shutdown_connections 2024-04-17T22:28:48.965 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:48.977 7f967f799700 1 -- 172.21.15.102:0/1890907698 wait complete. 2024-04-17T22:28:48.988 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.000 7f967f799700 1 Processor -- start 2024-04-17T22:28:48.988 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.000 7f967f799700 1 -- start start 2024-04-17T22:28:48.988 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.000 7f967f799700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f96781a64a0 0x7f96781ad830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:28:48.988 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.000 7f967f799700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f96781ade60 0x7f96781b4140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:28:48.988 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.000 7f967f799700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f96781ae2b0 0x7f96781b4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:28:48.988 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.000 7f967f799700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96780ac410 con 0x7f96781a64a0 2024-04-17T22:28:48.988 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.000 7f967f799700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96780cfa00 con 0x7f96781ade60 2024-04-17T22:28:48.988 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.000 7f967f799700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f96781b4da0 con 0x7f96781ae2b0 2024-04-17T22:28:48.988 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.000 7f966ffff700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f96781ade60 0x7f96781b4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:28:48.988 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.000 7f966ffff700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f96781ade60 0x7f96781b4140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:59964/0 (socket says 172.21.15.102:59964) 2024-04-17T22:28:48.988 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.000 7f966ffff700 1 -- 172.21.15.102:0/2360345096 learned_addr learned my addr 172.21.15.102:0/2360345096 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:28:48.988 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.000 7f9677fff700 1 --2- 172.21.15.102:0/2360345096 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f96781a64a0 0x7f96781ad830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:28:48.989 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.000 7f967cd95700 1 --2- 172.21.15.102:0/2360345096 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f96781ae2b0 0x7f96781b4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:28:48.989 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.001 7f966ffff700 1 -- 172.21.15.102:0/2360345096 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f96781ae2b0 msgr2=0x7f96781b4770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:28:48.989 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.001 7f966ffff700 1 --2- 172.21.15.102:0/2360345096 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f96781ae2b0 0x7f96781b4770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:28:48.989 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.001 7f966ffff700 1 -- 172.21.15.102:0/2360345096 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f96781a64a0 msgr2=0x7f96781ad830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:28:48.989 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.001 7f966ffff700 1 --2- 172.21.15.102:0/2360345096 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f96781a64a0 0x7f96781ad830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:28:48.989 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.001 7f966ffff700 1 -- 172.21.15.102:0/2360345096 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96781b4fd0 con 0x7f96781ade60 2024-04-17T22:28:48.989 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.001 7f966ffff700 1 --2- 172.21.15.102:0/2360345096 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f96781ade60 0x7f96781b4140 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0x7f966000c000 tx=0x7f966000ea90).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:28:48.989 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.001 7f9675ffb700 1 -- 172.21.15.102:0/2360345096 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f96781b4fd0 con 0x7f96781ade60 2024-04-17T22:28:48.989 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.001 7f9675ffb700 1 -- 172.21.15.102:0/2360345096 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9660012600 con 0x7f96781ade60 2024-04-17T22:28:48.989 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.001 7f967f799700 1 -- 172.21.15.102:0/2360345096 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96781bb940 con 0x7f96781ade60 2024-04-17T22:28:48.989 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.001 7f9675ffb700 1 -- 172.21.15.102:0/2360345096 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f966000f040 con 0x7f96781ade60 2024-04-17T22:28:48.989 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.001 7f967f799700 1 -- 172.21.15.102:0/2360345096 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96781bbd20 con 0x7f96781ade60 2024-04-17T22:28:48.989 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.001 7f9675ffb700 1 -- 172.21.15.102:0/2360345096 <== mon.1 v2:172.21.15.102:3301/0 4 ==== mgrmap(e 1) v1 ==== 190+0+0 (secure 0 0 0) 0x7f9660022060 con 0x7f96781ade60 2024-04-17T22:28:48.989 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.001 7f9675ffb700 1 -- 172.21.15.102:0/2360345096 <== mon.1 v2:172.21.15.102:3301/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 824+0+0 (secure 0 0 0) 0x7f9660010040 con 0x7f96781ade60 2024-04-17T22:28:48.990 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.002 7f967f799700 1 -- 172.21.15.102:0/2360345096 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96780f09f0 con 0x7f96781ade60 2024-04-17T22:28:48.990 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.004 7f9675ffb700 1 -- 172.21.15.102:0/2360345096 <== mon.1 v2:172.21.15.102:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+66062 (secure 0 0 0) 0x7f96600123f0 con 0x7f96781ade60 2024-04-17T22:28:49.061 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.076 7f967f799700 1 -- 172.21.15.102:0/2360345096 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "osd new", "uuid": "04f67eac-f36b-48f5-982d-4cfc776596d0", "id": 1} v 0) v1 -- 0x7f96780f00c0 con 0x7f96781ade60 2024-04-17T22:28:49.072 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.086 7f9675ffb700 1 -- 172.21.15.102:0/2360345096 <== mon.1 v2:172.21.15.102:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "04f67eac-f36b-48f5-982d-4cfc776596d0", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7f9660000c30 con 0x7f96781ade60 2024-04-17T22:28:49.072 INFO:teuthology.orchestra.run.smithi102.stdout:1 2024-04-17T22:28:49.073 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.087 7f967f799700 1 -- 172.21.15.102:0/2360345096 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f96781ade60 msgr2=0x7f96781b4140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:28:49.073 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.087 7f967f799700 1 --2- 172.21.15.102:0/2360345096 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f96781ade60 0x7f96781b4140 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0x7f966000c000 tx=0x7f966000ea90).stop 2024-04-17T22:28:49.073 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.087 7f967f799700 1 -- 172.21.15.102:0/2360345096 shutdown_connections 2024-04-17T22:28:49.073 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.087 7f967f799700 1 --2- 172.21.15.102:0/2360345096 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f96781ae2b0 0x7f96781b4770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:28:49.073 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.087 7f967f799700 1 --2- 172.21.15.102:0/2360345096 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f96781a64a0 0x7f96781ad830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:28:49.073 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.087 7f967f799700 1 --2- 172.21.15.102:0/2360345096 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f96781ade60 0x7f96781b4140 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:28:49.073 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.087 7f967f799700 1 -- 172.21.15.102:0/2360345096 >> 172.21.15.102:0/2360345096 conn(0x7f96780c3a90 msgr2=0x7f96780db030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:28:49.074 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.087 7f967f799700 1 -- 172.21.15.102:0/2360345096 shutdown_connections 2024-04-17T22:28:49.074 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.087 7f967f799700 1 -- 172.21.15.102:0/2360345096 wait complete. 2024-04-17T22:28:49.084 DEBUG:teuthology.orchestra.run.smithi102:> sudo ceph --cluster ceph osd new d524204b-70b2-40a8-8f73-f9e7d44318af 2 2024-04-17T22:28:49.212 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.224 7fdf36a3e700 1 Processor -- start 2024-04-17T22:28:49.212 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.224 7fdf36a3e700 1 -- start start 2024-04-17T22:28:49.212 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.225 7fdf36a3e700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fdf3010e790 0x7fdf3010ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:28:49.212 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.225 7fdf36a3e700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fdf301038e0 0x7fdf301a83f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:28:49.212 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.225 7fdf36a3e700 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf300ad030 con 0x7fdf3010f210 2024-04-17T22:28:49.213 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.225 7fdf36a3e700 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf30104630 con 0x7fdf3010e790 2024-04-17T22:28:49.213 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.225 7fdf36a3e700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdf300f2ae0 con 0x7fdf301038e0 2024-04-17T22:28:49.213 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.225 7fdf2f7fe700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fdf3010e790 0x7fdf3010ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:28:49.213 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.225 7fdf2f7fe700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fdf3010e790 0x7fdf3010ebe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:39808/0 (socket says 172.21.15.102:39808) 2024-04-17T22:28:49.213 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.225 7fdf2f7fe700 1 -- 172.21.15.102:0/2300071268 learned_addr learned my addr 172.21.15.102:0/2300071268 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:28:49.213 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.225 7fdf2ffff700 1 --2- 172.21.15.102:0/2300071268 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fdf301038e0 0x7fdf301a83f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:28:49.213 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.225 7fdf2f7fe700 1 -- 172.21.15.102:0/2300071268 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fdf301038e0 msgr2=0x7fdf301a83f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:28:49.213 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.225 7fdf2f7fe700 1 --2- 172.21.15.102:0/2300071268 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fdf301038e0 0x7fdf301a83f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:28:49.213 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.225 7fdf2f7fe700 1 -- 172.21.15.102:0/2300071268 >> v1:172.21.15.102:6789/0 conn(0x7fdf3010f210 legacy=0x7fdf301030e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:28:49.213 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.225 7fdf2f7fe700 1 -- 172.21.15.102:0/2300071268 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf30103d30 con 0x7fdf3010e790 2024-04-17T22:28:49.213 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.225 7fdf2ffff700 1 --2- 172.21.15.102:0/2300071268 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fdf301038e0 0x7fdf301a83f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:28:49.213 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.225 7fdf2f7fe700 1 --2- 172.21.15.102:0/2300071268 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdf3010e790 0x7fdf3010ebe0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 rx=0x7fdf28002f70 tx=0x7fdf28009a50).ready entity=mon.0 client_cookie=bbbcab7b504c91f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:28:49.213 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.225 7fdf2e7fc700 1 -- 172.21.15.102:0/2300071268 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fdf30103d30 con 0x7fdf3010e790 2024-04-17T22:28:49.213 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.225 7fdf2e7fc700 1 -- 172.21.15.102:0/2300071268 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf2800f510 con 0x7fdf3010e790 2024-04-17T22:28:49.214 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.225 7fdf2e7fc700 1 -- 172.21.15.102:0/2300071268 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fdf2800f6f0 con 0x7fdf3010e790 2024-04-17T22:28:49.214 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.225 7fdf36a3e700 1 -- 172.21.15.102:0/2300071268 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdf3010e790 msgr2=0x7fdf3010ebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:28:49.214 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.225 7fdf36a3e700 1 --2- 172.21.15.102:0/2300071268 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdf3010e790 0x7fdf3010ebe0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 rx=0x7fdf28002f70 tx=0x7fdf28009a50).stop 2024-04-17T22:28:49.214 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.226 7fdf36a3e700 1 -- 172.21.15.102:0/2300071268 shutdown_connections 2024-04-17T22:28:49.214 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.226 7fdf36a3e700 1 --2- 172.21.15.102:0/2300071268 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fdf301038e0 0x7fdf301a83f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:28:49.214 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.226 7fdf36a3e700 1 --2- 172.21.15.102:0/2300071268 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdf3010e790 0x7fdf3010ebe0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:28:49.214 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.226 7fdf36a3e700 1 -- 172.21.15.102:0/2300071268 >> 172.21.15.102:0/2300071268 conn(0x7fdf30113000 msgr2=0x7fdf30113450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:28:49.214 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.226 7fdf36a3e700 1 -- 172.21.15.102:0/2300071268 shutdown_connections 2024-04-17T22:28:49.214 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.226 7fdf36a3e700 1 -- 172.21.15.102:0/2300071268 wait complete. 2024-04-17T22:28:49.235 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.249 7fdf36a3e700 1 Processor -- start 2024-04-17T22:28:49.235 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.249 7fdf36a3e700 1 -- start start 2024-04-17T22:28:49.237 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.250 7fdf36a3e700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdf301a1de0 0x7fdf301a9c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:28:49.238 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.250 7fdf36a3e700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fdf301aa230 0x7fdf301b3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:28:49.238 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.250 7fdf36a3e700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fdf301aa680 0x7fdf301b41f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:28:49.238 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.250 7fdf36a3e700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf300cd9a0 con 0x7fdf301a1de0 2024-04-17T22:28:49.238 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.250 7fdf36a3e700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf301a2230 con 0x7fdf301aa680 2024-04-17T22:28:49.238 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.250 7fdf36a3e700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdf301a1a90 con 0x7fdf301aa230 2024-04-17T22:28:49.238 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.250 7fdf2f7fe700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdf301a1de0 0x7fdf301a9c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:28:49.238 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.250 7fdf2ffff700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fdf301aa680 0x7fdf301b41f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:28:49.238 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.250 7fdf2effd700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fdf301aa230 0x7fdf301b3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:28:49.238 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.250 7fdf2f7fe700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdf301a1de0 0x7fdf301a9c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:39814/0 (socket says 172.21.15.102:39814) 2024-04-17T22:28:49.238 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.250 7fdf2f7fe700 1 -- 172.21.15.102:0/2817405211 learned_addr learned my addr 172.21.15.102:0/2817405211 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:28:49.238 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.250 7fdf2ffff700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fdf301aa680 0x7fdf301b41f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:59978/0 (socket says 172.21.15.102:59978) 2024-04-17T22:28:49.238 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.250 7fdf2ffff700 1 -- 172.21.15.102:0/2817405211 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fdf301aa230 msgr2=0x7fdf301b3bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:28:49.238 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.250 7fdf2ffff700 1 --2- 172.21.15.102:0/2817405211 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fdf301aa230 0x7fdf301b3bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:28:49.238 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.250 7fdf2ffff700 1 -- 172.21.15.102:0/2817405211 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdf301a1de0 msgr2=0x7fdf301a9c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:28:49.239 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.250 7fdf2ffff700 1 --2- 172.21.15.102:0/2817405211 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdf301a1de0 0x7fdf301a9c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:28:49.239 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.250 7fdf2ffff700 1 -- 172.21.15.102:0/2817405211 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf301b4820 con 0x7fdf301aa680 2024-04-17T22:28:49.239 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.250 7fdf2f7fe700 1 --2- 172.21.15.102:0/2817405211 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdf301a1de0 0x7fdf301a9c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:28:49.239 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.250 7fdf2effd700 1 --2- 172.21.15.102:0/2817405211 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fdf301aa230 0x7fdf301b3bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:28:49.239 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.250 7fdf2ffff700 1 --2- 172.21.15.102:0/2817405211 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fdf301aa680 0x7fdf301b41f0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 rx=0x7fdf2000c000 tx=0x7fdf2000ed40).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:28:49.239 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.250 7fdf2cff9700 1 -- 172.21.15.102:0/2817405211 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fdf301b4820 con 0x7fdf301aa680 2024-04-17T22:28:49.239 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.250 7fdf2cff9700 1 -- 172.21.15.102:0/2817405211 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf2000c880 con 0x7fdf301aa680 2024-04-17T22:28:49.239 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.251 7fdf36a3e700 1 -- 172.21.15.102:0/2817405211 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf301bb2b0 con 0x7fdf301aa680 2024-04-17T22:28:49.239 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.251 7fdf2cff9700 1 -- 172.21.15.102:0/2817405211 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fdf2000f040 con 0x7fdf301aa680 2024-04-17T22:28:49.239 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.251 7fdf36a3e700 1 -- 172.21.15.102:0/2817405211 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf301bb690 con 0x7fdf301aa680 2024-04-17T22:28:49.239 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.251 7fdf2cff9700 1 -- 172.21.15.102:0/2817405211 <== mon.1 v2:172.21.15.102:3301/0 4 ==== mgrmap(e 1) v1 ==== 190+0+0 (secure 0 0 0) 0x7fdf20022060 con 0x7fdf301aa680 2024-04-17T22:28:49.239 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.251 7fdf2cff9700 1 -- 172.21.15.102:0/2817405211 <== mon.1 v2:172.21.15.102:3301/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 927+0+0 (secure 0 0 0) 0x7fdf20010040 con 0x7fdf301aa680 2024-04-17T22:28:49.239 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.251 7fdf36a3e700 1 -- 172.21.15.102:0/2817405211 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf300f09f0 con 0x7fdf301aa680 2024-04-17T22:28:49.240 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.253 7fdf2cff9700 1 -- 172.21.15.102:0/2817405211 <== mon.1 v2:172.21.15.102:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+66062 (secure 0 0 0) 0x7fdf2000c670 con 0x7fdf301aa680 2024-04-17T22:28:49.311 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.326 7fdf36a3e700 1 -- 172.21.15.102:0/2817405211 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "osd new", "uuid": "d524204b-70b2-40a8-8f73-f9e7d44318af", "id": 2} v 0) v1 -- 0x7fdf301b4ae0 con 0x7fdf301aa680 2024-04-17T22:28:49.314 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.329 7fdf2cff9700 1 -- 172.21.15.102:0/2817405211 <== mon.1 v2:172.21.15.102:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "d524204b-70b2-40a8-8f73-f9e7d44318af", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7fdf30113d90 con 0x7fdf301aa680 2024-04-17T22:28:49.315 INFO:teuthology.orchestra.run.smithi102.stdout:2 2024-04-17T22:28:49.316 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.330 7fdf36a3e700 1 -- 172.21.15.102:0/2817405211 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fdf301aa680 msgr2=0x7fdf301b41f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:28:49.316 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.330 7fdf36a3e700 1 --2- 172.21.15.102:0/2817405211 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fdf301aa680 0x7fdf301b41f0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 rx=0x7fdf2000c000 tx=0x7fdf2000ed40).stop 2024-04-17T22:28:49.316 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.330 7fdf36a3e700 1 -- 172.21.15.102:0/2817405211 shutdown_connections 2024-04-17T22:28:49.316 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.330 7fdf36a3e700 1 --2- 172.21.15.102:0/2817405211 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fdf301aa680 0x7fdf301b41f0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:28:49.316 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.330 7fdf36a3e700 1 --2- 172.21.15.102:0/2817405211 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdf301a1de0 0x7fdf301a9c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:28:49.316 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.330 7fdf36a3e700 1 --2- 172.21.15.102:0/2817405211 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fdf301aa230 0x7fdf301b3bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:28:49.316 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.330 7fdf36a3e700 1 -- 172.21.15.102:0/2817405211 >> 172.21.15.102:0/2817405211 conn(0x7fdf301a4090 msgr2=0x7fdf300dffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:28:49.316 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.330 7fdf36a3e700 1 -- 172.21.15.102:0/2817405211 shutdown_connections 2024-04-17T22:28:49.316 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.330 7fdf36a3e700 1 -- 172.21.15.102:0/2817405211 wait complete. 2024-04-17T22:28:49.326 DEBUG:teuthology.orchestra.run.smithi102:> sudo ceph --cluster ceph osd new 4fc9d20c-1ad0-484f-aafa-2c9a7574245b 3 2024-04-17T22:28:49.453 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.465 7fd8c0be0700 1 Processor -- start 2024-04-17T22:28:49.453 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.465 7fd8c0be0700 1 -- start start 2024-04-17T22:28:49.453 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.466 7fd8c0be0700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd8bc0f5840 0x7fd8bc10b010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:28:49.453 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.466 7fd8c0be0700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd8bc10b640 0x7fd8bc19faf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:28:49.453 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.466 7fd8c0be0700 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8bc103b40 con 0x7fd8bc10ba90 2024-04-17T22:28:49.454 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.466 7fd8c0be0700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8bc10bee0 con 0x7fd8bc0f5840 2024-04-17T22:28:49.454 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.466 7fd8c0be0700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd8bc0fa370 con 0x7fd8bc10b640 2024-04-17T22:28:49.454 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.466 7fd8b9850700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd8bc0f5840 0x7fd8bc10b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:28:49.454 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.466 7fd8b9850700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd8bc0f5840 0x7fd8bc10b010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:59980/0 (socket says 172.21.15.102:59980) 2024-04-17T22:28:49.454 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.466 7fd8b9850700 1 -- 172.21.15.102:0/350421233 learned_addr learned my addr 172.21.15.102:0/350421233 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:28:49.454 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.466 7fd8b904f700 1 --2- 172.21.15.102:0/350421233 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd8bc10b640 0x7fd8bc19faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:28:49.454 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.466 7fd8b9850700 1 -- 172.21.15.102:0/350421233 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd8bc10b640 msgr2=0x7fd8bc19faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:28:49.454 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.466 7fd8b9850700 1 --2- 172.21.15.102:0/350421233 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd8bc10b640 0x7fd8bc19faf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:28:49.454 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.466 7fd8b884e700 1 -- 172.21.15.102:0/350421233 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 115639993 0 0) 0x7fd8ac001490 con 0x7fd8bc10ba90 2024-04-17T22:28:49.454 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.466 7fd8b9850700 1 -- 172.21.15.102:0/350421233 >> v1:172.21.15.102:6789/0 conn(0x7fd8bc10ba90 legacy=0x7fd8bc1a0120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:28:49.454 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.466 7fd8b9850700 1 -- 172.21.15.102:0/350421233 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8bc1a0920 con 0x7fd8bc0f5840 2024-04-17T22:28:49.454 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.466 7fd8b9850700 1 --2- 172.21.15.102:0/350421233 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd8bc0f5840 0x7fd8bc10b010 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 rx=0x7fd8bc103d60 tx=0x7fd8b4009ab0).ready entity=mon.1 client_cookie=cda7f0e062d32e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:28:49.454 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.467 7fd8b884e700 1 -- 172.21.15.102:0/350421233 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd8bc1a0920 con 0x7fd8bc0f5840 2024-04-17T22:28:49.454 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.467 7fd8b884e700 1 -- 172.21.15.102:0/350421233 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8b400f510 con 0x7fd8bc0f5840 2024-04-17T22:28:49.455 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.467 7fd8b884e700 1 -- 172.21.15.102:0/350421233 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd8b400f6f0 con 0x7fd8bc0f5840 2024-04-17T22:28:49.455 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.467 7fd8c0be0700 1 -- 172.21.15.102:0/350421233 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd8bc0f5840 msgr2=0x7fd8bc10b010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:28:49.455 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.467 7fd8c0be0700 1 --2- 172.21.15.102:0/350421233 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd8bc0f5840 0x7fd8bc10b010 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 rx=0x7fd8bc103d60 tx=0x7fd8b4009ab0).stop 2024-04-17T22:28:49.455 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.467 7fd8c0be0700 1 -- 172.21.15.102:0/350421233 shutdown_connections 2024-04-17T22:28:49.455 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.467 7fd8c0be0700 1 --2- 172.21.15.102:0/350421233 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd8bc0f5840 0x7fd8bc10b010 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:28:49.455 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.467 7fd8c0be0700 1 --2- 172.21.15.102:0/350421233 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd8bc10b640 0x7fd8bc19faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:28:49.455 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.467 7fd8c0be0700 1 -- 172.21.15.102:0/350421233 >> 172.21.15.102:0/350421233 conn(0x7fd8bc1138a0 msgr2=0x7fd8bc0f4f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:28:49.455 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.467 7fd8c0be0700 1 -- 172.21.15.102:0/350421233 shutdown_connections 2024-04-17T22:28:49.455 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.467 7fd8c0be0700 1 -- 172.21.15.102:0/350421233 wait complete. 2024-04-17T22:28:49.478 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.491 7fd8c0be0700 1 Processor -- start 2024-04-17T22:28:49.478 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.491 7fd8c0be0700 1 -- start start 2024-04-17T22:28:49.479 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.491 7fd8c0be0700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd8bc1b1370 0x7fd8bc1b17c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:28:49.479 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.491 7fd8c0be0700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd8bc1b1df0 0x7fd8bc1b4700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:28:49.479 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.491 7fd8c0be0700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd8bc1b4d30 0x7fd8bc1b5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:28:49.479 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.491 7fd8c0be0700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8bc088fc0 con 0x7fd8bc1b4d30 2024-04-17T22:28:49.479 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.491 7fd8c0be0700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8bc0c2330 con 0x7fd8bc1b1df0 2024-04-17T22:28:49.479 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.491 7fd8c0be0700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd8bc1b57b0 con 0x7fd8bc1b1370 2024-04-17T22:28:49.479 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.491 7fd8b904f700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd8bc1b1df0 0x7fd8bc1b4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:28:49.479 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.491 7fd8b9850700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd8bc1b1370 0x7fd8bc1b17c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:28:49.479 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.491 7fd8b904f700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd8bc1b1df0 0x7fd8bc1b4700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:59996/0 (socket says 172.21.15.102:59996) 2024-04-17T22:28:49.479 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.491 7fd8b904f700 1 -- 172.21.15.102:0/4003978775 learned_addr learned my addr 172.21.15.102:0/4003978775 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:28:49.479 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.491 7fd8ba051700 1 --2- 172.21.15.102:0/4003978775 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd8bc1b4d30 0x7fd8bc1b5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:28:49.479 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.492 7fd8b9850700 1 -- 172.21.15.102:0/4003978775 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd8bc1b1df0 msgr2=0x7fd8bc1b4700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:28:49.479 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.492 7fd8b9850700 1 --2- 172.21.15.102:0/4003978775 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd8bc1b1df0 0x7fd8bc1b4700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:28:49.479 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.492 7fd8b9850700 1 -- 172.21.15.102:0/4003978775 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd8bc1b4d30 msgr2=0x7fd8bc1b5180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:28:49.480 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.492 7fd8b9850700 1 --2- 172.21.15.102:0/4003978775 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd8bc1b4d30 0x7fd8bc1b5180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:28:49.480 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.492 7fd8b9850700 1 -- 172.21.15.102:0/4003978775 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8bc0d5e00 con 0x7fd8bc1b1370 2024-04-17T22:28:49.480 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.492 7fd8b904f700 1 --2- 172.21.15.102:0/4003978775 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd8bc1b1df0 0x7fd8bc1b4700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:28:49.480 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.492 7fd8b9850700 1 --2- 172.21.15.102:0/4003978775 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd8bc1b1370 0x7fd8bc1b17c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0x7fd8bc0b95e0 tx=0x7fd8b40068c0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:28:49.480 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.492 7fd8aaffd700 1 -- 172.21.15.102:0/4003978775 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd8bc0d5e00 con 0x7fd8bc1b1370 2024-04-17T22:28:49.480 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.492 7fd8aaffd700 1 -- 172.21.15.102:0/4003978775 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8b40119c0 con 0x7fd8bc1b1370 2024-04-17T22:28:49.480 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.492 7fd8c0be0700 1 -- 172.21.15.102:0/4003978775 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8bc0d6f90 con 0x7fd8bc1b1370 2024-04-17T22:28:49.480 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.492 7fd8aaffd700 1 -- 172.21.15.102:0/4003978775 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd8b4011d90 con 0x7fd8bc1b1370 2024-04-17T22:28:49.480 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.492 7fd8c0be0700 1 -- 172.21.15.102:0/4003978775 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8bc1bbfd0 con 0x7fd8bc1b1370 2024-04-17T22:28:49.480 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.492 7fd8aaffd700 1 -- 172.21.15.102:0/4003978775 <== mon.2 v2:172.21.15.102:3302/0 4 ==== mgrmap(e 1) v1 ==== 190+0+0 (secure 0 0 0) 0x7fd8b4027460 con 0x7fd8bc1b1370 2024-04-17T22:28:49.480 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.493 7fd8aaffd700 1 -- 172.21.15.102:0/4003978775 <== mon.2 v2:172.21.15.102:3302/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1030+0+0 (secure 0 0 0) 0x7fd8b4006b90 con 0x7fd8bc1b1370 2024-04-17T22:28:49.481 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.493 7fd8c0be0700 1 -- 172.21.15.102:0/4003978775 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8bc0f0800 con 0x7fd8bc1b1370 2024-04-17T22:28:49.481 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.495 7fd8aaffd700 1 -- 172.21.15.102:0/4003978775 <== mon.2 v2:172.21.15.102:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+66062 (secure 0 0 0) 0x7fd8b40117b0 con 0x7fd8bc1b1370 2024-04-17T22:28:49.552 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.567 7fd8c0be0700 1 -- 172.21.15.102:0/4003978775 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "osd new", "uuid": "4fc9d20c-1ad0-484f-aafa-2c9a7574245b", "id": 3} v 0) v1 -- 0x7fd8bc0d7480 con 0x7fd8bc1b1370 2024-04-17T22:28:49.555 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.570 7fd8aaffd700 1 -- 172.21.15.102:0/4003978775 <== mon.2 v2:172.21.15.102:3302/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "4fc9d20c-1ad0-484f-aafa-2c9a7574245b", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7fd8b400f300 con 0x7fd8bc1b1370 2024-04-17T22:28:49.555 INFO:teuthology.orchestra.run.smithi102.stdout:3 2024-04-17T22:28:49.557 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.571 7fd8c0be0700 1 -- 172.21.15.102:0/4003978775 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd8bc1b1370 msgr2=0x7fd8bc1b17c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:28:49.557 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.571 7fd8c0be0700 1 --2- 172.21.15.102:0/4003978775 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd8bc1b1370 0x7fd8bc1b17c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0x7fd8bc0b95e0 tx=0x7fd8b40068c0).stop 2024-04-17T22:28:49.557 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.571 7fd8c0be0700 1 -- 172.21.15.102:0/4003978775 shutdown_connections 2024-04-17T22:28:49.557 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.571 7fd8c0be0700 1 --2- 172.21.15.102:0/4003978775 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd8bc1b4d30 0x7fd8bc1b5180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:28:49.557 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.571 7fd8c0be0700 1 --2- 172.21.15.102:0/4003978775 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd8bc1b1370 0x7fd8bc1b17c0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:28:49.557 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.571 7fd8c0be0700 1 --2- 172.21.15.102:0/4003978775 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd8bc1b1df0 0x7fd8bc1b4700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:28:49.558 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.571 7fd8c0be0700 1 -- 172.21.15.102:0/4003978775 >> 172.21.15.102:0/4003978775 conn(0x7fd8bc0e0f60 msgr2=0x7fd8bc0db000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:28:49.558 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.571 7fd8c0be0700 1 -- 172.21.15.102:0/4003978775 shutdown_connections 2024-04-17T22:28:49.558 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.571 7fd8c0be0700 1 -- 172.21.15.102:0/4003978775 wait complete. 2024-04-17T22:28:49.571 INFO:tasks.ceph.osd.0:Restarting daemon 2024-04-17T22:28:49.571 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-04-17T22:28:49.573 INFO:tasks.ceph.osd.0:Started 2024-04-17T22:28:49.573 INFO:tasks.ceph.osd.1:Restarting daemon 2024-04-17T22:28:49.574 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-04-17T22:28:49.575 INFO:tasks.ceph.osd.1:Started 2024-04-17T22:28:49.575 INFO:tasks.ceph.osd.2:Restarting daemon 2024-04-17T22:28:49.575 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-04-17T22:28:49.577 INFO:tasks.ceph.osd.2:Started 2024-04-17T22:28:49.577 INFO:tasks.ceph.osd.3:Restarting daemon 2024-04-17T22:28:49.578 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-04-17T22:28:49.579 INFO:tasks.ceph.osd.3:Started 2024-04-17T22:28:49.580 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T22:28:49.657 INFO:tasks.ceph.osd.3.smithi102.stderr:warning: line 130: 'mon_down_mkfs_grace' in section 'mon' redefined 2024-04-17T22:28:49.657 INFO:tasks.ceph.osd.2.smithi102.stderr:warning: line 130: 'mon_down_mkfs_grace' in section 'mon' redefined 2024-04-17T22:28:49.668 INFO:tasks.ceph.osd.0.smithi102.stderr:warning: line 130: 'mon_down_mkfs_grace' in section 'mon' redefined 2024-04-17T22:28:49.675 INFO:tasks.ceph.osd.1.smithi102.stderr:warning: line 130: 'mon_down_mkfs_grace' in section 'mon' redefined 2024-04-17T22:28:49.756 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.770 7effbb6e6700 1 Processor -- start 2024-04-17T22:28:49.756 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.770 7effbb6e6700 1 -- start start 2024-04-17T22:28:49.757 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.770 7effbb6e6700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7effb410b830 0x7effb41a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:28:49.757 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.770 7effbb6e6700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7effb410bc80 0x7effb41a7920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:28:49.757 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.770 7effbb6e6700 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7effb4103b60 con 0x7effb40f5860 2024-04-17T22:28:49.757 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.770 7effbb6e6700 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effb40fa390 con 0x7effb410b830 2024-04-17T22:28:49.757 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.770 7effbb6e6700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effb41a0b20 con 0x7effb410bc80 2024-04-17T22:28:49.757 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.771 7effb3fff700 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7effb40f5860 0x7effb410b030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:41944/0 (socket says 172.21.15.102:41944) 2024-04-17T22:28:49.757 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.771 7effb3fff700 1 -- 172.21.15.102:0/2047662100 learned_addr learned my addr 172.21.15.102:0/2047662100 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-17T22:28:49.757 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.771 7effb37fe700 1 --2- 172.21.15.102:0/2047662100 >> v2:172.21.15.102:3300/0 conn(0x7effb410b830 0x7effb41a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:28:49.757 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.771 7effb8ce2700 1 --2- 172.21.15.102:0/2047662100 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7effb410bc80 0x7effb41a7920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:28:49.758 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.771 7effb2ffd700 1 -- 172.21.15.102:0/2047662100 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4059581301 0 0) 0x7effa4001110 con 0x7effb40f5860 2024-04-17T22:28:49.758 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.771 7effb2ffd700 1 -- 172.21.15.102:0/2047662100 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7effa0007680 con 0x7effb40f5860 2024-04-17T22:28:49.758 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.771 7effb37fe700 1 -- 172.21.15.102:0/2047662100 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7effb410bc80 msgr2=0x7effb41a7920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:28:49.758 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.771 7effb37fe700 1 --2- 172.21.15.102:0/2047662100 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7effb410bc80 0x7effb41a7920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:28:49.758 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.771 7effb37fe700 1 -- 172.21.15.102:0/2047662100 >> v1:172.21.15.102:6789/0 conn(0x7effb40f5860 legacy=0x7effb410b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:28:49.758 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.771 7effb37fe700 1 -- 172.21.15.102:0/2047662100 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effb41a8f60 con 0x7effb410b830 2024-04-17T22:28:49.758 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.771 7effb37fe700 1 --2- 172.21.15.102:0/2047662100 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7effb410b830 0x7effb41a52e0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7effa8002f70 tx=0x7effa800dc00).ready entity=mon.0 client_cookie=24abbda3fbb06652 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:28:49.759 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.771 7effb2ffd700 1 -- 172.21.15.102:0/2047662100 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7effb41a8f60 con 0x7effb410b830 2024-04-17T22:28:49.759 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.771 7effb2ffd700 1 -- 172.21.15.102:0/2047662100 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effa8009570 con 0x7effb410b830 2024-04-17T22:28:49.759 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.771 7effb2ffd700 1 -- 172.21.15.102:0/2047662100 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7effa801b070 con 0x7effb410b830 2024-04-17T22:28:49.759 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.771 7effbb6e6700 1 -- 172.21.15.102:0/2047662100 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7effb410b830 msgr2=0x7effb41a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:28:49.759 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.771 7effbb6e6700 1 --2- 172.21.15.102:0/2047662100 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7effb410b830 0x7effb41a52e0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7effa8002f70 tx=0x7effa800dc00).stop 2024-04-17T22:28:49.759 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.772 7effbb6e6700 1 -- 172.21.15.102:0/2047662100 shutdown_connections 2024-04-17T22:28:49.759 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.772 7effbb6e6700 1 --2- 172.21.15.102:0/2047662100 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7effb410bc80 0x7effb41a7920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:28:49.759 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.772 7effbb6e6700 1 --2- 172.21.15.102:0/2047662100 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7effb410b830 0x7effb41a52e0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:28:49.759 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.772 7effbb6e6700 1 -- 172.21.15.102:0/2047662100 >> 172.21.15.102:0/2047662100 conn(0x7effb41138c0 msgr2=0x7effb40f4f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:28:49.759 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.772 7effbb6e6700 1 -- 172.21.15.102:0/2047662100 shutdown_connections 2024-04-17T22:28:49.759 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.772 7effbb6e6700 1 -- 172.21.15.102:0/2047662100 wait complete. 2024-04-17T22:28:49.782 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.795 7effbb6e6700 1 Processor -- start 2024-04-17T22:28:49.782 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.795 7effbb6e6700 1 -- start start 2024-04-17T22:28:49.783 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.796 7effbb6e6700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7effb40d2f10 0x7effb40d3360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:28:49.783 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.796 7effbb6e6700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7effb40d3990 0x7effb41afd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:28:49.783 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.796 7effbb6e6700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7effb41b0360 0x7effb41b07b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:28:49.783 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.796 7effbb6e6700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effb41b0de0 con 0x7effb40d3990 2024-04-17T22:28:49.783 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.796 7effbb6e6700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effb40cd700 con 0x7effb41b0360 2024-04-17T22:28:49.783 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.796 7effbb6e6700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7effb40dc6d0 con 0x7effb40d2f10 2024-04-17T22:28:49.783 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.796 7effb37fe700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7effb40d3990 0x7effb41afd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:28:49.783 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.796 7effb37fe700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7effb40d3990 0x7effb41afd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:39884/0 (socket says 172.21.15.102:39884) 2024-04-17T22:28:49.783 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.796 7effb37fe700 1 -- 172.21.15.102:0/2756137248 learned_addr learned my addr 172.21.15.102:0/2756137248 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:28:49.783 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.796 7effb3fff700 1 --2- 172.21.15.102:0/2756137248 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7effb40d2f10 0x7effb40d3360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:28:49.783 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.796 7effb8ce2700 1 --2- 172.21.15.102:0/2756137248 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7effb41b0360 0x7effb41b07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:28:49.783 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.796 7effb37fe700 1 -- 172.21.15.102:0/2756137248 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7effb40d2f10 msgr2=0x7effb40d3360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:28:49.783 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.796 7effb37fe700 1 --2- 172.21.15.102:0/2756137248 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7effb40d2f10 0x7effb40d3360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:28:49.784 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.796 7effb37fe700 1 -- 172.21.15.102:0/2756137248 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7effb41b0360 msgr2=0x7effb41b07b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:28:49.784 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.796 7effb37fe700 1 --2- 172.21.15.102:0/2756137248 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7effb41b0360 0x7effb41b07b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:28:49.784 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.796 7effb37fe700 1 -- 172.21.15.102:0/2756137248 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effb40dc260 con 0x7effb40d3990 2024-04-17T22:28:49.784 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.796 7effb8ce2700 1 --2- 172.21.15.102:0/2756137248 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7effb41b0360 0x7effb41b07b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:28:49.784 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.797 7effb37fe700 1 --2- 172.21.15.102:0/2756137248 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7effb40d3990 0x7effb41afd30 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7effb40d65f0 tx=0x7effa80068c0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:28:49.784 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.797 7effb17fa700 1 -- 172.21.15.102:0/2756137248 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7effb40dc260 con 0x7effb40d3990 2024-04-17T22:28:49.784 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.797 7effb17fa700 1 -- 172.21.15.102:0/2756137248 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effa8020670 con 0x7effb40d3990 2024-04-17T22:28:49.784 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.797 7effb17fa700 1 -- 172.21.15.102:0/2756137248 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7effa8027030 con 0x7effb40d3990 2024-04-17T22:28:49.784 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.797 7effbb6e6700 1 -- 172.21.15.102:0/2756137248 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effb40d9d70 con 0x7effb40d3990 2024-04-17T22:28:49.785 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.797 7effbb6e6700 1 -- 172.21.15.102:0/2756137248 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effb41b7660 con 0x7effb40d3990 2024-04-17T22:28:49.785 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.797 7effb17fa700 1 -- 172.21.15.102:0/2756137248 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 1) v1 ==== 190+0+0 (secure 0 0 0) 0x7effa800d030 con 0x7effb40d3990 2024-04-17T22:28:49.785 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.797 7effb17fa700 1 -- 172.21.15.102:0/2756137248 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1133+0+0 (secure 0 0 0) 0x7effa8020d00 con 0x7effb40d3990 2024-04-17T22:28:49.785 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.798 7effbb6e6700 1 -- 172.21.15.102:0/2756137248 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7effb40f0820 con 0x7effb40d3990 2024-04-17T22:28:49.785 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.800 7effb17fa700 1 -- 172.21.15.102:0/2756137248 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+66062 (secure 0 0 0) 0x7effa8009d60 con 0x7effb40d3990 2024-04-17T22:28:49.858 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.872 7effbb6e6700 1 -- 172.21.15.102:0/2756137248 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7effb41aa4f0 con 0x7effb40d3990 2024-04-17T22:28:49.858 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.873 7effb17fa700 1 -- 172.21.15.102:0/2756137248 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v6) v1 ==== 74+0+3216 (secure 0 0 0) 0x7effa8016b70 con 0x7effb40d3990 2024-04-17T22:28:49.859 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-17T22:28:49.859 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":6,"fsid":"cb63e789-091d-49b1-b120-55f9db526871","created":"2024-04-17 22:28:48.209635","modified":"2024-04-17 22:28:49.568219","last_up_change":"0.000000","last_in_change":"0.000000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":4,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[],"osds":[{"osd":0,"uuid":"3351ef85-ade4-4b90-a8f4-213f4e00ba01","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":":/0","cluster_addr":":/0","heartbeat_back_addr":":/0","heartbeat_front_addr":":/0","state":["exists","new"]},{"osd":1,"uuid":"04f67eac-f36b-48f5-982d-4cfc776596d0","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":":/0","cluster_addr":":/0","heartbeat_back_addr":":/0","heartbeat_front_addr":":/0","state":["exists","new"]},{"osd":2,"uuid":"d524204b-70b2-40a8-8f73-f9e7d44318af","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":":/0","cluster_addr":":/0","heartbeat_back_addr":":/0","heartbeat_front_addr":":/0","state":["exists","new"]},{"osd":3,"uuid":"4fc9d20c-1ad0-484f-aafa-2c9a7574245b","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":":/0","cluster_addr":":/0","heartbeat_back_addr":":/0","heartbeat_front_addr":":/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2024-04-17T22:28:49.860 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.873 7effbb6e6700 1 -- 172.21.15.102:0/2756137248 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7effb40d3990 msgr2=0x7effb41afd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:28:49.860 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.873 7effbb6e6700 1 --2- 172.21.15.102:0/2756137248 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7effb40d3990 0x7effb41afd30 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7effb40d65f0 tx=0x7effa80068c0).stop 2024-04-17T22:28:49.860 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.874 7effbb6e6700 1 -- 172.21.15.102:0/2756137248 shutdown_connections 2024-04-17T22:28:49.860 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.874 7effbb6e6700 1 --2- 172.21.15.102:0/2756137248 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7effb41b0360 0x7effb41b07b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:28:49.860 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.874 7effbb6e6700 1 --2- 172.21.15.102:0/2756137248 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7effb40d2f10 0x7effb40d3360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:28:49.860 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.874 7effbb6e6700 1 --2- 172.21.15.102:0/2756137248 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7effb40d3990 0x7effb41afd30 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:28:49.860 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.874 7effbb6e6700 1 -- 172.21.15.102:0/2756137248 >> 172.21.15.102:0/2756137248 conn(0x7effb40dffb0 msgr2=0x7effb40c4b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:28:49.860 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.874 7effbb6e6700 1 -- 172.21.15.102:0/2756137248 shutdown_connections 2024-04-17T22:28:49.860 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:49.874 7effbb6e6700 1 -- 172.21.15.102:0/2756137248 wait complete. 2024-04-17T22:28:49.870 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-04-17T22:28:49.870 INFO:tasks.ceph:Waiting for OSDs to come up 2024-04-17T22:28:50.229 INFO:tasks.ceph.osd.3.smithi102.stderr:2024-04-17 22:28:50.243 7fdfcdd33dc0 -1 Falling back to public interface 2024-04-17T22:28:50.259 INFO:tasks.ceph.osd.2.smithi102.stderr:2024-04-17 22:28:50.273 7f2e03c6ddc0 -1 Falling back to public interface 2024-04-17T22:28:50.267 INFO:tasks.ceph.osd.1.smithi102.stderr:2024-04-17 22:28:50.281 7f2f530b4dc0 -1 Falling back to public interface 2024-04-17T22:28:50.271 INFO:tasks.ceph.osd.0.smithi102.stderr:2024-04-17 22:28:50.285 7efcf55bbdc0 -1 Falling back to public interface 2024-04-17T22:28:50.273 DEBUG:teuthology.orchestra.run.smithi102:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-17T22:28:50.395 INFO:teuthology.misc.health.smithi102.stderr:2024-04-17 22:28:50.409 7fd5b9dab700 1 Processor -- start 2024-04-17T22:28:50.396 INFO:teuthology.misc.health.smithi102.stderr:2024-04-17 22:28:50.409 7fd5b9dab700 1 -- start start 2024-04-17T22:28:50.396 INFO:teuthology.misc.health.smithi102.stderr:2024-04-17 22:28:50.410 7fd5b9dab700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd5b41a6440 0x7fd5b41a88a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:28:50.396 INFO:teuthology.misc.health.smithi102.stderr:2024-04-17 22:28:50.410 7fd5b9dab700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fd5b41a8ed0 0x7fd5b41ab330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:28:50.396 INFO:teuthology.misc.health.smithi102.stderr:2024-04-17 22:28:50.410 7fd5b9dab700 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5b410ee20 con 0x7fd5b410e1d0 2024-04-17T22:28:50.396 INFO:teuthology.misc.health.smithi102.stderr:2024-04-17 22:28:50.410 7fd5b9dab700 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5b40f9ea0 con 0x7fd5b41a8ed0 2024-04-17T22:28:50.396 INFO:teuthology.misc.health.smithi102.stderr:2024-04-17 22:28:50.410 7fd5b9dab700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5b40f8fc0 con 0x7fd5b41a6440 2024-04-17T22:28:50.396 INFO:teuthology.misc.health.smithi102.stderr:2024-04-17 22:28:50.410 7fd5b32b3700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fd5b41a8ed0 0x7fd5b41ab330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:28:50.396 INFO:teuthology.misc.health.smithi102.stderr:2024-04-17 22:28:50.410 7fd5b2ab2700 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7fd5b410e1d0 0x7fd5b410e620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:41952/0 (socket says 172.21.15.102:41952) 2024-04-17T22:28:50.396 INFO:teuthology.misc.health.smithi102.stderr:2024-04-17 22:28:50.410 7fd5b2ab2700 1 -- 172.21.15.102:0/924213277 learned_addr learned my addr 172.21.15.102:0/924213277 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-17T22:28:50.396 INFO:teuthology.misc.health.smithi102.stderr:2024-04-17 22:28:50.410 7fd5b22b1700 1 --2- 172.21.15.102:0/924213277 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd5b41a6440 0x7fd5b41a88a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:28:50.398 INFO:teuthology.misc.health.smithi102.stderr:2024-04-17 22:28:50.410 7fd5b22b1700 1 -- 172.21.15.102:0/924213277 >> v2:172.21.15.102:3300/0 conn(0x7fd5b41a8ed0 msgr2=0x7fd5b41ab330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:28:50.399 INFO:teuthology.misc.health.smithi102.stderr:2024-04-17 22:28:50.410 7fd5b22b1700 1 --2- 172.21.15.102:0/924213277 >> v2:172.21.15.102:3300/0 conn(0x7fd5b41a8ed0 0x7fd5b41ab330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:28:50.399 INFO:teuthology.misc.health.smithi102.stderr:2024-04-17 22:28:50.410 7fd5b22b1700 1 -- 172.21.15.102:0/924213277 >> v1:172.21.15.102:6789/0 conn(0x7fd5b410e1d0 legacy=0x7fd5b410e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:28:50.399 INFO:teuthology.misc.health.smithi102.stderr:2024-04-17 22:28:50.410 7fd5b22b1700 1 -- 172.21.15.102:0/924213277 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5b41ac970 con 0x7fd5b41a6440 2024-04-17T22:28:50.399 INFO:teuthology.misc.health.smithi102.stderr:2024-04-17 22:28:50.410 7fd5b32b3700 1 --2- 172.21.15.102:0/924213277 >> v2:172.21.15.102:3300/0 conn(0x7fd5b41a8ed0 0x7fd5b41ab330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:28:50.399 INFO:teuthology.misc.health.smithi102.stderr:2024-04-17 22:28:50.410 7fd5b22b1700 1 --2- 172.21.15.102:0/924213277 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd5b41a6440 0x7fd5b41a88a0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 rx=0x7fd59c002f70 tx=0x7fd59c00b970).ready entity=mon.1 client_cookie=4bf301fcce441cfd server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:28:50.399 INFO:teuthology.misc.health.smithi102.stderr:2024-04-17 22:28:50.411 7fd5b1ab0700 1 -- 172.21.15.102:0/924213277 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd5b41ac970 con 0x7fd5b41a6440 2024-04-17T22:28:50.399 INFO:teuthology.misc.health.smithi102.stderr:2024-04-17 22:28:50.411 7fd5b1ab0700 1 -- 172.21.15.102:0/924213277 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd59c00d610 con 0x7fd5b41a6440 2024-04-17T22:28:50.399 INFO:teuthology.misc.health.smithi102.stderr:2024-04-17 22:28:50.411 7fd5b1ab0700 1 -- 172.21.15.102:0/924213277 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd59c00d9e0 con 0x7fd5b41a6440 2024-04-17T22:28:50.399 INFO:teuthology.misc.health.smithi102.stderr:2024-04-17 22:28:50.411 7fd5b9dab700 1 -- 172.21.15.102:0/924213277 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd5b41a6440 msgr2=0x7fd5b41a88a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:28:50.399 INFO:teuthology.misc.health.smithi102.stderr:2024-04-17 22:28:50.411 7fd5b9dab700 1 --2- 172.21.15.102:0/924213277 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd5b41a6440 0x7fd5b41a88a0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 rx=0x7fd59c002f70 tx=0x7fd59c00b970).stop 2024-04-17T22:28:50.399 INFO:teuthology.misc.health.smithi102.stderr:2024-04-17 22:28:50.411 7fd5b9dab700 1 -- 172.21.15.102:0/924213277 shutdown_connections 2024-04-17T22:28:50.399 INFO:teuthology.misc.health.smithi102.stderr:2024-04-17 22:28:50.411 7fd5b9dab700 1 --2- 172.21.15.102:0/924213277 >> v2:172.21.15.102:3300/0 conn(0x7fd5b41a8ed0 0x7fd5b41ab330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:28:50.400 INFO:teuthology.misc.health.smithi102.stderr:2024-04-17 22:28:50.411 7fd5b9dab700 1 --2- 172.21.15.102:0/924213277 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd5b41a6440 0x7fd5b41a88a0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:28:50.400 INFO:teuthology.misc.health.smithi102.stderr:2024-04-17 22:28:50.411 7fd5b9dab700 1 -- 172.21.15.102:0/924213277 >> 172.21.15.102:0/924213277 conn(0x7fd5b4113830 msgr2=0x7fd5b40f4ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:28:50.400 INFO:teuthology.misc.health.smithi102.stderr:2024-04-17 22:28:50.411 7fd5b9dab700 1 -- 172.21.15.102:0/924213277 shutdown_connections 2024-04-17T22:28:50.400 INFO:teuthology.misc.health.smithi102.stderr:2024-04-17 22:28:50.411 7fd5b9dab700 1 -- 172.21.15.102:0/924213277 wait complete. 2024-04-17T22:28:50.423 INFO:teuthology.misc.health.smithi102.stderr:2024-04-17 22:28:50.435 7fd5b9dab700 1 Processor -- start 2024-04-17T22:28:50.423 INFO:teuthology.misc.health.smithi102.stderr:2024-04-17 22:28:50.435 7fd5b9dab700 1 -- start start 2024-04-17T22:28:50.423 INFO:teuthology.misc.health.smithi102.stderr:2024-04-17 22:28:50.435 7fd5b9dab700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd5b40d5610 0x7fd5b40d5a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:28:50.423 INFO:teuthology.misc.health.smithi102.stderr:2024-04-17 22:28:50.435 7fd5b9dab700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd5b41b41b0 0x7fd5b41b4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:28:50.423 INFO:teuthology.misc.health.smithi102.stderr:2024-04-17 22:28:50.435 7fd5b9dab700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd5b41b4c30 0x7fd5b41bb690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:28:50.423 INFO:teuthology.misc.health.smithi102.stderr:2024-04-17 22:28:50.435 7fd5b9dab700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5b4113a40 con 0x7fd5b41b4c30 2024-04-17T22:28:50.423 INFO:teuthology.misc.health.smithi102.stderr:2024-04-17 22:28:50.435 7fd5b9dab700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5b40fa140 con 0x7fd5b41b41b0 2024-04-17T22:28:50.423 INFO:teuthology.misc.health.smithi102.stderr:2024-04-17 22:28:50.435 7fd5b9dab700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd5b40e05d0 con 0x7fd5b40d5610 2024-04-17T22:28:50.423 INFO:teuthology.misc.health.smithi102.stderr:2024-04-17 22:28:50.435 7fd5b32b3700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd5b41b4c30 0x7fd5b41bb690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:28:50.423 INFO:teuthology.misc.health.smithi102.stderr:2024-04-17 22:28:50.435 7fd5b2ab2700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd5b40d5610 0x7fd5b40d5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:28:50.423 INFO:teuthology.misc.health.smithi102.stderr:2024-04-17 22:28:50.435 7fd5b32b3700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd5b41b4c30 0x7fd5b41bb690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:39892/0 (socket says 172.21.15.102:39892) 2024-04-17T22:28:50.424 INFO:teuthology.misc.health.smithi102.stderr:2024-04-17 22:28:50.435 7fd5b32b3700 1 -- 172.21.15.102:0/405675064 learned_addr learned my addr 172.21.15.102:0/405675064 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:28:50.424 INFO:teuthology.misc.health.smithi102.stderr:2024-04-17 22:28:50.435 7fd5b22b1700 1 --2- 172.21.15.102:0/405675064 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd5b41b41b0 0x7fd5b41b4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:28:50.424 INFO:teuthology.misc.health.smithi102.stderr:2024-04-17 22:28:50.436 7fd5b32b3700 1 -- 172.21.15.102:0/405675064 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd5b40d5610 msgr2=0x7fd5b40d5a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:28:50.424 INFO:teuthology.misc.health.smithi102.stderr:2024-04-17 22:28:50.436 7fd5b32b3700 1 --2- 172.21.15.102:0/405675064 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd5b40d5610 0x7fd5b40d5a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:28:50.424 INFO:teuthology.misc.health.smithi102.stderr:2024-04-17 22:28:50.436 7fd5b32b3700 1 -- 172.21.15.102:0/405675064 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd5b41b41b0 msgr2=0x7fd5b41b4600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:28:50.424 INFO:teuthology.misc.health.smithi102.stderr:2024-04-17 22:28:50.436 7fd5b32b3700 1 --2- 172.21.15.102:0/405675064 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd5b41b41b0 0x7fd5b41b4600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:28:50.424 INFO:teuthology.misc.health.smithi102.stderr:2024-04-17 22:28:50.436 7fd5b32b3700 1 -- 172.21.15.102:0/405675064 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5b40dfbf0 con 0x7fd5b41b4c30 2024-04-17T22:28:50.424 INFO:teuthology.misc.health.smithi102.stderr:2024-04-17 22:28:50.436 7fd5b2ab2700 1 --2- 172.21.15.102:0/405675064 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd5b40d5610 0x7fd5b40d5a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:28:50.424 INFO:teuthology.misc.health.smithi102.stderr:2024-04-17 22:28:50.436 7fd5b22b1700 1 --2- 172.21.15.102:0/405675064 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd5b41b41b0 0x7fd5b41b4600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:28:50.424 INFO:teuthology.misc.health.smithi102.stderr:2024-04-17 22:28:50.436 7fd5b32b3700 1 --2- 172.21.15.102:0/405675064 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd5b41b4c30 0x7fd5b41bb690 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7fd5b41a8300 tx=0x7fd5ac00ecd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:28:50.424 INFO:teuthology.misc.health.smithi102.stderr:2024-04-17 22:28:50.436 7fd5abfff700 1 -- 172.21.15.102:0/405675064 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd5b40dfbf0 con 0x7fd5b41b4c30 2024-04-17T22:28:50.424 INFO:teuthology.misc.health.smithi102.stderr:2024-04-17 22:28:50.436 7fd5abfff700 1 -- 172.21.15.102:0/405675064 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5ac00c790 con 0x7fd5b41b4c30 2024-04-17T22:28:50.424 INFO:teuthology.misc.health.smithi102.stderr:2024-04-17 22:28:50.436 7fd5b9dab700 1 -- 172.21.15.102:0/405675064 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5b41b5080 con 0x7fd5b41b4c30 2024-04-17T22:28:50.424 INFO:teuthology.misc.health.smithi102.stderr:2024-04-17 22:28:50.436 7fd5abfff700 1 -- 172.21.15.102:0/405675064 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd5ac00f040 con 0x7fd5b41b4c30 2024-04-17T22:28:50.424 INFO:teuthology.misc.health.smithi102.stderr:2024-04-17 22:28:50.436 7fd5b9dab700 1 -- 172.21.15.102:0/405675064 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5b41bbcc0 con 0x7fd5b41b4c30 2024-04-17T22:28:50.425 INFO:teuthology.misc.health.smithi102.stderr:2024-04-17 22:28:50.436 7fd5abfff700 1 -- 172.21.15.102:0/405675064 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 1) v1 ==== 190+0+0 (secure 0 0 0) 0x7fd5ac011070 con 0x7fd5b41b4c30 2024-04-17T22:28:50.425 INFO:teuthology.misc.health.smithi102.stderr:2024-04-17 22:28:50.436 7fd5abfff700 1 -- 172.21.15.102:0/405675064 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1133+0+0 (secure 0 0 0) 0x7fd5ac010040 con 0x7fd5b41b4c30 2024-04-17T22:28:50.425 INFO:teuthology.misc.health.smithi102.stderr:2024-04-17 22:28:50.437 7fd5b9dab700 1 -- 172.21.15.102:0/405675064 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5b40f0790 con 0x7fd5b41b4c30 2024-04-17T22:28:50.425 INFO:teuthology.misc.health.smithi102.stderr:2024-04-17 22:28:50.439 7fd5abfff700 1 -- 172.21.15.102:0/405675064 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+66062 (secure 0 0 0) 0x7fd5ac00ee10 con 0x7fd5b41b4c30 2024-04-17T22:28:50.484 INFO:teuthology.misc.health.smithi102.stderr:2024-04-17 22:28:50.498 7fd5b9dab700 1 -- 172.21.15.102:0/405675064 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd5b41ae8f0 con 0x7fd5b41b4c30 2024-04-17T22:28:50.484 INFO:teuthology.misc.health.smithi102.stderr:2024-04-17 22:28:50.499 7fd5abfff700 1 -- 172.21.15.102:0/405675064 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v6) v1 ==== 74+0+3216 (secure 0 0 0) 0x7fd5b41143b0 con 0x7fd5b41b4c30 2024-04-17T22:28:50.485 INFO:teuthology.misc.health.smithi102.stdout: 2024-04-17T22:28:50.485 INFO:teuthology.misc.health.smithi102.stdout:{"epoch":6,"fsid":"cb63e789-091d-49b1-b120-55f9db526871","created":"2024-04-17 22:28:48.209635","modified":"2024-04-17 22:28:49.568219","last_up_change":"0.000000","last_in_change":"0.000000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":4,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[],"osds":[{"osd":0,"uuid":"3351ef85-ade4-4b90-a8f4-213f4e00ba01","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":":/0","cluster_addr":":/0","heartbeat_back_addr":":/0","heartbeat_front_addr":":/0","state":["exists","new"]},{"osd":1,"uuid":"04f67eac-f36b-48f5-982d-4cfc776596d0","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":":/0","cluster_addr":":/0","heartbeat_back_addr":":/0","heartbeat_front_addr":":/0","state":["exists","new"]},{"osd":2,"uuid":"d524204b-70b2-40a8-8f73-f9e7d44318af","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":":/0","cluster_addr":":/0","heartbeat_back_addr":":/0","heartbeat_front_addr":":/0","state":["exists","new"]},{"osd":3,"uuid":"4fc9d20c-1ad0-484f-aafa-2c9a7574245b","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":":/0","cluster_addr":":/0","heartbeat_back_addr":":/0","heartbeat_front_addr":":/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2024-04-17T22:28:50.486 INFO:teuthology.misc.health.smithi102.stderr:2024-04-17 22:28:50.500 7fd5b9dab700 1 -- 172.21.15.102:0/405675064 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd5b41b4c30 msgr2=0x7fd5b41bb690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:28:50.486 INFO:teuthology.misc.health.smithi102.stderr:2024-04-17 22:28:50.500 7fd5b9dab700 1 --2- 172.21.15.102:0/405675064 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd5b41b4c30 0x7fd5b41bb690 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7fd5b41a8300 tx=0x7fd5ac00ecd0).stop 2024-04-17T22:28:50.487 INFO:teuthology.misc.health.smithi102.stderr:2024-04-17 22:28:50.500 7fd5b9dab700 1 -- 172.21.15.102:0/405675064 shutdown_connections 2024-04-17T22:28:50.487 INFO:teuthology.misc.health.smithi102.stderr:2024-04-17 22:28:50.500 7fd5b9dab700 1 --2- 172.21.15.102:0/405675064 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd5b41b4c30 0x7fd5b41bb690 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:28:50.487 INFO:teuthology.misc.health.smithi102.stderr:2024-04-17 22:28:50.500 7fd5b9dab700 1 --2- 172.21.15.102:0/405675064 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd5b40d5610 0x7fd5b40d5a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:28:50.487 INFO:teuthology.misc.health.smithi102.stderr:2024-04-17 22:28:50.500 7fd5b9dab700 1 --2- 172.21.15.102:0/405675064 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd5b41b41b0 0x7fd5b41b4600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:28:50.487 INFO:teuthology.misc.health.smithi102.stderr:2024-04-17 22:28:50.500 7fd5b9dab700 1 -- 172.21.15.102:0/405675064 >> 172.21.15.102:0/405675064 conn(0x7fd5b40c6af0 msgr2=0x7fd5b40d63f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:28:50.487 INFO:teuthology.misc.health.smithi102.stderr:2024-04-17 22:28:50.500 7fd5b9dab700 1 -- 172.21.15.102:0/405675064 shutdown_connections 2024-04-17T22:28:50.487 INFO:teuthology.misc.health.smithi102.stderr:2024-04-17 22:28:50.500 7fd5b9dab700 1 -- 172.21.15.102:0/405675064 wait complete. 2024-04-17T22:28:50.495 DEBUG:teuthology.misc:0 of 4 OSDs are up 2024-04-17T22:28:51.539 INFO:tasks.ceph.osd.3.smithi102.stderr:2024-04-17 22:28:51.553 7fdfcdd33dc0 -1 osd.3 0 log_to_monitors {default=true} 2024-04-17T22:28:51.589 INFO:tasks.ceph.osd.0.smithi102.stderr:2024-04-17 22:28:51.603 7efcf55bbdc0 -1 osd.0 0 log_to_monitors {default=true} 2024-04-17T22:28:51.589 INFO:tasks.ceph.osd.2.smithi102.stderr:2024-04-17 22:28:51.604 7f2e03c6ddc0 -1 osd.2 0 log_to_monitors {default=true} 2024-04-17T22:28:51.598 INFO:tasks.ceph.osd.1.smithi102.stderr:2024-04-17 22:28:51.612 7f2f530b4dc0 -1 osd.1 0 log_to_monitors {default=true} 2024-04-17T22:28:53.221 INFO:tasks.ceph.osd.0.smithi102.stderr:2024-04-17 22:28:53.235 7efcdb2d8700 -1 osd.0 0 waiting for initial osdmap 2024-04-17T22:28:53.221 INFO:tasks.ceph.osd.3.smithi102.stderr:2024-04-17 22:28:53.236 7fdfb3a50700 -1 osd.3 0 waiting for initial osdmap 2024-04-17T22:28:53.221 INFO:tasks.ceph.osd.2.smithi102.stderr:2024-04-17 22:28:53.236 7f2de998a700 -1 osd.2 0 waiting for initial osdmap 2024-04-17T22:28:53.226 INFO:tasks.ceph.osd.1.smithi102.stderr:2024-04-17 22:28:53.239 7f2f38dd1700 -1 osd.1 0 waiting for initial osdmap 2024-04-17T22:28:53.227 INFO:tasks.ceph.osd.0.smithi102.stderr:2024-04-17 22:28:53.241 7efce74f6700 -1 osd.0 8 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-17T22:28:53.227 INFO:tasks.ceph.osd.3.smithi102.stderr:2024-04-17 22:28:53.242 7fdfbfc6e700 -1 osd.3 8 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-17T22:28:53.227 INFO:tasks.ceph.osd.2.smithi102.stderr:2024-04-17 22:28:53.242 7f2df5ba8700 -1 osd.2 8 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-17T22:28:53.228 INFO:tasks.ceph.osd.1.smithi102.stderr:2024-04-17 22:28:53.242 7f2f44fef700 -1 osd.1 8 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-17T22:28:56.899 DEBUG:teuthology.orchestra.run.smithi102:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-17T22:28:57.028 INFO:teuthology.misc.health.smithi102.stderr:2024-04-17 22:28:57.040 7fe9e8073700 1 Processor -- start 2024-04-17T22:28:57.028 INFO:teuthology.misc.health.smithi102.stderr:2024-04-17 22:28:57.040 7fe9e8073700 1 -- start start 2024-04-17T22:28:57.028 INFO:teuthology.misc.health.smithi102.stderr:2024-04-17 22:28:57.040 7fe9e8073700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fe9e00f56e0 0x7fe9e010c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:28:57.028 INFO:teuthology.misc.health.smithi102.stderr:2024-04-17 22:28:57.040 7fe9e8073700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fe9e010cb00 0x7fe9e01a1370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:28:57.028 INFO:teuthology.misc.health.smithi102.stderr:2024-04-17 22:28:57.040 7fe9e8073700 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9e01a19a0 con 0x7fe9e010c6b0 2024-04-17T22:28:57.028 INFO:teuthology.misc.health.smithi102.stderr:2024-04-17 22:28:57.040 7fe9e8073700 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9e00f9350 con 0x7fe9e00f56e0 2024-04-17T22:28:57.028 INFO:teuthology.misc.health.smithi102.stderr:2024-04-17 22:28:57.040 7fe9e8073700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe9e01a1bc0 con 0x7fe9e010cb00 2024-04-17T22:28:57.028 INFO:teuthology.misc.health.smithi102.stderr:2024-04-17 22:28:57.040 7fe9e566f700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fe9e010cb00 0x7fe9e01a1370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:28:57.028 INFO:teuthology.misc.health.smithi102.stderr:2024-04-17 22:28:57.040 7fe9e4e6e700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fe9e00f56e0 0x7fe9e010c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:28:57.028 INFO:teuthology.misc.health.smithi102.stderr:2024-04-17 22:28:57.040 7fe9e566f700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fe9e010cb00 0x7fe9e01a1370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:52274/0 (socket says 172.21.15.102:52274) 2024-04-17T22:28:57.028 INFO:teuthology.misc.health.smithi102.stderr:2024-04-17 22:28:57.040 7fe9e4e6e700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fe9e00f56e0 0x7fe9e010c080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:43842/0 (socket says 172.21.15.102:43842) 2024-04-17T22:28:57.029 INFO:teuthology.misc.health.smithi102.stderr:2024-04-17 22:28:57.040 7fe9e566f700 1 -- 172.21.15.102:0/3108349511 learned_addr learned my addr 172.21.15.102:0/3108349511 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:28:57.029 INFO:teuthology.misc.health.smithi102.stderr:2024-04-17 22:28:57.040 7fe9dffff700 1 --1- 172.21.15.102:0/3108349511 >> v1:172.21.15.102:6789/0 conn(0x7fe9e010c6b0 0x7fe9e01a0b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:33678/0 (socket says 172.21.15.102:33678) 2024-04-17T22:28:57.029 INFO:teuthology.misc.health.smithi102.stderr:2024-04-17 22:28:57.040 7fe9e566f700 1 -- 172.21.15.102:0/3108349511 >> v2:172.21.15.102:3300/0 conn(0x7fe9e00f56e0 msgr2=0x7fe9e010c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:28:57.029 INFO:teuthology.misc.health.smithi102.stderr:2024-04-17 22:28:57.040 7fe9e566f700 1 --2- 172.21.15.102:0/3108349511 >> v2:172.21.15.102:3300/0 conn(0x7fe9e00f56e0 0x7fe9e010c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:28:57.029 INFO:teuthology.misc.health.smithi102.stderr:2024-04-17 22:28:57.040 7fe9e566f700 1 -- 172.21.15.102:0/3108349511 >> v1:172.21.15.102:6789/0 conn(0x7fe9e010c6b0 legacy=0x7fe9e01a0b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:28:57.029 INFO:teuthology.misc.health.smithi102.stderr:2024-04-17 22:28:57.041 7fe9e566f700 1 -- 172.21.15.102:0/3108349511 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9e01aa2a0 con 0x7fe9e010cb00 2024-04-17T22:28:57.029 INFO:teuthology.misc.health.smithi102.stderr:2024-04-17 22:28:57.041 7fe9e4e6e700 1 --2- 172.21.15.102:0/3108349511 >> v2:172.21.15.102:3300/0 conn(0x7fe9e00f56e0 0x7fe9e010c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:28:57.029 INFO:teuthology.misc.health.smithi102.stderr:2024-04-17 22:28:57.041 7fe9e566f700 1 --2- 172.21.15.102:0/3108349511 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fe9e010cb00 0x7fe9e01a1370 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 rx=0x7fe9e00f4580 tx=0x7fe9d8009d90).ready entity=mon.2 client_cookie=e1ea2ec5a3a44f75 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:28:57.029 INFO:teuthology.misc.health.smithi102.stderr:2024-04-17 22:28:57.041 7fe9df7fe700 1 -- 172.21.15.102:0/3108349511 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe9e01aa2a0 con 0x7fe9e010cb00 2024-04-17T22:28:57.029 INFO:teuthology.misc.health.smithi102.stderr:2024-04-17 22:28:57.041 7fe9df7fe700 1 -- 172.21.15.102:0/3108349511 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9d800b840 con 0x7fe9e010cb00 2024-04-17T22:28:57.029 INFO:teuthology.misc.health.smithi102.stderr:2024-04-17 22:28:57.041 7fe9df7fe700 1 -- 172.21.15.102:0/3108349511 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe9d800ba20 con 0x7fe9e010cb00 2024-04-17T22:28:57.029 INFO:teuthology.misc.health.smithi102.stderr:2024-04-17 22:28:57.041 7fe9e8073700 1 -- 172.21.15.102:0/3108349511 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fe9e010cb00 msgr2=0x7fe9e01a1370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:28:57.029 INFO:teuthology.misc.health.smithi102.stderr:2024-04-17 22:28:57.041 7fe9e8073700 1 --2- 172.21.15.102:0/3108349511 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fe9e010cb00 0x7fe9e01a1370 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 rx=0x7fe9e00f4580 tx=0x7fe9d8009d90).stop 2024-04-17T22:28:57.029 INFO:teuthology.misc.health.smithi102.stderr:2024-04-17 22:28:57.041 7fe9e8073700 1 -- 172.21.15.102:0/3108349511 shutdown_connections 2024-04-17T22:28:57.029 INFO:teuthology.misc.health.smithi102.stderr:2024-04-17 22:28:57.041 7fe9e8073700 1 --2- 172.21.15.102:0/3108349511 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fe9e010cb00 0x7fe9e01a1370 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:28:57.030 INFO:teuthology.misc.health.smithi102.stderr:2024-04-17 22:28:57.041 7fe9e8073700 1 --2- 172.21.15.102:0/3108349511 >> v2:172.21.15.102:3300/0 conn(0x7fe9e00f56e0 0x7fe9e010c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:28:57.030 INFO:teuthology.misc.health.smithi102.stderr:2024-04-17 22:28:57.041 7fe9e8073700 1 -- 172.21.15.102:0/3108349511 >> 172.21.15.102:0/3108349511 conn(0x7fe9e0113830 msgr2=0x7fe9e00f4ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:28:57.030 INFO:teuthology.misc.health.smithi102.stderr:2024-04-17 22:28:57.041 7fe9e8073700 1 -- 172.21.15.102:0/3108349511 shutdown_connections 2024-04-17T22:28:57.030 INFO:teuthology.misc.health.smithi102.stderr:2024-04-17 22:28:57.041 7fe9e8073700 1 -- 172.21.15.102:0/3108349511 wait complete. 2024-04-17T22:28:57.049 INFO:teuthology.misc.health.smithi102.stderr:2024-04-17 22:28:57.064 7fe9e8073700 1 Processor -- start 2024-04-17T22:28:57.050 INFO:teuthology.misc.health.smithi102.stderr:2024-04-17 22:28:57.064 7fe9e8073700 1 -- start start 2024-04-17T22:28:57.052 INFO:teuthology.misc.health.smithi102.stderr:2024-04-17 22:28:57.064 7fe9e8073700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe9e00dade0 0x7fe9e00c6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:28:57.052 INFO:teuthology.misc.health.smithi102.stderr:2024-04-17 22:28:57.064 7fe9e8073700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe9e00c6790 0x7fe9e01a9030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:28:57.052 INFO:teuthology.misc.health.smithi102.stderr:2024-04-17 22:28:57.064 7fe9e8073700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fe9e00c6be0 0x7fe9e01a9660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:28:57.052 INFO:teuthology.misc.health.smithi102.stderr:2024-04-17 22:28:57.064 7fe9e8073700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9e01a9c90 con 0x7fe9e00dade0 2024-04-17T22:28:57.052 INFO:teuthology.misc.health.smithi102.stderr:2024-04-17 22:28:57.064 7fe9e8073700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9e01a9e70 con 0x7fe9e00c6790 2024-04-17T22:28:57.052 INFO:teuthology.misc.health.smithi102.stderr:2024-04-17 22:28:57.064 7fe9e8073700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe9e00db370 con 0x7fe9e00c6be0 2024-04-17T22:28:57.052 INFO:teuthology.misc.health.smithi102.stderr:2024-04-17 22:28:57.065 7fe9e4e6e700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe9e00dade0 0x7fe9e00c6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:28:57.052 INFO:teuthology.misc.health.smithi102.stderr:2024-04-17 22:28:57.065 7fe9dffff700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe9e00c6790 0x7fe9e01a9030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:28:57.052 INFO:teuthology.misc.health.smithi102.stderr:2024-04-17 22:28:57.065 7fe9e566f700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fe9e00c6be0 0x7fe9e01a9660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:28:57.052 INFO:teuthology.misc.health.smithi102.stderr:2024-04-17 22:28:57.065 7fe9e4e6e700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe9e00dade0 0x7fe9e00c6160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:43852/0 (socket says 172.21.15.102:43852) 2024-04-17T22:28:57.053 INFO:teuthology.misc.health.smithi102.stderr:2024-04-17 22:28:57.065 7fe9e566f700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fe9e00c6be0 0x7fe9e01a9660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:52278/0 (socket says 172.21.15.102:52278) 2024-04-17T22:28:57.053 INFO:teuthology.misc.health.smithi102.stderr:2024-04-17 22:28:57.065 7fe9dffff700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe9e00c6790 0x7fe9e01a9030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:37660/0 (socket says 172.21.15.102:37660) 2024-04-17T22:28:57.053 INFO:teuthology.misc.health.smithi102.stderr:2024-04-17 22:28:57.065 7fe9e4e6e700 1 -- 172.21.15.102:0/3574157486 learned_addr learned my addr 172.21.15.102:0/3574157486 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:28:57.053 INFO:teuthology.misc.health.smithi102.stderr:2024-04-17 22:28:57.065 7fe9e4e6e700 1 -- 172.21.15.102:0/3574157486 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fe9e00c6be0 msgr2=0x7fe9e01a9660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:28:57.053 INFO:teuthology.misc.health.smithi102.stderr:2024-04-17 22:28:57.065 7fe9e4e6e700 1 --2- 172.21.15.102:0/3574157486 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fe9e00c6be0 0x7fe9e01a9660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:28:57.053 INFO:teuthology.misc.health.smithi102.stderr:2024-04-17 22:28:57.065 7fe9e4e6e700 1 -- 172.21.15.102:0/3574157486 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe9e00c6790 msgr2=0x7fe9e01a9030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:28:57.053 INFO:teuthology.misc.health.smithi102.stderr:2024-04-17 22:28:57.065 7fe9e4e6e700 1 --2- 172.21.15.102:0/3574157486 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe9e00c6790 0x7fe9e01a9030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:28:57.053 INFO:teuthology.misc.health.smithi102.stderr:2024-04-17 22:28:57.065 7fe9e4e6e700 1 -- 172.21.15.102:0/3574157486 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9e01b6f90 con 0x7fe9e00dade0 2024-04-17T22:28:57.053 INFO:teuthology.misc.health.smithi102.stderr:2024-04-17 22:28:57.065 7fe9e566f700 1 --2- 172.21.15.102:0/3574157486 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fe9e00c6be0 0x7fe9e01a9660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:28:57.053 INFO:teuthology.misc.health.smithi102.stderr:2024-04-17 22:28:57.065 7fe9dffff700 1 --2- 172.21.15.102:0/3574157486 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe9e00c6790 0x7fe9e01a9030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:28:57.053 INFO:teuthology.misc.health.smithi102.stderr:2024-04-17 22:28:57.065 7fe9e4e6e700 1 --2- 172.21.15.102:0/3574157486 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe9e00dade0 0x7fe9e00c6160 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7fe9e00c35e0 tx=0x7fe9d400bf50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:28:57.053 INFO:teuthology.misc.health.smithi102.stderr:2024-04-17 22:28:57.065 7fe9ddffb700 1 -- 172.21.15.102:0/3574157486 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe9e01b6f90 con 0x7fe9e00dade0 2024-04-17T22:28:57.053 INFO:teuthology.misc.health.smithi102.stderr:2024-04-17 22:28:57.065 7fe9ddffb700 1 -- 172.21.15.102:0/3574157486 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9d400ab00 con 0x7fe9e00dade0 2024-04-17T22:28:57.053 INFO:teuthology.misc.health.smithi102.stderr:2024-04-17 22:28:57.065 7fe9e8073700 1 -- 172.21.15.102:0/3574157486 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9e01b7190 con 0x7fe9e00dade0 2024-04-17T22:28:57.054 INFO:teuthology.misc.health.smithi102.stderr:2024-04-17 22:28:57.065 7fe9ddffb700 1 -- 172.21.15.102:0/3574157486 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe9d4016030 con 0x7fe9e00dade0 2024-04-17T22:28:57.054 INFO:teuthology.misc.health.smithi102.stderr:2024-04-17 22:28:57.065 7fe9e8073700 1 -- 172.21.15.102:0/3574157486 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9e01b7570 con 0x7fe9e00dade0 2024-04-17T22:28:57.054 INFO:teuthology.misc.health.smithi102.stderr:2024-04-17 22:28:57.066 7fe9ddffb700 1 -- 172.21.15.102:0/3574157486 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7fe9d401a420 con 0x7fe9e00dade0 2024-04-17T22:28:57.054 INFO:teuthology.misc.health.smithi102.stderr:2024-04-17 22:28:57.066 7fe9ddffb700 1 -- 172.21.15.102:0/3574157486 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 2021+0+0 (secure 0 0 0) 0x7fe9d401c950 con 0x7fe9e00dade0 2024-04-17T22:28:57.054 INFO:teuthology.misc.health.smithi102.stderr:2024-04-17 22:28:57.066 7fe9e8073700 1 -- 172.21.15.102:0/3574157486 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9e00f0790 con 0x7fe9e00dade0 2024-04-17T22:28:57.055 INFO:teuthology.misc.health.smithi102.stderr:2024-04-17 22:28:57.070 7fe9ddffb700 1 -- 172.21.15.102:0/3574157486 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+110455 (secure 0 0 0) 0x7fe9d400a700 con 0x7fe9e00dade0 2024-04-17T22:28:57.161 INFO:teuthology.misc.health.smithi102.stderr:2024-04-17 22:28:57.175 7fe9e8073700 1 -- 172.21.15.102:0/3574157486 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe9e01aa020 con 0x7fe9e00dade0 2024-04-17T22:28:57.161 INFO:teuthology.misc.health.smithi102.stderr:2024-04-17 22:28:57.176 7fe9ddffb700 1 -- 172.21.15.102:0/3574157486 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v9) v1 ==== 74+0+4536 (secure 0 0 0) 0x7fe9d4000c30 con 0x7fe9e00dade0 2024-04-17T22:28:57.161 INFO:teuthology.misc.health.smithi102.stdout: 2024-04-17T22:28:57.161 INFO:teuthology.misc.health.smithi102.stdout:{"epoch":9,"fsid":"cb63e789-091d-49b1-b120-55f9db526871","created":"2024-04-17 22:28:48.209635","modified":"2024-04-17 22:28:54.236215","last_up_change":"2024-04-17 22:28:54.236215","last_in_change":"2024-04-17 22:28:54.236215","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":4,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[],"osds":[{"osd":0,"uuid":"3351ef85-ade4-4b90-a8f4-213f4e00ba01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6812","nonce":42955}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6813","nonce":42955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6815","nonce":42955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6814","nonce":42955}]},"public_addr":"172.21.15.102:6812/42955","cluster_addr":"172.21.15.102:6813/42955","heartbeat_back_addr":"172.21.15.102:6815/42955","heartbeat_front_addr":"172.21.15.102:6814/42955","state":["exists","up"]},{"osd":1,"uuid":"04f67eac-f36b-48f5-982d-4cfc776596d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6808","nonce":42958}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6809","nonce":42958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6811","nonce":42958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6810","nonce":42958}]},"public_addr":"172.21.15.102:6808/42958","cluster_addr":"172.21.15.102:6809/42958","heartbeat_back_addr":"172.21.15.102:6811/42958","heartbeat_front_addr":"172.21.15.102:6810/42958","state":["exists","up"]},{"osd":2,"uuid":"d524204b-70b2-40a8-8f73-f9e7d44318af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6804","nonce":42954}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6805","nonce":42954}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6807","nonce":42954}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6806","nonce":42954}]},"public_addr":"172.21.15.102:6804/42954","cluster_addr":"172.21.15.102:6805/42954","heartbeat_back_addr":"172.21.15.102:6807/42954","heartbeat_front_addr":"172.21.15.102:6806/42954","state":["exists","up"]},{"osd":3,"uuid":"4fc9d20c-1ad0-484f-aafa-2c9a7574245b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6800","nonce":42953}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6801","nonce":42953}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6803","nonce":42953}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6802","nonce":42953}]},"public_addr":"172.21.15.102:6800/42953","cluster_addr":"172.21.15.102:6801/42953","heartbeat_back_addr":"172.21.15.102:6803/42953","heartbeat_front_addr":"172.21.15.102:6802/42953","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2024-04-17T22:28:57.163 INFO:teuthology.misc.health.smithi102.stderr:2024-04-17 22:28:57.176 7fe9e8073700 1 -- 172.21.15.102:0/3574157486 >> v1:172.21.15.102:6816/42360 conn(0x7fe9c80145a0 legacy=0x7fe9c8016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:28:57.163 INFO:teuthology.misc.health.smithi102.stderr:2024-04-17 22:28:57.177 7fe9e8073700 1 -- 172.21.15.102:0/3574157486 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe9e00dade0 msgr2=0x7fe9e00c6160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:28:57.163 INFO:teuthology.misc.health.smithi102.stderr:2024-04-17 22:28:57.177 7fe9e8073700 1 --2- 172.21.15.102:0/3574157486 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe9e00dade0 0x7fe9e00c6160 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7fe9e00c35e0 tx=0x7fe9d400bf50).stop 2024-04-17T22:28:57.163 INFO:teuthology.misc.health.smithi102.stderr:2024-04-17 22:28:57.177 7fe9e8073700 1 -- 172.21.15.102:0/3574157486 shutdown_connections 2024-04-17T22:28:57.163 INFO:teuthology.misc.health.smithi102.stderr:2024-04-17 22:28:57.177 7fe9e8073700 1 --2- 172.21.15.102:0/3574157486 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fe9e00c6be0 0x7fe9e01a9660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:28:57.163 INFO:teuthology.misc.health.smithi102.stderr:2024-04-17 22:28:57.177 7fe9e8073700 1 --2- 172.21.15.102:0/3574157486 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe9e00dade0 0x7fe9e00c6160 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:28:57.163 INFO:teuthology.misc.health.smithi102.stderr:2024-04-17 22:28:57.177 7fe9e8073700 1 --2- 172.21.15.102:0/3574157486 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe9e00c6790 0x7fe9e01a9030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:28:57.163 INFO:teuthology.misc.health.smithi102.stderr:2024-04-17 22:28:57.177 7fe9e8073700 1 -- 172.21.15.102:0/3574157486 >> 172.21.15.102:0/3574157486 conn(0x7fe9e00e0de0 msgr2=0x7fe9e00de420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:28:57.163 INFO:teuthology.misc.health.smithi102.stderr:2024-04-17 22:28:57.177 7fe9e8073700 1 -- 172.21.15.102:0/3574157486 shutdown_connections 2024-04-17T22:28:57.163 INFO:teuthology.misc.health.smithi102.stderr:2024-04-17 22:28:57.177 7fe9e8073700 1 -- 172.21.15.102:0/3574157486 wait complete. 2024-04-17T22:28:57.172 DEBUG:teuthology.misc:4 of 4 OSDs are up 2024-04-17T22:28:57.172 INFO:tasks.ceph:Creating RBD pool 2024-04-17T22:28:57.172 DEBUG:teuthology.orchestra.run.smithi102:> sudo ceph --cluster ceph osd pool create rbd 8 2024-04-17T22:28:57.301 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:57.313 7f6a8f2d0700 1 Processor -- start 2024-04-17T22:28:57.301 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:57.313 7f6a8f2d0700 1 -- start start 2024-04-17T22:28:57.301 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:57.313 7f6a8f2d0700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6a880f5750 0x7f6a8810c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:28:57.301 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:57.313 7f6a8f2d0700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f6a8810c720 0x7f6a881a0be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:28:57.301 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:57.313 7f6a8f2d0700 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a881a1a10 con 0x7f6a8810cb70 2024-04-17T22:28:57.301 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:57.313 7f6a8f2d0700 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a880f93c0 con 0x7f6a8810c720 2024-04-17T22:28:57.301 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:57.313 7f6a8f2d0700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a881a1c30 con 0x7f6a880f5750 2024-04-17T22:28:57.301 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:57.313 7f6a87fff700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6a880f5750 0x7f6a8810c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:28:57.301 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:57.313 7f6a7ffff700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f6a8810c720 0x7f6a881a0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:28:57.301 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:57.313 7f6a87fff700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6a880f5750 0x7f6a8810c0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:37668/0 (socket says 172.21.15.102:37668) 2024-04-17T22:28:57.301 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:57.313 7f6a8c8cc700 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f6a8810cb70 0x7f6a881a1210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:33692/0 (socket says 172.21.15.102:33692) 2024-04-17T22:28:57.302 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:57.313 7f6a87fff700 1 -- 172.21.15.102:0/2065094848 learned_addr learned my addr 172.21.15.102:0/2065094848 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:28:57.302 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:57.314 7f6a87fff700 1 -- 172.21.15.102:0/2065094848 >> v2:172.21.15.102:3300/0 conn(0x7f6a8810c720 msgr2=0x7f6a881a0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:28:57.302 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:57.314 7f6a87fff700 1 --2- 172.21.15.102:0/2065094848 >> v2:172.21.15.102:3300/0 conn(0x7f6a8810c720 0x7f6a881a0be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:28:57.302 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:57.314 7f6a87fff700 1 -- 172.21.15.102:0/2065094848 >> v1:172.21.15.102:6789/0 conn(0x7f6a8810cb70 legacy=0x7f6a881a1210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:28:57.302 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:57.314 7f6a87fff700 1 -- 172.21.15.102:0/2065094848 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a881aa310 con 0x7f6a880f5750 2024-04-17T22:28:57.302 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:57.314 7f6a877fe700 1 -- 172.21.15.102:0/2065094848 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3816039073 0 0) 0x7f6a80001490 con 0x7f6a8810cb70 2024-04-17T22:28:57.302 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:57.314 7f6a7ffff700 1 --2- 172.21.15.102:0/2065094848 >> v2:172.21.15.102:3300/0 conn(0x7f6a8810c720 0x7f6a881a0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:28:57.302 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:57.314 7f6a87fff700 1 --2- 172.21.15.102:0/2065094848 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6a880f5750 0x7f6a8810c0f0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 rx=0x7f6a78002f70 tx=0x7f6a780099d0).ready entity=mon.1 client_cookie=4edcf1275b464369 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:28:57.302 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:57.314 7f6a877fe700 1 -- 172.21.15.102:0/2065094848 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f6a881aa310 con 0x7f6a880f5750 2024-04-17T22:28:57.302 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:57.314 7f6a877fe700 1 -- 172.21.15.102:0/2065094848 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a7800f510 con 0x7f6a880f5750 2024-04-17T22:28:57.302 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:57.314 7f6a877fe700 1 -- 172.21.15.102:0/2065094848 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f6a7800f6f0 con 0x7f6a880f5750 2024-04-17T22:28:57.302 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:57.314 7f6a8f2d0700 1 -- 172.21.15.102:0/2065094848 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6a880f5750 msgr2=0x7f6a8810c0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:28:57.302 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:57.314 7f6a8f2d0700 1 --2- 172.21.15.102:0/2065094848 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6a880f5750 0x7f6a8810c0f0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 rx=0x7f6a78002f70 tx=0x7f6a780099d0).stop 2024-04-17T22:28:57.303 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:57.315 7f6a8f2d0700 1 -- 172.21.15.102:0/2065094848 shutdown_connections 2024-04-17T22:28:57.303 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:57.315 7f6a8f2d0700 1 --2- 172.21.15.102:0/2065094848 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6a880f5750 0x7f6a8810c0f0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:28:57.303 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:57.315 7f6a8f2d0700 1 --2- 172.21.15.102:0/2065094848 >> v2:172.21.15.102:3300/0 conn(0x7f6a8810c720 0x7f6a881a0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:28:57.303 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:57.315 7f6a8f2d0700 1 -- 172.21.15.102:0/2065094848 >> 172.21.15.102:0/2065094848 conn(0x7f6a881138a0 msgr2=0x7f6a880f4f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:28:57.303 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:57.315 7f6a8f2d0700 1 -- 172.21.15.102:0/2065094848 shutdown_connections 2024-04-17T22:28:57.303 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:57.315 7f6a8f2d0700 1 -- 172.21.15.102:0/2065094848 wait complete. 2024-04-17T22:28:57.325 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:57.337 7f6a8f2d0700 1 Processor -- start 2024-04-17T22:28:57.325 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:57.337 7f6a8f2d0700 1 -- start start 2024-04-17T22:28:57.325 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:57.338 7f6a8f2d0700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f6a881ad500 0x7f6a881ad950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:28:57.325 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:57.338 7f6a8f2d0700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6a881adf80 0x7f6a881b3e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:28:57.325 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:57.338 7f6a8f2d0700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6a881b4460 0x7f6a881b48b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:28:57.325 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:57.338 7f6a8f2d0700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a880c43f0 con 0x7f6a881adf80 2024-04-17T22:28:57.325 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:57.338 7f6a8f2d0700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a881a6700 con 0x7f6a881b4460 2024-04-17T22:28:57.325 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:57.338 7f6a8f2d0700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6a881a6b50 con 0x7f6a881ad500 2024-04-17T22:28:57.325 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:57.338 7f6a7ffff700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6a881adf80 0x7f6a881b3e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:28:57.325 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:57.338 7f6a87fff700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f6a881ad500 0x7f6a881ad950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:28:57.325 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:57.338 7f6a7ffff700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6a881adf80 0x7f6a881b3e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:43870/0 (socket says 172.21.15.102:43870) 2024-04-17T22:28:57.326 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:57.338 7f6a7ffff700 1 -- 172.21.15.102:0/3078831956 learned_addr learned my addr 172.21.15.102:0/3078831956 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:28:57.326 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:57.338 7f6a8c8cc700 1 --2- 172.21.15.102:0/3078831956 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6a881b4460 0x7f6a881b48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:28:57.326 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:57.338 7f6a7ffff700 1 -- 172.21.15.102:0/3078831956 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f6a881ad500 msgr2=0x7f6a881ad950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:28:57.326 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:57.338 7f6a7ffff700 1 --2- 172.21.15.102:0/3078831956 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f6a881ad500 0x7f6a881ad950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:28:57.326 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:57.338 7f6a7ffff700 1 -- 172.21.15.102:0/3078831956 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6a881b4460 msgr2=0x7f6a881b48b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:28:57.326 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:57.338 7f6a7ffff700 1 --2- 172.21.15.102:0/3078831956 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6a881b4460 0x7f6a881b48b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:28:57.326 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:57.338 7f6a7ffff700 1 -- 172.21.15.102:0/3078831956 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a881bb5b0 con 0x7f6a881adf80 2024-04-17T22:28:57.326 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:57.338 7f6a87fff700 1 --2- 172.21.15.102:0/3078831956 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f6a881ad500 0x7f6a881ad950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:28:57.326 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:57.338 7f6a7ffff700 1 --2- 172.21.15.102:0/3078831956 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6a881adf80 0x7f6a881b3e30 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7f6a7000c000 tx=0x7f6a7000ecd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:28:57.326 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:57.338 7f6a85ffb700 1 -- 172.21.15.102:0/3078831956 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f6a881bb5b0 con 0x7f6a881adf80 2024-04-17T22:28:57.326 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:57.338 7f6a85ffb700 1 -- 172.21.15.102:0/3078831956 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a7000c7b0 con 0x7f6a881adf80 2024-04-17T22:28:57.326 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:57.338 7f6a85ffb700 1 -- 172.21.15.102:0/3078831956 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f6a7000f040 con 0x7f6a881adf80 2024-04-17T22:28:57.327 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:57.338 7f6a8f2d0700 1 -- 172.21.15.102:0/3078831956 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a881bb7b0 con 0x7f6a881adf80 2024-04-17T22:28:57.327 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:57.338 7f6a8f2d0700 1 -- 172.21.15.102:0/3078831956 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a881bbb90 con 0x7f6a881adf80 2024-04-17T22:28:57.327 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:57.339 7f6a85ffb700 1 -- 172.21.15.102:0/3078831956 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7f6a70011070 con 0x7f6a881adf80 2024-04-17T22:28:57.327 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:57.339 7f6a85ffb700 1 -- 172.21.15.102:0/3078831956 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 2021+0+0 (secure 0 0 0) 0x7f6a70010040 con 0x7f6a881adf80 2024-04-17T22:28:57.327 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:57.339 7f6a8f2d0700 1 -- 172.21.15.102:0/3078831956 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a880f0800 con 0x7f6a881adf80 2024-04-17T22:28:57.328 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:57.343 7f6a85ffb700 1 -- 172.21.15.102:0/3078831956 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+110455 (secure 0 0 0) 0x7f6a7000c5a0 con 0x7f6a881adf80 2024-04-17T22:28:57.433 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:57.447 7f6a8f2d0700 1 -- 172.21.15.102:0/3078831956 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "rbd", "pg_num": 8} v 0) v1 -- 0x7f6a881ae590 con 0x7f6a881adf80 2024-04-17T22:28:58.229 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:58.242 7f6a85ffb700 1 -- 172.21.15.102:0/3078831956 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}]=0 pool 'rbd' created v10) v1 ==== 109+0+0 (secure 0 0 0) 0x7f6a70000c30 con 0x7f6a881adf80 2024-04-17T22:28:58.232 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:58.245 7f6a8f2d0700 1 -- 172.21.15.102:0/3078831956 >> v1:172.21.15.102:6816/42360 conn(0x7f6a6c0145a0 legacy=0x7f6a6c016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:28:58.232 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:58.245 7f6a8f2d0700 1 -- 172.21.15.102:0/3078831956 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6a881adf80 msgr2=0x7f6a881b3e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:28:58.232 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:58.245 7f6a8f2d0700 1 --2- 172.21.15.102:0/3078831956 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6a881adf80 0x7f6a881b3e30 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7f6a7000c000 tx=0x7f6a7000ecd0).stop 2024-04-17T22:28:58.232 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:58.245 7f6a8f2d0700 1 -- 172.21.15.102:0/3078831956 shutdown_connections 2024-04-17T22:28:58.232 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:58.245 7f6a8f2d0700 1 --2- 172.21.15.102:0/3078831956 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6a881b4460 0x7f6a881b48b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:28:58.232 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:58.245 7f6a8f2d0700 1 --2- 172.21.15.102:0/3078831956 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f6a881ad500 0x7f6a881ad950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:28:58.232 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:58.245 7f6a8f2d0700 1 --2- 172.21.15.102:0/3078831956 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6a881adf80 0x7f6a881b3e30 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:28:58.232 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:58.245 7f6a8f2d0700 1 -- 172.21.15.102:0/3078831956 >> 172.21.15.102:0/3078831956 conn(0x7f6a880c5390 msgr2=0x7f6a880dada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:28:58.232 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:58.245 7f6a8f2d0700 1 -- 172.21.15.102:0/3078831956 shutdown_connections 2024-04-17T22:28:58.232 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:58.245 7f6a8f2d0700 1 -- 172.21.15.102:0/3078831956 wait complete. 2024-04-17T22:28:58.233 INFO:teuthology.orchestra.run.smithi102.stderr:pool 'rbd' created 2024-04-17T22:28:58.246 DEBUG:teuthology.orchestra.run.smithi102:> sudo ceph --cluster ceph osd pool application enable rbd rbd --yes-i-really-mean-it 2024-04-17T22:28:58.379 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:58.390 7f9a68fa4700 1 Processor -- start 2024-04-17T22:28:58.379 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:58.390 7f9a68fa4700 1 -- start start 2024-04-17T22:28:58.379 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:58.391 7f9a68fa4700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f9a640f57b0 0x7f9a6410ae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:28:58.379 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:58.391 7f9a68fa4700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f9a6410b8a0 0x7f9a641a02d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:28:58.379 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:58.391 7f9a68fa4700 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a641a0900 con 0x7f9a6410b450 2024-04-17T22:28:58.379 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:58.391 7f9a68fa4700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a641a0b20 con 0x7f9a6410b8a0 2024-04-17T22:28:58.380 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:58.391 7f9a68fa4700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9a641056b0 con 0x7f9a640f57b0 2024-04-17T22:28:58.380 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:58.391 7f9a625a0700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f9a6410b8a0 0x7f9a641a02d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:28:58.380 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:58.391 7f9a6159e700 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f9a6410b450 0x7f9a6419fad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:33704/0 (socket says 172.21.15.102:33704) 2024-04-17T22:28:58.380 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:58.391 7f9a61d9f700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f9a640f57b0 0x7f9a6410ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:28:58.380 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:58.391 7f9a6159e700 1 -- 172.21.15.102:0/2333837271 learned_addr learned my addr 172.21.15.102:0/2333837271 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-17T22:28:58.380 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:58.391 7f9a60d9d700 1 -- 172.21.15.102:0/2333837271 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1088295049 0 0) 0x7f9a54001110 con 0x7f9a6410b450 2024-04-17T22:28:58.380 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:58.391 7f9a60d9d700 1 -- 172.21.15.102:0/2333837271 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9a48003270 con 0x7f9a6410b450 2024-04-17T22:28:58.380 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:58.391 7f9a625a0700 1 -- 172.21.15.102:0/2333837271 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f9a640f57b0 msgr2=0x7f9a6410ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:28:58.380 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:58.391 7f9a625a0700 1 --2- 172.21.15.102:0/2333837271 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f9a640f57b0 0x7f9a6410ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:28:58.380 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:58.391 7f9a625a0700 1 -- 172.21.15.102:0/2333837271 >> v1:172.21.15.102:6789/0 conn(0x7f9a6410b450 legacy=0x7f9a6419fad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:28:58.380 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:58.391 7f9a625a0700 1 -- 172.21.15.102:0/2333837271 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a641a91e0 con 0x7f9a6410b8a0 2024-04-17T22:28:58.380 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:58.391 7f9a61d9f700 1 --2- 172.21.15.102:0/2333837271 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f9a640f57b0 0x7f9a6410ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:28:58.380 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:58.391 7f9a625a0700 1 --2- 172.21.15.102:0/2333837271 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f9a6410b8a0 0x7f9a641a02d0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 rx=0x7f9a5c0032f0 tx=0x7f9a5c00bdf0).ready entity=mon.1 client_cookie=2ad5fadfe13bd3fd server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:28:58.380 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:58.392 7f9a60d9d700 1 -- 172.21.15.102:0/2333837271 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9a641a91e0 con 0x7f9a6410b8a0 2024-04-17T22:28:58.381 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:58.392 7f9a60d9d700 1 -- 172.21.15.102:0/2333837271 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a5c00cd90 con 0x7f9a6410b8a0 2024-04-17T22:28:58.381 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:58.392 7f9a60d9d700 1 -- 172.21.15.102:0/2333837271 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9a5c014070 con 0x7f9a6410b8a0 2024-04-17T22:28:58.381 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:58.392 7f9a68fa4700 1 -- 172.21.15.102:0/2333837271 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f9a6410b8a0 msgr2=0x7f9a641a02d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:28:58.381 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:58.392 7f9a68fa4700 1 --2- 172.21.15.102:0/2333837271 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f9a6410b8a0 0x7f9a641a02d0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 rx=0x7f9a5c0032f0 tx=0x7f9a5c00bdf0).stop 2024-04-17T22:28:58.381 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:58.392 7f9a68fa4700 1 -- 172.21.15.102:0/2333837271 shutdown_connections 2024-04-17T22:28:58.381 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:58.392 7f9a68fa4700 1 --2- 172.21.15.102:0/2333837271 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f9a6410b8a0 0x7f9a641a02d0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:28:58.381 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:58.392 7f9a68fa4700 1 --2- 172.21.15.102:0/2333837271 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f9a640f57b0 0x7f9a6410ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:28:58.381 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:58.392 7f9a68fa4700 1 -- 172.21.15.102:0/2333837271 >> 172.21.15.102:0/2333837271 conn(0x7f9a641138d0 msgr2=0x7f9a640f4fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:28:58.381 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:58.392 7f9a68fa4700 1 -- 172.21.15.102:0/2333837271 shutdown_connections 2024-04-17T22:28:58.381 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:58.392 7f9a68fa4700 1 -- 172.21.15.102:0/2333837271 wait complete. 2024-04-17T22:28:58.403 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:58.415 7f9a68fa4700 1 Processor -- start 2024-04-17T22:28:58.403 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:58.415 7f9a68fa4700 1 -- start start 2024-04-17T22:28:58.403 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:58.415 7f9a68fa4700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9a640ce290 0x7f9a640ce6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:28:58.403 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:58.415 7f9a68fa4700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f9a640ced10 0x7f9a641afe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:28:58.403 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:58.415 7f9a68fa4700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f9a641b0490 0x7f9a641b08e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:28:58.403 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:58.415 7f9a68fa4700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a640ec110 con 0x7f9a640ce290 2024-04-17T22:28:58.404 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:58.415 7f9a68fa4700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a640d6680 con 0x7f9a641b0490 2024-04-17T22:28:58.404 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:58.415 7f9a68fa4700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9a641b0f10 con 0x7f9a640ced10 2024-04-17T22:28:58.404 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:58.415 7f9a61d9f700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9a640ce290 0x7f9a640ce6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:28:58.404 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:58.415 7f9a6159e700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f9a640ced10 0x7f9a641afe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:28:58.404 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:58.415 7f9a61d9f700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9a640ce290 0x7f9a640ce6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:43872/0 (socket says 172.21.15.102:43872) 2024-04-17T22:28:58.404 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:58.415 7f9a61d9f700 1 -- 172.21.15.102:0/1130129530 learned_addr learned my addr 172.21.15.102:0/1130129530 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:28:58.404 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:58.415 7f9a625a0700 1 --2- 172.21.15.102:0/1130129530 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f9a641b0490 0x7f9a641b08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:28:58.404 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:58.416 7f9a6159e700 1 -- 172.21.15.102:0/1130129530 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f9a641b0490 msgr2=0x7f9a641b08e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:28:58.404 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:58.416 7f9a6159e700 1 --2- 172.21.15.102:0/1130129530 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f9a641b0490 0x7f9a641b08e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:28:58.404 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:58.416 7f9a6159e700 1 -- 172.21.15.102:0/1130129530 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9a640ce290 msgr2=0x7f9a640ce6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:28:58.404 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:58.416 7f9a6159e700 1 --2- 172.21.15.102:0/1130129530 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9a640ce290 0x7f9a640ce6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:28:58.404 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:58.416 7f9a6159e700 1 -- 172.21.15.102:0/1130129530 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a640d70d0 con 0x7f9a640ced10 2024-04-17T22:28:58.404 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:58.416 7f9a61d9f700 1 --2- 172.21.15.102:0/1130129530 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9a640ce290 0x7f9a640ce6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:28:58.404 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:58.416 7f9a6159e700 1 --2- 172.21.15.102:0/1130129530 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f9a640ced10 0x7f9a641afe60 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 rx=0x7f9a640c17e0 tx=0x7f9a5400c760).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:28:58.405 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:58.416 7f9a52ffd700 1 -- 172.21.15.102:0/1130129530 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9a640d70d0 con 0x7f9a640ced10 2024-04-17T22:28:58.405 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:58.416 7f9a52ffd700 1 -- 172.21.15.102:0/1130129530 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a540070d0 con 0x7f9a640ced10 2024-04-17T22:28:58.405 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:58.416 7f9a68fa4700 1 -- 172.21.15.102:0/1130129530 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a640d7790 con 0x7f9a640ced10 2024-04-17T22:28:58.405 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:58.416 7f9a52ffd700 1 -- 172.21.15.102:0/1130129530 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9a54005620 con 0x7f9a640ced10 2024-04-17T22:28:58.405 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:58.416 7f9a68fa4700 1 -- 172.21.15.102:0/1130129530 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a641b7840 con 0x7f9a640ced10 2024-04-17T22:28:58.405 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:58.416 7f9a52ffd700 1 -- 172.21.15.102:0/1130129530 <== mon.2 v2:172.21.15.102:3302/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7f9a54005820 con 0x7f9a640ced10 2024-04-17T22:28:58.405 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:58.416 7f9a52ffd700 1 -- 172.21.15.102:0/1130129530 <== mon.2 v2:172.21.15.102:3302/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 2363+0+0 (secure 0 0 0) 0x7f9a54004d40 con 0x7f9a640ced10 2024-04-17T22:28:58.405 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:58.417 7f9a68fa4700 1 -- 172.21.15.102:0/1130129530 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a640f0860 con 0x7f9a640ced10 2024-04-17T22:28:58.407 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:58.420 7f9a52ffd700 1 -- 172.21.15.102:0/1130129530 <== mon.2 v2:172.21.15.102:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+110455 (secure 0 0 0) 0x7f9a5400cdb0 con 0x7f9a640ced10 2024-04-17T22:28:58.512 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:58.526 7f9a68fa4700 1 -- 172.21.15.102:0/1130129530 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f9a641aa5c0 con 0x7f9a640ced10 2024-04-17T22:28:59.233 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.247 7f9a52ffd700 1 -- 172.21.15.102:0/1130129530 <== mon.2 v2:172.21.15.102:3302/0 7 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true}]=0 enabled application 'rbd' on pool 'rbd' v11) v1 ==== 173+0+0 (secure 0 0 0) 0x7f9a640f14a0 con 0x7f9a640ced10 2024-04-17T22:28:59.237 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.250 7f9a68fa4700 1 -- 172.21.15.102:0/1130129530 >> v1:172.21.15.102:6816/42360 conn(0x7f9a400145a0 legacy=0x7f9a40016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:28:59.237 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.250 7f9a68fa4700 1 -- 172.21.15.102:0/1130129530 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f9a640ced10 msgr2=0x7f9a641afe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:28:59.237 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.250 7f9a68fa4700 1 --2- 172.21.15.102:0/1130129530 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f9a640ced10 0x7f9a641afe60 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 rx=0x7f9a640c17e0 tx=0x7f9a5400c760).stop 2024-04-17T22:28:59.238 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.251 7f9a68fa4700 1 -- 172.21.15.102:0/1130129530 shutdown_connections 2024-04-17T22:28:59.238 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.251 7f9a68fa4700 1 --2- 172.21.15.102:0/1130129530 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f9a641b0490 0x7f9a641b08e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:28:59.238 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.251 7f9a68fa4700 1 --2- 172.21.15.102:0/1130129530 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9a640ce290 0x7f9a640ce6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:28:59.238 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.251 7f9a68fa4700 1 --2- 172.21.15.102:0/1130129530 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f9a640ced10 0x7f9a641afe60 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:28:59.238 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.251 7f9a68fa4700 1 -- 172.21.15.102:0/1130129530 >> 172.21.15.102:0/1130129530 conn(0x7f9a640c53b0 msgr2=0x7f9a641a3fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:28:59.238 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.251 7f9a68fa4700 1 -- 172.21.15.102:0/1130129530 shutdown_connections 2024-04-17T22:28:59.238 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.251 7f9a68fa4700 1 -- 172.21.15.102:0/1130129530 wait complete. 2024-04-17T22:28:59.241 INFO:teuthology.orchestra.run.smithi102.stderr:enabled application 'rbd' on pool 'rbd' 2024-04-17T22:28:59.252 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-04-17T22:28:59.252 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-17T22:28:59.252 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-17T22:28:59.255 INFO:tasks.ceph.mds.a:Started 2024-04-17T22:28:59.255 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-04-17T22:28:59.256 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T22:28:59.328 INFO:tasks.ceph.mds.a.smithi102.stderr:warning: line 130: 'mon_down_mkfs_grace' in section 'mon' redefined 2024-04-17T22:28:59.378 INFO:tasks.ceph.mds.a.smithi102.stdout:starting mds.a at 2024-04-17T22:28:59.387 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.401 7fc027985700 1 Processor -- start 2024-04-17T22:28:59.387 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.401 7fc027985700 1 -- start start 2024-04-17T22:28:59.390 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.401 7fc027985700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fc0200f5860 0x7fc02010b030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:28:59.390 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.402 7fc027985700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc02010bab0 0x7fc0201a0310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:28:59.390 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.402 7fc027985700 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc020103b60 con 0x7fc02010b660 2024-04-17T22:28:59.390 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.402 7fc027985700 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc02010bf00 con 0x7fc0200f5860 2024-04-17T22:28:59.390 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.402 7fc027985700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0200fa390 con 0x7fc02010bab0 2024-04-17T22:28:59.390 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.402 7fc01ffff700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fc0200f5860 0x7fc02010b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:28:59.391 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.402 7fc01ffff700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fc0200f5860 0x7fc02010b030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:43876/0 (socket says 172.21.15.102:43876) 2024-04-17T22:28:59.391 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.402 7fc01ffff700 1 -- 172.21.15.102:0/1800161507 learned_addr learned my addr 172.21.15.102:0/1800161507 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:28:59.391 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.402 7fc017fff700 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7fc02010b660 0x7fc02019fb10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:33738/0 (socket says 172.21.15.102:33738) 2024-04-17T22:28:59.391 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.402 7fc024f81700 1 --2- 172.21.15.102:0/1800161507 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc02010bab0 0x7fc0201a0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:28:59.391 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.402 7fc01ffff700 1 -- 172.21.15.102:0/1800161507 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc02010bab0 msgr2=0x7fc0201a0310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:28:59.391 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.402 7fc01ffff700 1 --2- 172.21.15.102:0/1800161507 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc02010bab0 0x7fc0201a0310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:28:59.391 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.402 7fc01ffff700 1 -- 172.21.15.102:0/1800161507 >> v1:172.21.15.102:6789/0 conn(0x7fc02010b660 legacy=0x7fc02019fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:28:59.391 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.402 7fc01ffff700 1 -- 172.21.15.102:0/1800161507 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0201a0940 con 0x7fc0200f5860 2024-04-17T22:28:59.391 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.402 7fc01f7fe700 1 -- 172.21.15.102:0/1800161507 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4156583409 0 0) 0x7fc008001110 con 0x7fc02010b660 2024-04-17T22:28:59.391 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.402 7fc01ffff700 1 --2- 172.21.15.102:0/1800161507 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc0200f5860 0x7fc02010b030 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7fc020103d80 tx=0x7fc01000bcb0).ready entity=mon.0 client_cookie=d0008c1767dbeb24 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:28:59.391 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.402 7fc01f7fe700 1 -- 172.21.15.102:0/1800161507 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc0201a0940 con 0x7fc0200f5860 2024-04-17T22:28:59.391 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.402 7fc01f7fe700 1 -- 172.21.15.102:0/1800161507 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc01000dbc0 con 0x7fc0200f5860 2024-04-17T22:28:59.391 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.402 7fc01f7fe700 1 -- 172.21.15.102:0/1800161507 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc010010470 con 0x7fc0200f5860 2024-04-17T22:28:59.391 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.402 7fc027985700 1 -- 172.21.15.102:0/1800161507 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc0200f5860 msgr2=0x7fc02010b030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:28:59.392 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.402 7fc027985700 1 --2- 172.21.15.102:0/1800161507 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc0200f5860 0x7fc02010b030 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7fc020103d80 tx=0x7fc01000bcb0).stop 2024-04-17T22:28:59.392 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.403 7fc027985700 1 -- 172.21.15.102:0/1800161507 shutdown_connections 2024-04-17T22:28:59.392 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.403 7fc027985700 1 --2- 172.21.15.102:0/1800161507 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc02010bab0 0x7fc0201a0310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:28:59.392 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.403 7fc027985700 1 --2- 172.21.15.102:0/1800161507 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc0200f5860 0x7fc02010b030 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:28:59.392 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.403 7fc027985700 1 -- 172.21.15.102:0/1800161507 >> 172.21.15.102:0/1800161507 conn(0x7fc0201138c0 msgr2=0x7fc0200f4f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:28:59.392 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.403 7fc027985700 1 -- 172.21.15.102:0/1800161507 shutdown_connections 2024-04-17T22:28:59.392 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.403 7fc027985700 1 -- 172.21.15.102:0/1800161507 wait complete. 2024-04-17T22:28:59.414 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.426 7fc027985700 1 Processor -- start 2024-04-17T22:28:59.415 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.426 7fc027985700 1 -- start start 2024-04-17T22:28:59.415 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.426 7fc027985700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fc0200c9b60 0x7fc0200d4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:28:59.415 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.426 7fc027985700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc0200d5490 0x7fc0201afbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:28:59.415 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.426 7fc027985700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc0200d58e0 0x7fc0201b0220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:28:59.415 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.426 7fc027985700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0200e2d30 con 0x7fc0200d58e0 2024-04-17T22:28:59.415 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.426 7fc027985700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0200c33f0 con 0x7fc0200d5490 2024-04-17T22:28:59.415 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.426 7fc027985700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc0200cbb00 con 0x7fc0200c9b60 2024-04-17T22:28:59.415 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.426 7fc017fff700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc0200d5490 0x7fc0201afbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:28:59.415 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.426 7fc01ffff700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fc0200c9b60 0x7fc0200d4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:28:59.415 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.426 7fc024f81700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc0200d58e0 0x7fc0201b0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:28:59.415 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.426 7fc017fff700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc0200d5490 0x7fc0201afbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:37728/0 (socket says 172.21.15.102:37728) 2024-04-17T22:28:59.415 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.426 7fc01ffff700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fc0200c9b60 0x7fc0200d4e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:52336/0 (socket says 172.21.15.102:52336) 2024-04-17T22:28:59.415 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.426 7fc017fff700 1 -- 172.21.15.102:0/4270530916 learned_addr learned my addr 172.21.15.102:0/4270530916 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:28:59.416 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.427 7fc017fff700 1 -- 172.21.15.102:0/4270530916 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fc0200c9b60 msgr2=0x7fc0200d4e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:28:59.416 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.427 7fc017fff700 1 --2- 172.21.15.102:0/4270530916 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fc0200c9b60 0x7fc0200d4e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:28:59.416 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.427 7fc017fff700 1 -- 172.21.15.102:0/4270530916 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc0200d58e0 msgr2=0x7fc0201b0220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:28:59.416 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.427 7fc017fff700 1 --2- 172.21.15.102:0/4270530916 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc0200d58e0 0x7fc0201b0220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:28:59.416 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.427 7fc017fff700 1 -- 172.21.15.102:0/4270530916 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0201b0850 con 0x7fc0200d5490 2024-04-17T22:28:59.416 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.427 7fc024f81700 1 --2- 172.21.15.102:0/4270530916 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc0200d58e0 0x7fc0201b0220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:28:59.416 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.427 7fc017fff700 1 --2- 172.21.15.102:0/4270530916 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc0200d5490 0x7fc0201afbf0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7fc0200d61a0 tx=0x7fc00800c890).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:28:59.416 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.427 7fc01dffb700 1 -- 172.21.15.102:0/4270530916 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc0201b0850 con 0x7fc0200d5490 2024-04-17T22:28:59.416 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.427 7fc01dffb700 1 -- 172.21.15.102:0/4270530916 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0080072e0 con 0x7fc0200d5490 2024-04-17T22:28:59.416 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.427 7fc01dffb700 1 -- 172.21.15.102:0/4270530916 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc008005750 con 0x7fc0200d5490 2024-04-17T22:28:59.416 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.427 7fc027985700 1 -- 172.21.15.102:0/4270530916 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0201b73f0 con 0x7fc0200d5490 2024-04-17T22:28:59.416 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.427 7fc027985700 1 -- 172.21.15.102:0/4270530916 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0201b77d0 con 0x7fc0200d5490 2024-04-17T22:28:59.416 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.427 7fc01dffb700 1 -- 172.21.15.102:0/4270530916 <== mon.1 v2:172.21.15.102:3301/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7fc00801b070 con 0x7fc0200d5490 2024-04-17T22:28:59.416 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.427 7fc01dffb700 1 -- 172.21.15.102:0/4270530916 <== mon.1 v2:172.21.15.102:3301/0 5 ==== osd_map(11..11 src has 1..11) v4 ==== 2374+0+0 (secure 0 0 0) 0x7fc008013450 con 0x7fc0200d5490 2024-04-17T22:28:59.417 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.428 7fc027985700 1 -- 172.21.15.102:0/4270530916 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0200f0820 con 0x7fc0200d5490 2024-04-17T22:28:59.418 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.431 7fc01dffb700 1 -- 172.21.15.102:0/4270530916 <== mon.1 v2:172.21.15.102:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+110455 (secure 0 0 0) 0x7fc0080070d0 con 0x7fc0200d5490 2024-04-17T22:28:59.522 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.535 7fc027985700 1 -- 172.21.15.102:0/4270530916 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc0201aa350 con 0x7fc0200d5490 2024-04-17T22:28:59.523 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.536 7fc01dffb700 1 -- 172.21.15.102:0/4270530916 <== mon.1 v2:172.21.15.102:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v11) v1 ==== 74+0+5917 (secure 0 0 0) 0x7fc008000c30 con 0x7fc0200d5490 2024-04-17T22:28:59.523 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-17T22:28:59.523 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":11,"fsid":"cb63e789-091d-49b1-b120-55f9db526871","created":"2024-04-17 22:28:48.209635","modified":"2024-04-17 22:28:59.242990","last_up_change":"2024-04-17 22:28:54.236215","last_in_change":"2024-04-17 22:28:54.236215","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":4,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-04-17 22:28:57.478545","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"3351ef85-ade4-4b90-a8f4-213f4e00ba01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":10,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6812","nonce":42955}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6813","nonce":42955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6815","nonce":42955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6814","nonce":42955}]},"public_addr":"172.21.15.102:6812/42955","cluster_addr":"172.21.15.102:6813/42955","heartbeat_back_addr":"172.21.15.102:6815/42955","heartbeat_front_addr":"172.21.15.102:6814/42955","state":["exists","up"]},{"osd":1,"uuid":"04f67eac-f36b-48f5-982d-4cfc776596d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":10,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6808","nonce":42958}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6809","nonce":42958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6811","nonce":42958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6810","nonce":42958}]},"public_addr":"172.21.15.102:6808/42958","cluster_addr":"172.21.15.102:6809/42958","heartbeat_back_addr":"172.21.15.102:6811/42958","heartbeat_front_addr":"172.21.15.102:6810/42958","state":["exists","up"]},{"osd":2,"uuid":"d524204b-70b2-40a8-8f73-f9e7d44318af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":10,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6804","nonce":42954}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6805","nonce":42954}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6807","nonce":42954}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6806","nonce":42954}]},"public_addr":"172.21.15.102:6804/42954","cluster_addr":"172.21.15.102:6805/42954","heartbeat_back_addr":"172.21.15.102:6807/42954","heartbeat_front_addr":"172.21.15.102:6806/42954","state":["exists","up"]},{"osd":3,"uuid":"4fc9d20c-1ad0-484f-aafa-2c9a7574245b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":10,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6800","nonce":42953}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6801","nonce":42953}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6803","nonce":42953}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6802","nonce":42953}]},"public_addr":"172.21.15.102:6800/42953","cluster_addr":"172.21.15.102:6801/42953","heartbeat_back_addr":"172.21.15.102:6803/42953","heartbeat_front_addr":"172.21.15.102:6802/42953","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2024-04-17T22:28:59.524 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.537 7fc027985700 1 -- 172.21.15.102:0/4270530916 >> v1:172.21.15.102:6816/42360 conn(0x7fc0040145a0 legacy=0x7fc004016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:28:59.524 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.537 7fc027985700 1 -- 172.21.15.102:0/4270530916 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc0200d5490 msgr2=0x7fc0201afbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:28:59.524 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.537 7fc027985700 1 --2- 172.21.15.102:0/4270530916 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc0200d5490 0x7fc0201afbf0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7fc0200d61a0 tx=0x7fc00800c890).stop 2024-04-17T22:28:59.524 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.537 7fc027985700 1 -- 172.21.15.102:0/4270530916 shutdown_connections 2024-04-17T22:28:59.524 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.537 7fc027985700 1 --2- 172.21.15.102:0/4270530916 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc0200d58e0 0x7fc0201b0220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:28:59.524 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.537 7fc027985700 1 --2- 172.21.15.102:0/4270530916 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fc0200c9b60 0x7fc0200d4e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:28:59.525 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.537 7fc027985700 1 --2- 172.21.15.102:0/4270530916 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc0200d5490 0x7fc0201afbf0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:28:59.525 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.537 7fc027985700 1 -- 172.21.15.102:0/4270530916 >> 172.21.15.102:0/4270530916 conn(0x7fc0200dff20 msgr2=0x7fc0200c1ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:28:59.525 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.537 7fc027985700 1 -- 172.21.15.102:0/4270530916 shutdown_connections 2024-04-17T22:28:59.525 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.537 7fc027985700 1 -- 172.21.15.102:0/4270530916 wait complete. 2024-04-17T22:28:59.535 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2024-04-17 22:28:57.478545', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '11', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2024-04-17T22:28:59.535 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2024-04-17T22:28:59.663 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.675 7ff060f88700 1 Processor -- start 2024-04-17T22:28:59.663 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.675 7ff060f88700 1 -- start start 2024-04-17T22:28:59.664 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.675 7ff060f88700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7ff05c10e780 0x7ff05c10ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:28:59.664 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.675 7ff060f88700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ff05c1a92b0 0x7ff05c1ab710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:28:59.664 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.675 7ff060f88700 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff05c1067d0 con 0x7ff05c0f0f20 2024-04-17T22:28:59.664 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.675 7ff060f88700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff05c106310 con 0x7ff05c1a92b0 2024-04-17T22:28:59.664 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.675 7ff060f88700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff05c0fb460 con 0x7ff05c10e780 2024-04-17T22:28:59.664 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.675 7ff059d77700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7ff05c10e780 0x7ff05c10ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:28:59.664 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.675 7ff059576700 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7ff05c0f0f20 0x7ff05c0f1370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:33742/0 (socket says 172.21.15.102:33742) 2024-04-17T22:28:59.664 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.675 7ff05a578700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ff05c1a92b0 0x7ff05c1ab710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:28:59.664 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.675 7ff059576700 1 -- 172.21.15.102:0/780312189 learned_addr learned my addr 172.21.15.102:0/780312189 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-17T22:28:59.664 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.676 7ff059d77700 1 -- 172.21.15.102:0/780312189 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ff05c1a92b0 msgr2=0x7ff05c1ab710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:28:59.664 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.675 7ff058d75700 1 -- 172.21.15.102:0/780312189 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1948290287 0 0) 0x7ff04c001110 con 0x7ff05c0f0f20 2024-04-17T22:28:59.664 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.676 7ff059d77700 1 --2- 172.21.15.102:0/780312189 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ff05c1a92b0 0x7ff05c1ab710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:28:59.664 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.676 7ff059d77700 1 -- 172.21.15.102:0/780312189 >> v1:172.21.15.102:6789/0 conn(0x7ff05c0f0f20 legacy=0x7ff05c0f1370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:28:59.665 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.676 7ff059d77700 1 -- 172.21.15.102:0/780312189 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff05c111ce0 con 0x7ff05c10e780 2024-04-17T22:28:59.665 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.676 7ff05a578700 1 --2- 172.21.15.102:0/780312189 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ff05c1a92b0 0x7ff05c1ab710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:28:59.665 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.676 7ff059d77700 1 --2- 172.21.15.102:0/780312189 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7ff05c10e780 0x7ff05c10ebd0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 rx=0x7ff05c0f56f0 tx=0x7ff04400b9d0).ready entity=mon.2 client_cookie=e54fe873ad10093d server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:28:59.665 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.676 7ff058d75700 1 -- 172.21.15.102:0/780312189 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff05c111ce0 con 0x7ff05c10e780 2024-04-17T22:28:59.665 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.676 7ff058d75700 1 -- 172.21.15.102:0/780312189 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff04400d570 con 0x7ff05c10e780 2024-04-17T22:28:59.665 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.676 7ff058d75700 1 -- 172.21.15.102:0/780312189 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff04400d940 con 0x7ff05c10e780 2024-04-17T22:28:59.665 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.676 7ff060f88700 1 -- 172.21.15.102:0/780312189 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7ff05c10e780 msgr2=0x7ff05c10ebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:28:59.665 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.676 7ff060f88700 1 --2- 172.21.15.102:0/780312189 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7ff05c10e780 0x7ff05c10ebd0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 rx=0x7ff05c0f56f0 tx=0x7ff04400b9d0).stop 2024-04-17T22:28:59.665 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.676 7ff060f88700 1 -- 172.21.15.102:0/780312189 shutdown_connections 2024-04-17T22:28:59.665 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.676 7ff060f88700 1 --2- 172.21.15.102:0/780312189 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ff05c1a92b0 0x7ff05c1ab710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:28:59.665 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.676 7ff060f88700 1 --2- 172.21.15.102:0/780312189 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7ff05c10e780 0x7ff05c10ebd0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:28:59.665 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.676 7ff060f88700 1 -- 172.21.15.102:0/780312189 >> 172.21.15.102:0/780312189 conn(0x7ff05c114760 msgr2=0x7ff05c114bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:28:59.665 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.676 7ff060f88700 1 -- 172.21.15.102:0/780312189 shutdown_connections 2024-04-17T22:28:59.665 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.677 7ff060f88700 1 -- 172.21.15.102:0/780312189 wait complete. 2024-04-17T22:28:59.688 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.699 7ff060f88700 1 Processor -- start 2024-04-17T22:28:59.688 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.699 7ff060f88700 1 -- start start 2024-04-17T22:28:59.688 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.699 7ff060f88700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ff05c0da240 0x7ff05c0c7290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:28:59.688 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.699 7ff060f88700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff05c0c78c0 0x7ff05c0d6130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:28:59.688 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.699 7ff060f88700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7ff05c0c7d10 0x7ff05c0d6760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:28:59.688 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.699 7ff060f88700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff05c0c8370 con 0x7ff05c0c78c0 2024-04-17T22:28:59.688 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.699 7ff060f88700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff05c0da870 con 0x7ff05c0da240 2024-04-17T22:28:59.688 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.699 7ff060f88700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff05c0d6d90 con 0x7ff05c0c7d10 2024-04-17T22:28:59.688 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.699 7ff059d77700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ff05c0da240 0x7ff05c0c7290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:28:59.688 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.699 7ff059576700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff05c0c78c0 0x7ff05c0d6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:28:59.688 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.699 7ff059d77700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ff05c0da240 0x7ff05c0c7290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:37732/0 (socket says 172.21.15.102:37732) 2024-04-17T22:28:59.689 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.699 7ff059d77700 1 -- 172.21.15.102:0/3635302005 learned_addr learned my addr 172.21.15.102:0/3635302005 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:28:59.689 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.699 7ff05a578700 1 --2- 172.21.15.102:0/3635302005 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7ff05c0c7d10 0x7ff05c0d6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:28:59.689 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.700 7ff059d77700 1 -- 172.21.15.102:0/3635302005 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7ff05c0c7d10 msgr2=0x7ff05c0d6760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:28:59.689 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.700 7ff059d77700 1 --2- 172.21.15.102:0/3635302005 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7ff05c0c7d10 0x7ff05c0d6760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:28:59.689 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.700 7ff059d77700 1 -- 172.21.15.102:0/3635302005 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff05c0c78c0 msgr2=0x7ff05c0d6130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:28:59.689 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.700 7ff059d77700 1 --2- 172.21.15.102:0/3635302005 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff05c0c78c0 0x7ff05c0d6130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:28:59.689 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.700 7ff059d77700 1 -- 172.21.15.102:0/3635302005 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff05c1bb4b0 con 0x7ff05c0da240 2024-04-17T22:28:59.689 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.700 7ff059576700 1 --2- 172.21.15.102:0/3635302005 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff05c0c78c0 0x7ff05c0d6130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:28:59.689 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.700 7ff059d77700 1 --2- 172.21.15.102:0/3635302005 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ff05c0da240 0x7ff05c0c7290 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7ff05c0d7670 tx=0x7ff0440068c0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:28:59.689 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.700 7ff052ffd700 1 -- 172.21.15.102:0/3635302005 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff05c1bb4b0 con 0x7ff05c0da240 2024-04-17T22:28:59.689 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.700 7ff052ffd700 1 -- 172.21.15.102:0/3635302005 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0440119a0 con 0x7ff05c0da240 2024-04-17T22:28:59.689 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.700 7ff052ffd700 1 -- 172.21.15.102:0/3635302005 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff044022020 con 0x7ff05c0da240 2024-04-17T22:28:59.689 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.700 7ff060f88700 1 -- 172.21.15.102:0/3635302005 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff05c1bb6b0 con 0x7ff05c0da240 2024-04-17T22:28:59.689 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.700 7ff060f88700 1 -- 172.21.15.102:0/3635302005 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff05c1bba90 con 0x7ff05c0da240 2024-04-17T22:28:59.689 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.700 7ff052ffd700 1 -- 172.21.15.102:0/3635302005 <== mon.1 v2:172.21.15.102:3301/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7ff04401d450 con 0x7ff05c0da240 2024-04-17T22:28:59.690 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.701 7ff052ffd700 1 -- 172.21.15.102:0/3635302005 <== mon.1 v2:172.21.15.102:3301/0 5 ==== osd_map(11..11 src has 1..11) v4 ==== 2374+0+0 (secure 0 0 0) 0x7ff044020af0 con 0x7ff05c0da240 2024-04-17T22:28:59.690 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.701 7ff060f88700 1 -- 172.21.15.102:0/3635302005 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff05c0d6fc0 con 0x7ff05c0da240 2024-04-17T22:28:59.691 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.704 7ff052ffd700 1 -- 172.21.15.102:0/3635302005 <== mon.1 v2:172.21.15.102:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+110455 (secure 0 0 0) 0x7ff044011790 con 0x7ff05c0da240 2024-04-17T22:28:59.796 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.809 7ff060f88700 1 -- 172.21.15.102:0/3635302005 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7ff05c0f0800 con 0x7ff05c0da240 2024-04-17T22:28:59.796 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.809 7ff052ffd700 1 -- 172.21.15.102:0/3635302005 <== mon.1 v2:172.21.15.102:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v11) v1 ==== 92+0+10 (secure 0 0 0) 0x7ff04400d360 con 0x7ff05c0da240 2024-04-17T22:28:59.796 INFO:teuthology.orchestra.run.smithi102.stdout:pg_num: 8 2024-04-17T22:28:59.797 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.810 7ff060f88700 1 -- 172.21.15.102:0/3635302005 >> v1:172.21.15.102:6816/42360 conn(0x7ff0400145a0 legacy=0x7ff040016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:28:59.798 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.810 7ff060f88700 1 -- 172.21.15.102:0/3635302005 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ff05c0da240 msgr2=0x7ff05c0c7290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:28:59.798 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.810 7ff060f88700 1 --2- 172.21.15.102:0/3635302005 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ff05c0da240 0x7ff05c0c7290 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7ff05c0d7670 tx=0x7ff0440068c0).stop 2024-04-17T22:28:59.798 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.810 7ff060f88700 1 -- 172.21.15.102:0/3635302005 shutdown_connections 2024-04-17T22:28:59.798 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.810 7ff060f88700 1 --2- 172.21.15.102:0/3635302005 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7ff05c0c7d10 0x7ff05c0d6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:28:59.798 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.810 7ff060f88700 1 --2- 172.21.15.102:0/3635302005 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ff05c0da240 0x7ff05c0c7290 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:28:59.798 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.811 7ff060f88700 1 --2- 172.21.15.102:0/3635302005 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff05c0c78c0 0x7ff05c0d6130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:28:59.798 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.811 7ff060f88700 1 -- 172.21.15.102:0/3635302005 >> 172.21.15.102:0/3635302005 conn(0x7ff05c0c6a60 msgr2=0x7ff05c0c30d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:28:59.798 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.811 7ff060f88700 1 -- 172.21.15.102:0/3635302005 shutdown_connections 2024-04-17T22:28:59.798 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.811 7ff060f88700 1 -- 172.21.15.102:0/3635302005 wait complete. 2024-04-17T22:28:59.808 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T22:28:59.935 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.946 7f6e545f9700 1 Processor -- start 2024-04-17T22:28:59.935 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.946 7f6e545f9700 1 -- start start 2024-04-17T22:28:59.935 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.946 7f6e545f9700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f6e4c0f57d0 0x7f6e4c10afa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:28:59.935 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.946 7f6e545f9700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6e4c10ba20 0x7f6e4c1a0280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:28:59.935 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.946 7f6e545f9700 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e4c103ad0 con 0x7f6e4c10b5d0 2024-04-17T22:28:59.935 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.946 7f6e545f9700 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e4c10be70 con 0x7f6e4c0f57d0 2024-04-17T22:28:59.935 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.946 7f6e545f9700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e4c0fa300 con 0x7f6e4c10ba20 2024-04-17T22:28:59.935 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.946 7f6e51bf5700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6e4c10ba20 0x7f6e4c1a0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:28:59.935 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.946 7f6e51bf5700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6e4c10ba20 0x7f6e4c1a0280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:37738/0 (socket says 172.21.15.102:37738) 2024-04-17T22:28:59.935 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.946 7f6e51bf5700 1 -- 172.21.15.102:0/2257094114 learned_addr learned my addr 172.21.15.102:0/2257094114 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:28:59.936 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.946 7f6e513f4700 1 --2- 172.21.15.102:0/2257094114 >> v2:172.21.15.102:3300/0 conn(0x7f6e4c0f57d0 0x7f6e4c10afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:28:59.936 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.947 7f6e51bf5700 1 -- 172.21.15.102:0/2257094114 >> v2:172.21.15.102:3300/0 conn(0x7f6e4c0f57d0 msgr2=0x7f6e4c10afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:28:59.936 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.947 7f6e51bf5700 1 --2- 172.21.15.102:0/2257094114 >> v2:172.21.15.102:3300/0 conn(0x7f6e4c0f57d0 0x7f6e4c10afa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:28:59.936 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.947 7f6e51bf5700 1 -- 172.21.15.102:0/2257094114 >> v1:172.21.15.102:6789/0 conn(0x7f6e4c10b5d0 legacy=0x7f6e4c19fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:28:59.936 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.947 7f6e51bf5700 1 -- 172.21.15.102:0/2257094114 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e4c1a08b0 con 0x7f6e4c10ba20 2024-04-17T22:28:59.936 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.947 7f6e51bf5700 1 --2- 172.21.15.102:0/2257094114 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6e4c10ba20 0x7f6e4c1a0280 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7f6e4c0f4400 tx=0x7f6e4400d5d0).ready entity=mon.1 client_cookie=ef3af7495399df46 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:28:59.936 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.947 7f6e3ffff700 1 -- 172.21.15.102:0/2257094114 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f6e4c1a08b0 con 0x7f6e4c10ba20 2024-04-17T22:28:59.936 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.947 7f6e3ffff700 1 -- 172.21.15.102:0/2257094114 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e44010660 con 0x7f6e4c10ba20 2024-04-17T22:28:59.936 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.947 7f6e3ffff700 1 -- 172.21.15.102:0/2257094114 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f6e44010a30 con 0x7f6e4c10ba20 2024-04-17T22:28:59.936 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.947 7f6e545f9700 1 -- 172.21.15.102:0/2257094114 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6e4c10ba20 msgr2=0x7f6e4c1a0280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:28:59.936 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.947 7f6e545f9700 1 --2- 172.21.15.102:0/2257094114 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6e4c10ba20 0x7f6e4c1a0280 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7f6e4c0f4400 tx=0x7f6e4400d5d0).stop 2024-04-17T22:28:59.936 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.948 7f6e545f9700 1 -- 172.21.15.102:0/2257094114 shutdown_connections 2024-04-17T22:28:59.936 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.948 7f6e545f9700 1 --2- 172.21.15.102:0/2257094114 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6e4c10ba20 0x7f6e4c1a0280 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:28:59.936 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.948 7f6e545f9700 1 --2- 172.21.15.102:0/2257094114 >> v2:172.21.15.102:3300/0 conn(0x7f6e4c0f57d0 0x7f6e4c10afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:28:59.937 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.948 7f6e545f9700 1 -- 172.21.15.102:0/2257094114 >> 172.21.15.102:0/2257094114 conn(0x7f6e4c113830 msgr2=0x7f6e4c0f4ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:28:59.937 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.948 7f6e545f9700 1 -- 172.21.15.102:0/2257094114 shutdown_connections 2024-04-17T22:28:59.937 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.948 7f6e545f9700 1 -- 172.21.15.102:0/2257094114 wait complete. 2024-04-17T22:28:59.959 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.970 7f6e545f9700 1 Processor -- start 2024-04-17T22:28:59.959 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.970 7f6e545f9700 1 -- start start 2024-04-17T22:28:59.959 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.971 7f6e545f9700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6e4c1ad6b0 0x7f6e4c1adb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:28:59.960 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.971 7f6e545f9700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f6e4c1ae130 0x7f6e4c0d9e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:28:59.960 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.971 7f6e545f9700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6e4c0da4b0 0x7f6e4c1bb860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:28:59.960 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.971 7f6e545f9700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e4c0e2d40 con 0x7f6e4c0da4b0 2024-04-17T22:28:59.960 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.971 7f6e545f9700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e4c0c3230 con 0x7f6e4c1ad6b0 2024-04-17T22:28:59.960 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.971 7f6e545f9700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6e4c0d4e20 con 0x7f6e4c1ae130 2024-04-17T22:28:59.960 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.971 7f6e513f4700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6e4c1ad6b0 0x7f6e4c1adb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:28:59.960 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.971 7f6e513f4700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6e4c1ad6b0 0x7f6e4c1adb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:37744/0 (socket says 172.21.15.102:37744) 2024-04-17T22:28:59.960 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.971 7f6e513f4700 1 -- 172.21.15.102:0/4175236081 learned_addr learned my addr 172.21.15.102:0/4175236081 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:28:59.960 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.971 7f6e51bf5700 1 --2- 172.21.15.102:0/4175236081 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6e4c0da4b0 0x7f6e4c1bb860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:28:59.960 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.971 7f6e50bf3700 1 --2- 172.21.15.102:0/4175236081 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f6e4c1ae130 0x7f6e4c0d9e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:28:59.960 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.971 7f6e51bf5700 1 -- 172.21.15.102:0/4175236081 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f6e4c1ae130 msgr2=0x7f6e4c0d9e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:28:59.960 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.971 7f6e51bf5700 1 --2- 172.21.15.102:0/4175236081 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f6e4c1ae130 0x7f6e4c0d9e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:28:59.960 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.971 7f6e51bf5700 1 -- 172.21.15.102:0/4175236081 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6e4c1ad6b0 msgr2=0x7f6e4c1adb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:28:59.960 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.971 7f6e51bf5700 1 --2- 172.21.15.102:0/4175236081 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6e4c1ad6b0 0x7f6e4c1adb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:28:59.960 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.971 7f6e51bf5700 1 -- 172.21.15.102:0/4175236081 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e4c0d4740 con 0x7f6e4c0da4b0 2024-04-17T22:28:59.961 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.971 7f6e513f4700 1 --2- 172.21.15.102:0/4175236081 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6e4c1ad6b0 0x7f6e4c1adb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:28:59.961 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.971 7f6e51bf5700 1 --2- 172.21.15.102:0/4175236081 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6e4c0da4b0 0x7f6e4c1bb860 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7f6e4c0c3a70 tx=0x7f6e44006c40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:28:59.961 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.971 7f6e3e7fc700 1 -- 172.21.15.102:0/4175236081 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f6e4c0d4740 con 0x7f6e4c0da4b0 2024-04-17T22:28:59.961 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.972 7f6e3e7fc700 1 -- 172.21.15.102:0/4175236081 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e44018800 con 0x7f6e4c0da4b0 2024-04-17T22:28:59.961 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.972 7f6e545f9700 1 -- 172.21.15.102:0/4175236081 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e4c0d56c0 con 0x7f6e4c0da4b0 2024-04-17T22:28:59.961 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.972 7f6e3e7fc700 1 -- 172.21.15.102:0/4175236081 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f6e44018bd0 con 0x7f6e4c0da4b0 2024-04-17T22:28:59.961 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.972 7f6e545f9700 1 -- 172.21.15.102:0/4175236081 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e4c0daae0 con 0x7f6e4c0da4b0 2024-04-17T22:28:59.961 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.972 7f6e3e7fc700 1 -- 172.21.15.102:0/4175236081 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7f6e4400f3d0 con 0x7f6e4c0da4b0 2024-04-17T22:28:59.961 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.972 7f6e3e7fc700 1 -- 172.21.15.102:0/4175236081 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(11..11 src has 1..11) v4 ==== 2374+0+0 (secure 0 0 0) 0x7f6e44007a00 con 0x7f6e4c0da4b0 2024-04-17T22:28:59.961 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.973 7f6e545f9700 1 -- 172.21.15.102:0/4175236081 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e4c0f0790 con 0x7f6e4c0da4b0 2024-04-17T22:28:59.962 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:28:59.976 7f6e3e7fc700 1 -- 172.21.15.102:0/4175236081 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+110455 (secure 0 0 0) 0x7f6e4400dc60 con 0x7f6e4c0da4b0 2024-04-17T22:29:00.098 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:00.111 7f6e545f9700 1 -- 172.21.15.102:0/4175236081 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6e4c0d5af0 con 0x7f6e4c0da4b0 2024-04-17T22:29:00.098 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:00.111 7f6e3e7fc700 1 -- 172.21.15.102:0/4175236081 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) v1 ==== 93+0+503 (secure 0 0 0) 0x7f6e44018400 con 0x7f6e4c0da4b0 2024-04-17T22:29:00.098 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-17T22:29:00.098 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[],"filesystems":[]} 2024-04-17T22:29:00.099 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:00.112 7f6e545f9700 1 -- 172.21.15.102:0/4175236081 >> v1:172.21.15.102:6816/42360 conn(0x7f6e2c0145a0 legacy=0x7f6e2c016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:00.100 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:00.112 7f6e545f9700 1 -- 172.21.15.102:0/4175236081 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6e4c0da4b0 msgr2=0x7f6e4c1bb860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:00.100 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:00.112 7f6e545f9700 1 --2- 172.21.15.102:0/4175236081 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6e4c0da4b0 0x7f6e4c1bb860 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7f6e4c0c3a70 tx=0x7f6e44006c40).stop 2024-04-17T22:29:00.100 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:00.112 7f6e545f9700 1 -- 172.21.15.102:0/4175236081 shutdown_connections 2024-04-17T22:29:00.100 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:00.112 7f6e545f9700 1 --2- 172.21.15.102:0/4175236081 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6e4c0da4b0 0x7f6e4c1bb860 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:00.100 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:00.112 7f6e545f9700 1 --2- 172.21.15.102:0/4175236081 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6e4c1ad6b0 0x7f6e4c1adb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:00.100 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:00.112 7f6e545f9700 1 --2- 172.21.15.102:0/4175236081 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f6e4c1ae130 0x7f6e4c0d9e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:00.100 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:00.112 7f6e545f9700 1 -- 172.21.15.102:0/4175236081 >> 172.21.15.102:0/4175236081 conn(0x7f6e4c0e0de0 msgr2=0x7f6e4c0c92b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:29:00.100 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:00.112 7f6e545f9700 1 -- 172.21.15.102:0/4175236081 shutdown_connections 2024-04-17T22:29:00.100 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:00.112 7f6e545f9700 1 -- 172.21.15.102:0/4175236081 wait complete. 2024-04-17T22:29:00.100 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 1 2024-04-17T22:29:02.110 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T22:29:02.248 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:02.259 7f76ac5a8700 1 Processor -- start 2024-04-17T22:29:02.249 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:02.260 7f76ac5a8700 1 -- start start 2024-04-17T22:29:02.249 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:02.260 7f76ac5a8700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f76a40f5860 0x7f76a410b030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:02.249 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:02.260 7f76ac5a8700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f76a410b660 0x7f76a419fb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:02.249 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:02.260 7f76ac5a8700 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76a4103b60 con 0x7f76a410bab0 2024-04-17T22:29:02.249 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:02.260 7f76ac5a8700 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76a410bf00 con 0x7f76a410b660 2024-04-17T22:29:02.249 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:02.260 7f76ac5a8700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76a40fa390 con 0x7f76a40f5860 2024-04-17T22:29:02.249 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:02.260 7f76a8ba2700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f76a410b660 0x7f76a419fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:02.249 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:02.260 7f76a8ba2700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f76a410b660 0x7f76a419fb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:43932/0 (socket says 172.21.15.102:43932) 2024-04-17T22:29:02.249 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:02.260 7f76a93a3700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f76a40f5860 0x7f76a410b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:02.249 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:02.260 7f76a8ba2700 1 -- 172.21.15.102:0/3257778930 learned_addr learned my addr 172.21.15.102:0/3257778930 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:29:02.249 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:02.260 7f76a8ba2700 1 -- 172.21.15.102:0/3257778930 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f76a40f5860 msgr2=0x7f76a410b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:29:02.249 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:02.260 7f76a8ba2700 1 --2- 172.21.15.102:0/3257778930 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f76a40f5860 0x7f76a410b030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:02.249 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:02.260 7f76a8ba2700 1 -- 172.21.15.102:0/3257778930 >> v1:172.21.15.102:6789/0 conn(0x7f76a410bab0 legacy=0x7f76a41a0140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:02.250 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:02.260 7f76a8ba2700 1 -- 172.21.15.102:0/3257778930 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76a41a0940 con 0x7f76a410b660 2024-04-17T22:29:02.250 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:02.260 7f7693fff700 1 -- 172.21.15.102:0/3257778930 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1544779363 0 0) 0x7f769c001490 con 0x7f76a410bab0 2024-04-17T22:29:02.250 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:02.260 7f76a93a3700 1 --2- 172.21.15.102:0/3257778930 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f76a40f5860 0x7f76a410b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:29:02.250 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:02.260 7f76a8ba2700 1 --2- 172.21.15.102:0/3257778930 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f76a410b660 0x7f76a419fb10 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7f76a0002f70 tx=0x7f76a0009a50).ready entity=mon.0 client_cookie=bb6eca785fa4fdc8 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:29:02.250 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:02.261 7f7693fff700 1 -- 172.21.15.102:0/3257778930 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f76a41a0940 con 0x7f76a410b660 2024-04-17T22:29:02.250 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:02.261 7f7693fff700 1 -- 172.21.15.102:0/3257778930 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76a000f510 con 0x7f76a410b660 2024-04-17T22:29:02.250 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:02.261 7f7693fff700 1 -- 172.21.15.102:0/3257778930 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f76a000f6f0 con 0x7f76a410b660 2024-04-17T22:29:02.250 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:02.261 7f76ac5a8700 1 -- 172.21.15.102:0/3257778930 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f76a410b660 msgr2=0x7f76a419fb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:02.250 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:02.261 7f76ac5a8700 1 --2- 172.21.15.102:0/3257778930 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f76a410b660 0x7f76a419fb10 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7f76a0002f70 tx=0x7f76a0009a50).stop 2024-04-17T22:29:02.250 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:02.261 7f76ac5a8700 1 -- 172.21.15.102:0/3257778930 shutdown_connections 2024-04-17T22:29:02.250 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:02.261 7f76ac5a8700 1 --2- 172.21.15.102:0/3257778930 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f76a40f5860 0x7f76a410b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:02.250 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:02.261 7f76ac5a8700 1 --2- 172.21.15.102:0/3257778930 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f76a410b660 0x7f76a419fb10 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:02.251 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:02.261 7f76ac5a8700 1 -- 172.21.15.102:0/3257778930 >> 172.21.15.102:0/3257778930 conn(0x7f76a41138c0 msgr2=0x7f76a40f4f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:29:02.251 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:02.261 7f76ac5a8700 1 -- 172.21.15.102:0/3257778930 shutdown_connections 2024-04-17T22:29:02.251 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:02.261 7f76ac5a8700 1 -- 172.21.15.102:0/3257778930 wait complete. 2024-04-17T22:29:02.272 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:02.284 7f76ac5a8700 1 Processor -- start 2024-04-17T22:29:02.273 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:02.284 7f76ac5a8700 1 -- start start 2024-04-17T22:29:02.273 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:02.284 7f76ac5a8700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f76a40e01c0 0x7f76a40cd6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:02.273 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:02.285 7f76ac5a8700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f76a40cdd00 0x7f76a41b3f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:02.273 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:02.285 7f76ac5a8700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f76a41b4550 0x7f76a41b49a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:02.273 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:02.285 7f76ac5a8700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76a41b4fd0 con 0x7f76a41b4550 2024-04-17T22:29:02.273 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:02.285 7f76ac5a8700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76a40c2350 con 0x7f76a40cdd00 2024-04-17T22:29:02.273 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:02.285 7f76ac5a8700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f76a40d4490 con 0x7f76a40e01c0 2024-04-17T22:29:02.273 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:02.285 7f76a8ba2700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f76a40cdd00 0x7f76a41b3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:02.273 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:02.285 7f76a9ba4700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f76a41b4550 0x7f76a41b49a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:02.273 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:02.285 7f76a93a3700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f76a40e01c0 0x7f76a40cd6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:02.273 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:02.285 7f76a9ba4700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f76a41b4550 0x7f76a41b49a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:43942/0 (socket says 172.21.15.102:43942) 2024-04-17T22:29:02.273 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:02.285 7f76a8ba2700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f76a40cdd00 0x7f76a41b3f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:37758/0 (socket says 172.21.15.102:37758) 2024-04-17T22:29:02.273 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:02.285 7f76a9ba4700 1 -- 172.21.15.102:0/1570201506 learned_addr learned my addr 172.21.15.102:0/1570201506 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:29:02.274 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:02.285 7f76a9ba4700 1 -- 172.21.15.102:0/1570201506 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f76a40e01c0 msgr2=0x7f76a40cd6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:02.274 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:02.285 7f76a9ba4700 1 --2- 172.21.15.102:0/1570201506 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f76a40e01c0 0x7f76a40cd6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:02.274 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:02.285 7f76a9ba4700 1 -- 172.21.15.102:0/1570201506 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f76a40cdd00 msgr2=0x7f76a41b3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:02.274 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:02.285 7f76a9ba4700 1 --2- 172.21.15.102:0/1570201506 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f76a40cdd00 0x7f76a41b3f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:02.274 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:02.285 7f76a9ba4700 1 -- 172.21.15.102:0/1570201506 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76a41bb400 con 0x7f76a41b4550 2024-04-17T22:29:02.274 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:02.285 7f76a93a3700 1 --2- 172.21.15.102:0/1570201506 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f76a40e01c0 0x7f76a40cd6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:29:02.274 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:02.285 7f76a8ba2700 1 --2- 172.21.15.102:0/1570201506 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f76a40cdd00 0x7f76a41b3f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:29:02.274 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:02.285 7f76a9ba4700 1 --2- 172.21.15.102:0/1570201506 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f76a41b4550 0x7f76a41b49a0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7f769c001250 tx=0x7f769c00cbe0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:29:02.274 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:02.285 7f76927fc700 1 -- 172.21.15.102:0/1570201506 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f76a41bb400 con 0x7f76a41b4550 2024-04-17T22:29:02.274 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:02.285 7f76927fc700 1 -- 172.21.15.102:0/1570201506 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f769c006ab0 con 0x7f76a41b4550 2024-04-17T22:29:02.274 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:02.285 7f76ac5a8700 1 -- 172.21.15.102:0/1570201506 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76a41bb600 con 0x7f76a41b4550 2024-04-17T22:29:02.274 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:02.285 7f76927fc700 1 -- 172.21.15.102:0/1570201506 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f769c007030 con 0x7f76a41b4550 2024-04-17T22:29:02.274 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:02.286 7f76ac5a8700 1 -- 172.21.15.102:0/1570201506 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76a41bb9e0 con 0x7f76a41b4550 2024-04-17T22:29:02.275 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:02.286 7f76927fc700 1 -- 172.21.15.102:0/1570201506 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7f769c007230 con 0x7f76a41b4550 2024-04-17T22:29:02.275 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:02.286 7f76927fc700 1 -- 172.21.15.102:0/1570201506 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(12..12 src has 1..12) v4 ==== 2374+0+0 (secure 0 0 0) 0x7f769c01ad50 con 0x7f76a41b4550 2024-04-17T22:29:02.275 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:02.286 7f76ac5a8700 1 -- 172.21.15.102:0/1570201506 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76a40f0820 con 0x7f76a41b4550 2024-04-17T22:29:02.276 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:02.290 7f76927fc700 1 -- 172.21.15.102:0/1570201506 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+110455 (secure 0 0 0) 0x7f769c0068a0 con 0x7f76a41b4550 2024-04-17T22:29:02.413 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:02.426 7f76ac5a8700 1 -- 172.21.15.102:0/1570201506 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f76a41ae680 con 0x7f76a41b4550 2024-04-17T22:29:02.413 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:02.426 7f76927fc700 1 -- 172.21.15.102:0/1570201506 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) v1 ==== 93+0+503 (secure 0 0 0) 0x7f76a4114440 con 0x7f76a41b4550 2024-04-17T22:29:02.413 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-17T22:29:02.413 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[],"filesystems":[]} 2024-04-17T22:29:02.415 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:02.427 7f76ac5a8700 1 -- 172.21.15.102:0/1570201506 >> v1:172.21.15.102:6816/42360 conn(0x7f76840145a0 legacy=0x7f7684016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:02.415 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:02.427 7f76ac5a8700 1 -- 172.21.15.102:0/1570201506 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f76a41b4550 msgr2=0x7f76a41b49a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:02.415 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:02.427 7f76ac5a8700 1 --2- 172.21.15.102:0/1570201506 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f76a41b4550 0x7f76a41b49a0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7f769c001250 tx=0x7f769c00cbe0).stop 2024-04-17T22:29:02.415 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:02.428 7f76ac5a8700 1 -- 172.21.15.102:0/1570201506 shutdown_connections 2024-04-17T22:29:02.415 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:02.428 7f76ac5a8700 1 --2- 172.21.15.102:0/1570201506 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f76a41b4550 0x7f76a41b49a0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:02.415 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:02.428 7f76ac5a8700 1 --2- 172.21.15.102:0/1570201506 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f76a40e01c0 0x7f76a40cd6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:02.415 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:02.428 7f76ac5a8700 1 --2- 172.21.15.102:0/1570201506 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f76a40cdd00 0x7f76a41b3f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:02.415 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:02.428 7f76ac5a8700 1 -- 172.21.15.102:0/1570201506 >> 172.21.15.102:0/1570201506 conn(0x7f76a40c58f0 msgr2=0x7f76a40bc780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:29:02.415 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:02.428 7f76ac5a8700 1 -- 172.21.15.102:0/1570201506 shutdown_connections 2024-04-17T22:29:02.415 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:02.428 7f76ac5a8700 1 -- 172.21.15.102:0/1570201506 wait complete. 2024-04-17T22:29:02.415 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 1 2024-04-17T22:29:04.426 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T22:29:04.568 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:04.580 7f898361d700 1 Processor -- start 2024-04-17T22:29:04.569 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:04.580 7f898361d700 1 -- start start 2024-04-17T22:29:04.569 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:04.580 7f898361d700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f897c1a64d0 0x7f897c1a8930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:04.569 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:04.580 7f898361d700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f897c1a8f60 0x7f897c1ab3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:04.569 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:04.580 7f898361d700 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f897c10eeb0 con 0x7f897c10e260 2024-04-17T22:29:04.569 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:04.580 7f898361d700 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f897c0f9f30 con 0x7f897c1a64d0 2024-04-17T22:29:04.569 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:04.580 7f898361d700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f897c0f9050 con 0x7f897c1a8f60 2024-04-17T22:29:04.569 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:04.580 7f8980c19700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f897c1a8f60 0x7f897c1ab3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:04.569 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:04.580 7f897b7fe700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f897c1a64d0 0x7f897c1a8930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:04.569 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:04.580 7f8980c19700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f897c1a8f60 0x7f897c1ab3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:40830/0 (socket says 172.21.15.102:40830) 2024-04-17T22:29:04.569 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:04.580 7f897bfff700 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f897c10e260 0x7f897c10e6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:46390/0 (socket says 172.21.15.102:46390) 2024-04-17T22:29:04.569 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:04.580 7f8980c19700 1 -- 172.21.15.102:0/573858598 learned_addr learned my addr 172.21.15.102:0/573858598 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:29:04.569 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:04.580 7f897b7fe700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f897c1a64d0 0x7f897c1a8930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:49058/0 (socket says 172.21.15.102:49058) 2024-04-17T22:29:04.570 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:04.581 7f8980c19700 1 -- 172.21.15.102:0/573858598 >> v2:172.21.15.102:3300/0 conn(0x7f897c1a64d0 msgr2=0x7f897c1a8930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:29:04.570 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:04.581 7f8980c19700 1 --2- 172.21.15.102:0/573858598 >> v2:172.21.15.102:3300/0 conn(0x7f897c1a64d0 0x7f897c1a8930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:04.570 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:04.581 7f8980c19700 1 -- 172.21.15.102:0/573858598 >> v1:172.21.15.102:6789/0 conn(0x7f897c10e260 legacy=0x7f897c10e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:04.570 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:04.581 7f8980c19700 1 -- 172.21.15.102:0/573858598 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f897c1aca00 con 0x7f897c1a8f60 2024-04-17T22:29:04.570 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:04.581 7f897affd700 1 -- 172.21.15.102:0/573858598 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1975305034 0 0) 0x7f8970001110 con 0x7f897c10e260 2024-04-17T22:29:04.570 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:04.581 7f897b7fe700 1 --2- 172.21.15.102:0/573858598 >> v2:172.21.15.102:3300/0 conn(0x7f897c1a64d0 0x7f897c1a8930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:29:04.570 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:04.581 7f8980c19700 1 --2- 172.21.15.102:0/573858598 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f897c1a8f60 0x7f897c1ab3c0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 rx=0x7f897c0f4490 tx=0x7f8974009d90).ready entity=mon.2 client_cookie=d2a2026c4a15a169 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:29:04.570 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:04.581 7f897affd700 1 -- 172.21.15.102:0/573858598 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f897c1aca00 con 0x7f897c1a8f60 2024-04-17T22:29:04.570 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:04.581 7f897affd700 1 -- 172.21.15.102:0/573858598 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f897400b840 con 0x7f897c1a8f60 2024-04-17T22:29:04.570 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:04.581 7f897affd700 1 -- 172.21.15.102:0/573858598 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f897400ba20 con 0x7f897c1a8f60 2024-04-17T22:29:04.570 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:04.581 7f898361d700 1 -- 172.21.15.102:0/573858598 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f897c1a8f60 msgr2=0x7f897c1ab3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:04.570 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:04.581 7f898361d700 1 --2- 172.21.15.102:0/573858598 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f897c1a8f60 0x7f897c1ab3c0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 rx=0x7f897c0f4490 tx=0x7f8974009d90).stop 2024-04-17T22:29:04.570 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:04.581 7f898361d700 1 -- 172.21.15.102:0/573858598 shutdown_connections 2024-04-17T22:29:04.571 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:04.581 7f898361d700 1 --2- 172.21.15.102:0/573858598 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f897c1a8f60 0x7f897c1ab3c0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:04.571 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:04.581 7f898361d700 1 --2- 172.21.15.102:0/573858598 >> v2:172.21.15.102:3300/0 conn(0x7f897c1a64d0 0x7f897c1a8930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:04.571 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:04.582 7f898361d700 1 -- 172.21.15.102:0/573858598 >> 172.21.15.102:0/573858598 conn(0x7f897c1138c0 msgr2=0x7f897c0f4f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:29:04.571 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:04.582 7f898361d700 1 -- 172.21.15.102:0/573858598 shutdown_connections 2024-04-17T22:29:04.571 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:04.582 7f898361d700 1 -- 172.21.15.102:0/573858598 wait complete. 2024-04-17T22:29:04.591 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:04.604 7f898361d700 1 Processor -- start 2024-04-17T22:29:04.591 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:04.604 7f898361d700 1 -- start start 2024-04-17T22:29:04.593 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:04.605 7f898361d700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f897c0cdf60 0x7f897c0d5fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:04.593 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:04.605 7f898361d700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f897c0d6600 0x7f897c1b4030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:04.593 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:04.605 7f898361d700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f897c0d6a50 0x7f897c1b4660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:04.593 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:04.605 7f898361d700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f897c08f2e0 con 0x7f897c0d6a50 2024-04-17T22:29:04.593 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:04.605 7f898361d700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f897c0fa1d0 con 0x7f897c0d6600 2024-04-17T22:29:04.593 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:04.605 7f898361d700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f897c0cd6c0 con 0x7f897c0cdf60 2024-04-17T22:29:04.594 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:04.605 7f897bfff700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f897c0cdf60 0x7f897c0d5fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:04.594 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:04.605 7f8980c19700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f897c0d6a50 0x7f897c1b4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:04.594 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:04.605 7f897b7fe700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f897c0d6600 0x7f897c1b4030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:04.594 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:04.605 7f897b7fe700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f897c0d6600 0x7f897c1b4030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:44006/0 (socket says 172.21.15.102:44006) 2024-04-17T22:29:04.594 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:04.605 7f897bfff700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f897c0cdf60 0x7f897c0d5fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:40844/0 (socket says 172.21.15.102:40844) 2024-04-17T22:29:04.594 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:04.605 7f8980c19700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f897c0d6a50 0x7f897c1b4660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:49062/0 (socket says 172.21.15.102:49062) 2024-04-17T22:29:04.594 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:04.605 7f897b7fe700 1 -- 172.21.15.102:0/124883561 learned_addr learned my addr 172.21.15.102:0/124883561 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:29:04.594 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:04.605 7f8980c19700 1 -- 172.21.15.102:0/124883561 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f897c0cdf60 msgr2=0x7f897c0d5fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:04.594 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:04.605 7f8980c19700 1 --2- 172.21.15.102:0/124883561 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f897c0cdf60 0x7f897c0d5fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:04.594 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:04.605 7f8980c19700 1 -- 172.21.15.102:0/124883561 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f897c0d6600 msgr2=0x7f897c1b4030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:04.594 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:04.605 7f8980c19700 1 --2- 172.21.15.102:0/124883561 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f897c0d6600 0x7f897c1b4030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:04.594 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:04.605 7f8980c19700 1 -- 172.21.15.102:0/124883561 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f897c0c44e0 con 0x7f897c0d6a50 2024-04-17T22:29:04.594 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:04.605 7f897bfff700 1 --2- 172.21.15.102:0/124883561 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f897c0cdf60 0x7f897c0d5fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:29:04.595 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:04.605 7f897b7fe700 1 --2- 172.21.15.102:0/124883561 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f897c0d6600 0x7f897c1b4030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:29:04.595 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:04.605 7f8980c19700 1 --2- 172.21.15.102:0/124883561 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f897c0d6a50 0x7f897c1b4660 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7f897c1b0ac0 tx=0x7f8974006c40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:29:04.595 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:04.606 7f89797fa700 1 -- 172.21.15.102:0/124883561 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8974017040 con 0x7f897c0d6a50 2024-04-17T22:29:04.595 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:04.606 7f89797fa700 1 -- 172.21.15.102:0/124883561 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8974011a40 con 0x7f897c0d6a50 2024-04-17T22:29:04.595 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:04.606 7f898361d700 1 -- 172.21.15.102:0/124883561 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f897c1b4c90 con 0x7f897c0d6a50 2024-04-17T22:29:04.595 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:04.606 7f89797fa700 1 -- 172.21.15.102:0/124883561 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f897401c470 con 0x7f897c0d6a50 2024-04-17T22:29:04.595 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:04.606 7f898361d700 1 -- 172.21.15.102:0/124883561 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f897c1bb720 con 0x7f897c0d6a50 2024-04-17T22:29:04.595 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:04.606 7f89797fa700 1 -- 172.21.15.102:0/124883561 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7f897401c670 con 0x7f897c0d6a50 2024-04-17T22:29:04.595 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:04.606 7f89797fa700 1 -- 172.21.15.102:0/124883561 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(12..12 src has 1..12) v4 ==== 2374+0+0 (secure 0 0 0) 0x7f8974018030 con 0x7f897c0d6a50 2024-04-17T22:29:04.595 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:04.607 7f898361d700 1 -- 172.21.15.102:0/124883561 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f897c0f0820 con 0x7f897c0d6a50 2024-04-17T22:29:04.596 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:04.610 7f89797fa700 1 -- 172.21.15.102:0/124883561 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+110455 (secure 0 0 0) 0x7f8974011830 con 0x7f897c0d6a50 2024-04-17T22:29:04.729 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:04.743 7f898361d700 1 -- 172.21.15.102:0/124883561 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f897c1ae730 con 0x7f897c0d6a50 2024-04-17T22:29:04.730 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:04.743 7f89797fa700 1 -- 172.21.15.102:0/124883561 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+780 (secure 0 0 0) 0x7f897400b630 con 0x7f897c0d6a50 2024-04-17T22:29:04.730 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-17T22:29:04.730 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4177,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.102:6817/1944542598","addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6817","nonce":1944542598}]},"export_targets":[],"features":4611087854035861503,"flags":0,"epoch":2}],"filesystems":[]} 2024-04-17T22:29:04.731 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:04.744 7f898361d700 1 -- 172.21.15.102:0/124883561 >> v1:172.21.15.102:6816/42360 conn(0x7f895c0145a0 legacy=0x7f895c016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:04.731 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:04.744 7f898361d700 1 -- 172.21.15.102:0/124883561 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f897c0d6a50 msgr2=0x7f897c1b4660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:04.731 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:04.744 7f898361d700 1 --2- 172.21.15.102:0/124883561 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f897c0d6a50 0x7f897c1b4660 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7f897c1b0ac0 tx=0x7f8974006c40).stop 2024-04-17T22:29:04.731 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:04.744 7f898361d700 1 -- 172.21.15.102:0/124883561 shutdown_connections 2024-04-17T22:29:04.731 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:04.744 7f898361d700 1 --2- 172.21.15.102:0/124883561 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f897c0d6a50 0x7f897c1b4660 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:04.731 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:04.744 7f898361d700 1 --2- 172.21.15.102:0/124883561 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f897c0cdf60 0x7f897c0d5fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:04.732 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:04.744 7f898361d700 1 --2- 172.21.15.102:0/124883561 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f897c0d6600 0x7f897c1b4030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:04.732 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:04.744 7f898361d700 1 -- 172.21.15.102:0/124883561 >> 172.21.15.102:0/124883561 conn(0x7f897c0c38b0 msgr2=0x7f897c0dae50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:29:04.732 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:04.744 7f898361d700 1 -- 172.21.15.102:0/124883561 shutdown_connections 2024-04-17T22:29:04.732 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:04.744 7f898361d700 1 -- 172.21.15.102:0/124883561 wait complete. 2024-04-17T22:29:04.732 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 2 2024-04-17T22:29:04.742 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T22:29:04.867 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:04.878 7f88fa6ae700 1 Processor -- start 2024-04-17T22:29:04.867 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:04.878 7f88fa6ae700 1 -- start start 2024-04-17T22:29:04.867 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:04.879 7f88fa6ae700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f88f410e1d0 0x7f88f410e620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:04.867 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:04.879 7f88fa6ae700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f88f410ec50 0x7f88f40f16c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:04.867 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:04.879 7f88fa6ae700 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88f40f2140 con 0x7f88f40f1cf0 2024-04-17T22:29:04.867 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:04.879 7f88fa6ae700 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88f40f9ea0 con 0x7f88f410e1d0 2024-04-17T22:29:04.867 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:04.879 7f88fa6ae700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f88f40f8fc0 con 0x7f88f410ec50 2024-04-17T22:29:04.868 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:04.879 7f88f33b0700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f88f410e1d0 0x7f88f410e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:04.868 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:04.879 7f88f2baf700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f88f410ec50 0x7f88f40f16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:04.868 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:04.879 7f88f33b0700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f88f410e1d0 0x7f88f410e620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:49074/0 (socket says 172.21.15.102:49074) 2024-04-17T22:29:04.868 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:04.879 7f88f33b0700 1 -- 172.21.15.102:0/2605694338 learned_addr learned my addr 172.21.15.102:0/2605694338 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:29:04.868 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:04.879 7f88f2baf700 1 -- 172.21.15.102:0/2605694338 >> v2:172.21.15.102:3300/0 conn(0x7f88f410e1d0 msgr2=0x7f88f410e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:29:04.868 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:04.879 7f88f23ae700 1 -- 172.21.15.102:0/2605694338 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4252467835 0 0) 0x7f88ec001490 con 0x7f88f40f1cf0 2024-04-17T22:29:04.868 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:04.879 7f88f2baf700 1 --2- 172.21.15.102:0/2605694338 >> v2:172.21.15.102:3300/0 conn(0x7f88f410e1d0 0x7f88f410e620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:04.868 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:04.879 7f88f2baf700 1 -- 172.21.15.102:0/2605694338 >> v1:172.21.15.102:6789/0 conn(0x7f88f40f1cf0 legacy=0x7f88f41aa460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:04.868 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:04.879 7f88f2baf700 1 -- 172.21.15.102:0/2605694338 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88f40f2360 con 0x7f88f410ec50 2024-04-17T22:29:04.868 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:04.879 7f88f2baf700 1 --2- 172.21.15.102:0/2605694338 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f88f410ec50 0x7f88f40f16c0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0x7f88e4002f70 tx=0x7f88e400b970).ready entity=mon.2 client_cookie=477bd80589589203 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:29:04.868 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:04.879 7f88f23ae700 1 -- 172.21.15.102:0/2605694338 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f88f40f2360 con 0x7f88f410ec50 2024-04-17T22:29:04.868 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:04.880 7f88f23ae700 1 -- 172.21.15.102:0/2605694338 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88e400d610 con 0x7f88f410ec50 2024-04-17T22:29:04.868 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:04.880 7f88f23ae700 1 -- 172.21.15.102:0/2605694338 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f88e400d9e0 con 0x7f88f410ec50 2024-04-17T22:29:04.868 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:04.880 7f88fa6ae700 1 -- 172.21.15.102:0/2605694338 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f88f410ec50 msgr2=0x7f88f40f16c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:04.868 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:04.880 7f88fa6ae700 1 --2- 172.21.15.102:0/2605694338 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f88f410ec50 0x7f88f40f16c0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0x7f88e4002f70 tx=0x7f88e400b970).stop 2024-04-17T22:29:04.869 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:04.880 7f88fa6ae700 1 -- 172.21.15.102:0/2605694338 shutdown_connections 2024-04-17T22:29:04.869 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:04.880 7f88fa6ae700 1 --2- 172.21.15.102:0/2605694338 >> v2:172.21.15.102:3300/0 conn(0x7f88f410e1d0 0x7f88f410e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:04.869 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:04.880 7f88fa6ae700 1 --2- 172.21.15.102:0/2605694338 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f88f410ec50 0x7f88f40f16c0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:04.869 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:04.880 7f88fa6ae700 1 -- 172.21.15.102:0/2605694338 >> 172.21.15.102:0/2605694338 conn(0x7f88f4113830 msgr2=0x7f88f40f4ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:29:04.869 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:04.880 7f88fa6ae700 1 -- 172.21.15.102:0/2605694338 shutdown_connections 2024-04-17T22:29:04.869 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:04.880 7f88fa6ae700 1 -- 172.21.15.102:0/2605694338 wait complete. 2024-04-17T22:29:04.889 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:04.903 7f88fa6ae700 1 Processor -- start 2024-04-17T22:29:04.889 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:04.903 7f88fa6ae700 1 -- start start 2024-04-17T22:29:04.892 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:04.903 7f88fa6ae700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f88f40cdef0 0x7f88f40d51d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:04.892 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:04.903 7f88fa6ae700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f88f40d5800 0x7f88f41b4150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:04.892 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:04.903 7f88fa6ae700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f88f40d5c50 0x7f88f41b4780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:04.892 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:04.903 7f88fa6ae700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88f4113a40 con 0x7f88f40d5800 2024-04-17T22:29:04.892 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:04.903 7f88fa6ae700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88f40fa140 con 0x7f88f40d5c50 2024-04-17T22:29:04.892 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:04.903 7f88fa6ae700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f88f40ce340 con 0x7f88f40cdef0 2024-04-17T22:29:04.892 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:04.903 7f88f33b0700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f88f40cdef0 0x7f88f40d51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:04.892 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:04.903 7f88f33b0700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f88f40cdef0 0x7f88f40d51d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:40856/0 (socket says 172.21.15.102:40856) 2024-04-17T22:29:04.892 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:04.903 7f88f33b0700 1 -- 172.21.15.102:0/1662489892 learned_addr learned my addr 172.21.15.102:0/1662489892 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:29:04.892 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:04.903 7f88f3bb1700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f88f40d5c50 0x7f88f41b4780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:04.892 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:04.903 7f88f2baf700 1 --2- 172.21.15.102:0/1662489892 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f88f40d5800 0x7f88f41b4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:04.892 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:04.904 7f88f2baf700 1 -- 172.21.15.102:0/1662489892 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f88f40cdef0 msgr2=0x7f88f40d51d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:04.893 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:04.904 7f88f2baf700 1 --2- 172.21.15.102:0/1662489892 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f88f40cdef0 0x7f88f40d51d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:04.893 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:04.904 7f88f2baf700 1 -- 172.21.15.102:0/1662489892 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f88f40d5c50 msgr2=0x7f88f41b4780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:04.893 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:04.904 7f88f2baf700 1 --2- 172.21.15.102:0/1662489892 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f88f40d5c50 0x7f88f41b4780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:04.893 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:04.904 7f88f2baf700 1 -- 172.21.15.102:0/1662489892 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88f40cea30 con 0x7f88f40d5800 2024-04-17T22:29:04.893 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:04.904 7f88f3bb1700 1 --2- 172.21.15.102:0/1662489892 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f88f40d5c50 0x7f88f41b4780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:29:04.893 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:04.904 7f88f2baf700 1 --2- 172.21.15.102:0/1662489892 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f88f40d5800 0x7f88f41b4150 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7f88e4002f70 tx=0x7f88e400a040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:29:04.893 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:04.904 7f88f0bab700 1 -- 172.21.15.102:0/1662489892 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f88f40cea30 con 0x7f88f40d5800 2024-04-17T22:29:04.893 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:04.904 7f88f0bab700 1 -- 172.21.15.102:0/1662489892 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88e4013a10 con 0x7f88f40d5800 2024-04-17T22:29:04.893 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:04.904 7f88fa6ae700 1 -- 172.21.15.102:0/1662489892 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88f40cbff0 con 0x7f88f40d5800 2024-04-17T22:29:04.893 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:04.904 7f88f0bab700 1 -- 172.21.15.102:0/1662489892 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f88e4013de0 con 0x7f88f40d5800 2024-04-17T22:29:04.893 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:04.904 7f88fa6ae700 1 -- 172.21.15.102:0/1662489892 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88f41b4f90 con 0x7f88f40d5800 2024-04-17T22:29:04.893 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:04.904 7f88f0bab700 1 -- 172.21.15.102:0/1662489892 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7f88e40283f0 con 0x7f88f40d5800 2024-04-17T22:29:04.893 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:04.904 7f88f0bab700 1 -- 172.21.15.102:0/1662489892 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(12..12 src has 1..12) v4 ==== 2374+0+0 (secure 0 0 0) 0x7f88e4022ae0 con 0x7f88f40d5800 2024-04-17T22:29:04.893 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:04.905 7f88fa6ae700 1 -- 172.21.15.102:0/1662489892 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88f40f0790 con 0x7f88f40d5800 2024-04-17T22:29:04.895 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:04.908 7f88f0bab700 1 -- 172.21.15.102:0/1662489892 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+110455 (secure 0 0 0) 0x7f88e4013800 con 0x7f88f40d5800 2024-04-17T22:29:04.999 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.013 7f88fa6ae700 1 -- 172.21.15.102:0/1662489892 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f88f41ae8b0 con 0x7f88f40d5800 2024-04-17T22:29:05.000 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.013 7f88f0bab700 1 -- 172.21.15.102:0/1662489892 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v12) v1 ==== 74+0+5904 (secure 0 0 0) 0x7f88e400d400 con 0x7f88f40d5800 2024-04-17T22:29:05.000 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-17T22:29:05.000 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":12,"fsid":"cb63e789-091d-49b1-b120-55f9db526871","created":"2024-04-17 22:28:48.209635","modified":"2024-04-17 22:29:00.246113","last_up_change":"2024-04-17 22:28:54.236215","last_in_change":"2024-04-17 22:28:54.236215","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":4,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-04-17 22:28:57.478545","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"3351ef85-ade4-4b90-a8f4-213f4e00ba01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":10,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6812","nonce":42955}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6813","nonce":42955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6815","nonce":42955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6814","nonce":42955}]},"public_addr":"172.21.15.102:6812/42955","cluster_addr":"172.21.15.102:6813/42955","heartbeat_back_addr":"172.21.15.102:6815/42955","heartbeat_front_addr":"172.21.15.102:6814/42955","state":["exists","up"]},{"osd":1,"uuid":"04f67eac-f36b-48f5-982d-4cfc776596d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":10,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6808","nonce":42958}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6809","nonce":42958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6811","nonce":42958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6810","nonce":42958}]},"public_addr":"172.21.15.102:6808/42958","cluster_addr":"172.21.15.102:6809/42958","heartbeat_back_addr":"172.21.15.102:6811/42958","heartbeat_front_addr":"172.21.15.102:6810/42958","state":["exists","up"]},{"osd":2,"uuid":"d524204b-70b2-40a8-8f73-f9e7d44318af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":10,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6804","nonce":42954}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6805","nonce":42954}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6807","nonce":42954}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6806","nonce":42954}]},"public_addr":"172.21.15.102:6804/42954","cluster_addr":"172.21.15.102:6805/42954","heartbeat_back_addr":"172.21.15.102:6807/42954","heartbeat_front_addr":"172.21.15.102:6806/42954","state":["exists","up"]},{"osd":3,"uuid":"4fc9d20c-1ad0-484f-aafa-2c9a7574245b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":10,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6800","nonce":42953}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6801","nonce":42953}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6803","nonce":42953}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6802","nonce":42953}]},"public_addr":"172.21.15.102:6800/42953","cluster_addr":"172.21.15.102:6801/42953","heartbeat_back_addr":"172.21.15.102:6803/42953","heartbeat_front_addr":"172.21.15.102:6802/42953","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2024-04-17T22:29:05.001 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.014 7f88fa6ae700 1 -- 172.21.15.102:0/1662489892 >> v1:172.21.15.102:6816/42360 conn(0x7f88d40145a0 legacy=0x7f88d4016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:05.001 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.014 7f88fa6ae700 1 -- 172.21.15.102:0/1662489892 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f88f40d5800 msgr2=0x7f88f41b4150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:05.002 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.014 7f88fa6ae700 1 --2- 172.21.15.102:0/1662489892 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f88f40d5800 0x7f88f41b4150 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7f88e4002f70 tx=0x7f88e400a040).stop 2024-04-17T22:29:05.002 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.014 7f88fa6ae700 1 -- 172.21.15.102:0/1662489892 shutdown_connections 2024-04-17T22:29:05.002 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.014 7f88fa6ae700 1 --2- 172.21.15.102:0/1662489892 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f88f40d5c50 0x7f88f41b4780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:05.002 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.014 7f88fa6ae700 1 --2- 172.21.15.102:0/1662489892 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f88f40cdef0 0x7f88f40d51d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:05.002 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.014 7f88fa6ae700 1 --2- 172.21.15.102:0/1662489892 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f88f40d5800 0x7f88f41b4150 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:05.002 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.014 7f88fa6ae700 1 -- 172.21.15.102:0/1662489892 >> 172.21.15.102:0/1662489892 conn(0x7f88f40c6af0 msgr2=0x7f88f41a8270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:29:05.002 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.015 7f88fa6ae700 1 -- 172.21.15.102:0/1662489892 shutdown_connections 2024-04-17T22:29:05.002 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.015 7f88fa6ae700 1 -- 172.21.15.102:0/1662489892 wait complete. 2024-04-17T22:29:05.012 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2024-04-17 22:28:57.478545', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '12', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2024-04-17T22:29:05.012 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2024-04-17T22:29:05.141 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.152 7f8e93b7e700 1 Processor -- start 2024-04-17T22:29:05.142 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.152 7f8e93b7e700 1 -- start start 2024-04-17T22:29:05.142 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.152 7f8e93b7e700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f8e8c0f5840 0x7f8e8c10b010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:05.142 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.153 7f8e93b7e700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f8e8c10ba90 0x7f8e8c1a02f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:05.142 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.153 7f8e93b7e700 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e8c103b40 con 0x7f8e8c10b640 2024-04-17T22:29:05.142 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.153 7f8e93b7e700 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e8c10bee0 con 0x7f8e8c0f5840 2024-04-17T22:29:05.142 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.153 7f8e93b7e700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e8c0fa370 con 0x7f8e8c10ba90 2024-04-17T22:29:05.142 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.153 7f8e90979700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f8e8c0f5840 0x7f8e8c10b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:05.142 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.153 7f8e9117a700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f8e8c10ba90 0x7f8e8c1a02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:05.142 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.153 7f8e8bfff700 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f8e8c10b640 0x7f8e8c19faf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:46420/0 (socket says 172.21.15.102:46420) 2024-04-17T22:29:05.142 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.153 7f8e8bfff700 1 -- 172.21.15.102:0/2032261433 learned_addr learned my addr 172.21.15.102:0/2032261433 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-17T22:29:05.143 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.153 7f8e90979700 1 -- 172.21.15.102:0/2032261433 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f8e8c10ba90 msgr2=0x7f8e8c1a02f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:29:05.143 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.153 7f8e90979700 1 --2- 172.21.15.102:0/2032261433 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f8e8c10ba90 0x7f8e8c1a02f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:05.143 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.153 7f8e90979700 1 -- 172.21.15.102:0/2032261433 >> v1:172.21.15.102:6789/0 conn(0x7f8e8c10b640 legacy=0x7f8e8c19faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:05.143 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.153 7f8e90979700 1 -- 172.21.15.102:0/2032261433 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e8c1a0920 con 0x7f8e8c0f5840 2024-04-17T22:29:05.143 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.153 7f8e8b7fe700 1 -- 172.21.15.102:0/2032261433 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3166081012 0 0) 0x7f8e78001110 con 0x7f8e8c10b640 2024-04-17T22:29:05.143 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.153 7f8e9117a700 1 --2- 172.21.15.102:0/2032261433 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f8e8c10ba90 0x7f8e8c1a02f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:29:05.143 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.153 7f8e90979700 1 --2- 172.21.15.102:0/2032261433 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8e8c0f5840 0x7f8e8c10b010 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7f8e8c103d60 tx=0x7f8e8000ba20).ready entity=mon.0 client_cookie=7e91b77da7064e7f server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:29:05.143 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.153 7f8e8b7fe700 1 -- 172.21.15.102:0/2032261433 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8e8c1a0920 con 0x7f8e8c0f5840 2024-04-17T22:29:05.143 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.153 7f8e8b7fe700 1 -- 172.21.15.102:0/2032261433 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e8000d5c0 con 0x7f8e8c0f5840 2024-04-17T22:29:05.143 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.153 7f8e8b7fe700 1 -- 172.21.15.102:0/2032261433 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8e8000d990 con 0x7f8e8c0f5840 2024-04-17T22:29:05.143 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.153 7f8e93b7e700 1 -- 172.21.15.102:0/2032261433 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8e8c0f5840 msgr2=0x7f8e8c10b010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:05.143 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.153 7f8e93b7e700 1 --2- 172.21.15.102:0/2032261433 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8e8c0f5840 0x7f8e8c10b010 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7f8e8c103d60 tx=0x7f8e8000ba20).stop 2024-04-17T22:29:05.143 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.154 7f8e93b7e700 1 -- 172.21.15.102:0/2032261433 shutdown_connections 2024-04-17T22:29:05.144 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.154 7f8e93b7e700 1 --2- 172.21.15.102:0/2032261433 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f8e8c10ba90 0x7f8e8c1a02f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:05.144 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.154 7f8e93b7e700 1 --2- 172.21.15.102:0/2032261433 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8e8c0f5840 0x7f8e8c10b010 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:05.144 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.154 7f8e93b7e700 1 -- 172.21.15.102:0/2032261433 >> 172.21.15.102:0/2032261433 conn(0x7f8e8c1138a0 msgr2=0x7f8e8c0f4f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:29:05.144 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.154 7f8e93b7e700 1 -- 172.21.15.102:0/2032261433 shutdown_connections 2024-04-17T22:29:05.144 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.154 7f8e93b7e700 1 -- 172.21.15.102:0/2032261433 wait complete. 2024-04-17T22:29:05.163 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.177 7f8e93b7e700 1 Processor -- start 2024-04-17T22:29:05.163 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.177 7f8e93b7e700 1 -- start start 2024-04-17T22:29:05.166 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.177 7f8e93b7e700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f8e8c0d5520 0x7f8e8c0d5970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:05.166 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.177 7f8e93b7e700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f8e8c0d5fa0 0x7f8e8c1b04a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:05.166 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.177 7f8e93b7e700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8e8c1b0ad0 0x7f8e8c1b0f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:05.166 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.177 7f8e93b7e700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e8c0d9350 con 0x7f8e8c1b0ad0 2024-04-17T22:29:05.166 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.177 7f8e93b7e700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e8c1b1550 con 0x7f8e8c0d5fa0 2024-04-17T22:29:05.166 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.177 7f8e93b7e700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8e8c0d9610 con 0x7f8e8c0d5520 2024-04-17T22:29:05.167 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.178 7f8e90979700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f8e8c0d5520 0x7f8e8c0d5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:05.167 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.178 7f8e9117a700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8e8c1b0ad0 0x7f8e8c1b0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:05.167 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.178 7f8e8bfff700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f8e8c0d5fa0 0x7f8e8c1b04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:05.167 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.178 7f8e90979700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f8e8c0d5520 0x7f8e8c0d5970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:40864/0 (socket says 172.21.15.102:40864) 2024-04-17T22:29:05.167 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.178 7f8e9117a700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8e8c1b0ad0 0x7f8e8c1b0f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:49106/0 (socket says 172.21.15.102:49106) 2024-04-17T22:29:05.167 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.178 7f8e90979700 1 -- 172.21.15.102:0/2814173620 learned_addr learned my addr 172.21.15.102:0/2814173620 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:29:05.167 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.178 7f8e90979700 1 -- 172.21.15.102:0/2814173620 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f8e8c0d5fa0 msgr2=0x7f8e8c1b04a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:05.167 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.178 7f8e90979700 1 --2- 172.21.15.102:0/2814173620 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f8e8c0d5fa0 0x7f8e8c1b04a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:05.167 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.178 7f8e90979700 1 -- 172.21.15.102:0/2814173620 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8e8c1b0ad0 msgr2=0x7f8e8c1b0f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:05.167 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.178 7f8e90979700 1 --2- 172.21.15.102:0/2814173620 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8e8c1b0ad0 0x7f8e8c1b0f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:05.167 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.178 7f8e90979700 1 -- 172.21.15.102:0/2814173620 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e8c0d9c70 con 0x7f8e8c0d5520 2024-04-17T22:29:05.167 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.178 7f8e8bfff700 1 --2- 172.21.15.102:0/2814173620 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f8e8c0d5fa0 0x7f8e8c1b04a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:29:05.167 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.178 7f8e9117a700 1 --2- 172.21.15.102:0/2814173620 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8e8c1b0ad0 0x7f8e8c1b0f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:29:05.167 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.178 7f8e90979700 1 --2- 172.21.15.102:0/2814173620 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f8e8c0d5520 0x7f8e8c0d5970 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7f8e8c0c4ab0 tx=0x7f8e800068c0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:29:05.168 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.178 7f8e89ffb700 1 -- 172.21.15.102:0/2814173620 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8e8c0d9c70 con 0x7f8e8c0d5520 2024-04-17T22:29:05.168 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.178 7f8e89ffb700 1 -- 172.21.15.102:0/2814173620 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e800119d0 con 0x7f8e8c0d5520 2024-04-17T22:29:05.168 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.178 7f8e93b7e700 1 -- 172.21.15.102:0/2814173620 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e8c1b7b90 con 0x7f8e8c0d5520 2024-04-17T22:29:05.168 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.178 7f8e89ffb700 1 -- 172.21.15.102:0/2814173620 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8e80022070 con 0x7f8e8c0d5520 2024-04-17T22:29:05.168 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.178 7f8e93b7e700 1 -- 172.21.15.102:0/2814173620 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e8c1b7f70 con 0x7f8e8c0d5520 2024-04-17T22:29:05.168 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.179 7f8e89ffb700 1 -- 172.21.15.102:0/2814173620 <== mon.2 v2:172.21.15.102:3302/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7f8e80026410 con 0x7f8e8c0d5520 2024-04-17T22:29:05.168 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.179 7f8e89ffb700 1 -- 172.21.15.102:0/2814173620 <== mon.2 v2:172.21.15.102:3302/0 5 ==== osd_map(12..12 src has 1..12) v4 ==== 2374+0+0 (secure 0 0 0) 0x7f8e80009a80 con 0x7f8e8c0d5520 2024-04-17T22:29:05.168 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.179 7f8e93b7e700 1 -- 172.21.15.102:0/2814173620 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e8c0f0800 con 0x7f8e8c0d5520 2024-04-17T22:29:05.169 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.183 7f8e89ffb700 1 -- 172.21.15.102:0/2814173620 <== mon.2 v2:172.21.15.102:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+110455 (secure 0 0 0) 0x7f8e800117c0 con 0x7f8e8c0d5520 2024-04-17T22:29:05.274 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.288 7f8e93b7e700 1 -- 172.21.15.102:0/2814173620 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f8e8c1aabc0 con 0x7f8e8c0d5520 2024-04-17T22:29:05.274 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.288 7f8e89ffb700 1 -- 172.21.15.102:0/2814173620 <== mon.2 v2:172.21.15.102:3302/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v12) v1 ==== 92+0+10 (secure 0 0 0) 0x7f8e8000d3b0 con 0x7f8e8c0d5520 2024-04-17T22:29:05.275 INFO:teuthology.orchestra.run.smithi102.stdout:pg_num: 8 2024-04-17T22:29:05.276 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.289 7f8e93b7e700 1 -- 172.21.15.102:0/2814173620 >> v1:172.21.15.102:6816/42360 conn(0x7f8e740145a0 legacy=0x7f8e74016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:05.276 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.289 7f8e93b7e700 1 -- 172.21.15.102:0/2814173620 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f8e8c0d5520 msgr2=0x7f8e8c0d5970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:05.276 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.289 7f8e93b7e700 1 --2- 172.21.15.102:0/2814173620 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f8e8c0d5520 0x7f8e8c0d5970 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7f8e8c0c4ab0 tx=0x7f8e800068c0).stop 2024-04-17T22:29:05.276 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.289 7f8e93b7e700 1 -- 172.21.15.102:0/2814173620 shutdown_connections 2024-04-17T22:29:05.276 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.289 7f8e93b7e700 1 --2- 172.21.15.102:0/2814173620 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8e8c1b0ad0 0x7f8e8c1b0f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:05.276 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.289 7f8e93b7e700 1 --2- 172.21.15.102:0/2814173620 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f8e8c0d5520 0x7f8e8c0d5970 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:05.277 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.289 7f8e93b7e700 1 --2- 172.21.15.102:0/2814173620 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f8e8c0d5fa0 0x7f8e8c1b04a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:05.277 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.289 7f8e93b7e700 1 -- 172.21.15.102:0/2814173620 >> 172.21.15.102:0/2814173620 conn(0x7f8e8c0d6700 msgr2=0x7f8e8c0ce6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:29:05.277 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.289 7f8e93b7e700 1 -- 172.21.15.102:0/2814173620 shutdown_connections 2024-04-17T22:29:05.277 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.289 7f8e93b7e700 1 -- 172.21.15.102:0/2814173620 wait complete. 2024-04-17T22:29:05.287 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-17T22:29:05.411 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.425 7f599d499700 1 Processor -- start 2024-04-17T22:29:05.412 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.425 7f599d499700 1 -- start start 2024-04-17T22:29:05.414 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.425 7f599d499700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f59980f5860 0x7f599810b030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:05.414 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.425 7f599d499700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f599810bab0 0x7f59981a0310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:05.414 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.425 7f599d499700 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5998103b60 con 0x7f599810b660 2024-04-17T22:29:05.414 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.425 7f599d499700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f599810bf00 con 0x7f59980f5860 2024-04-17T22:29:05.414 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.425 7f599d499700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f59980fa390 con 0x7f599810bab0 2024-04-17T22:29:05.414 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.426 7f599626f700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f59980f5860 0x7f599810b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:05.414 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.426 7f5996a70700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f599810bab0 0x7f59981a0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:05.414 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.426 7f599626f700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f59980f5860 0x7f599810b030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:44046/0 (socket says 172.21.15.102:44046) 2024-04-17T22:29:05.414 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.426 7f5996a70700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f599810bab0 0x7f59981a0310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:40874/0 (socket says 172.21.15.102:40874) 2024-04-17T22:29:05.415 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.426 7f599626f700 1 -- 172.21.15.102:0/1852452848 learned_addr learned my addr 172.21.15.102:0/1852452848 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:29:05.415 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.426 7f599626f700 1 -- 172.21.15.102:0/1852452848 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f599810bab0 msgr2=0x7f59981a0310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:29:05.415 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.426 7f599626f700 1 --2- 172.21.15.102:0/1852452848 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f599810bab0 0x7f59981a0310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:05.415 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.426 7f599626f700 1 -- 172.21.15.102:0/1852452848 >> v1:172.21.15.102:6789/0 conn(0x7f599810b660 legacy=0x7f599819fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:05.415 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.426 7f599626f700 1 -- 172.21.15.102:0/1852452848 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59981a0940 con 0x7f59980f5860 2024-04-17T22:29:05.415 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.426 7f599626f700 1 --2- 172.21.15.102:0/1852452848 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f59980f5860 0x7f599810b030 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7f5998103d80 tx=0x7f5988009ab0).ready entity=mon.1 client_cookie=a4ae9de4754847aa server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:29:05.415 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.426 7f5995a6e700 1 -- 172.21.15.102:0/1852452848 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f59981a0940 con 0x7f59980f5860 2024-04-17T22:29:05.415 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.426 7f5995a6e700 1 -- 172.21.15.102:0/1852452848 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f598800f510 con 0x7f59980f5860 2024-04-17T22:29:05.415 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.426 7f5995a6e700 1 -- 172.21.15.102:0/1852452848 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f598800f6f0 con 0x7f59980f5860 2024-04-17T22:29:05.415 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.426 7f599d499700 1 -- 172.21.15.102:0/1852452848 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f59980f5860 msgr2=0x7f599810b030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:05.415 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.426 7f599d499700 1 --2- 172.21.15.102:0/1852452848 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f59980f5860 0x7f599810b030 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7f5998103d80 tx=0x7f5988009ab0).stop 2024-04-17T22:29:05.415 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.427 7f599d499700 1 -- 172.21.15.102:0/1852452848 shutdown_connections 2024-04-17T22:29:05.415 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.427 7f599d499700 1 --2- 172.21.15.102:0/1852452848 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f599810bab0 0x7f59981a0310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:05.415 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.427 7f599d499700 1 --2- 172.21.15.102:0/1852452848 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f59980f5860 0x7f599810b030 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:05.415 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.427 7f599d499700 1 -- 172.21.15.102:0/1852452848 >> 172.21.15.102:0/1852452848 conn(0x7f59981138c0 msgr2=0x7f59980f4f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:29:05.416 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.427 7f599d499700 1 -- 172.21.15.102:0/1852452848 shutdown_connections 2024-04-17T22:29:05.416 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.427 7f599d499700 1 -- 172.21.15.102:0/1852452848 wait complete. 2024-04-17T22:29:05.438 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.450 7f599d499700 1 Processor -- start 2024-04-17T22:29:05.438 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.450 7f599d499700 1 -- start start 2024-04-17T22:29:05.438 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.450 7f599d499700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f59980d1d80 0x7f59980dcc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:05.438 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.450 7f599d499700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f59980dd270 0x7f59981ab050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:05.438 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.450 7f599d499700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f59980dd6c0 0x7f59981ab680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:05.438 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.450 7f599d499700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59980f2650 con 0x7f59980dd6c0 2024-04-17T22:29:05.439 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.450 7f599d499700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59980d3800 con 0x7f59980dd270 2024-04-17T22:29:05.439 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.450 7f599d499700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f59980f2230 con 0x7f59980d1d80 2024-04-17T22:29:05.439 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.450 7f5996a70700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f59980dd6c0 0x7f59981ab680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:05.439 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.450 7f599626f700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f59980d1d80 0x7f59980dcc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:05.439 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.450 7f598da6e700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f59980dd270 0x7f59981ab050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:05.439 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.450 7f5996a70700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f59980dd6c0 0x7f59981ab680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:49112/0 (socket says 172.21.15.102:49112) 2024-04-17T22:29:05.439 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.450 7f5996a70700 1 -- 172.21.15.102:0/3334905244 learned_addr learned my addr 172.21.15.102:0/3334905244 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:29:05.439 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.451 7f5996a70700 1 -- 172.21.15.102:0/3334905244 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f59980d1d80 msgr2=0x7f59980dcc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:05.439 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.451 7f5996a70700 1 --2- 172.21.15.102:0/3334905244 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f59980d1d80 0x7f59980dcc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:05.439 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.451 7f5996a70700 1 -- 172.21.15.102:0/3334905244 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f59980dd270 msgr2=0x7f59981ab050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:05.439 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.451 7f5996a70700 1 --2- 172.21.15.102:0/3334905244 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f59980dd270 0x7f59981ab050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:05.439 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.451 7f5996a70700 1 -- 172.21.15.102:0/3334905244 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59981abcb0 con 0x7f59980dd6c0 2024-04-17T22:29:05.440 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.451 7f599626f700 1 --2- 172.21.15.102:0/3334905244 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f59980d1d80 0x7f59980dcc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:29:05.440 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.451 7f598da6e700 1 --2- 172.21.15.102:0/3334905244 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f59980dd270 0x7f59981ab050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:29:05.440 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.451 7f5996a70700 1 --2- 172.21.15.102:0/3334905244 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f59980dd6c0 0x7f59981ab680 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7f5990006e10 tx=0x7f599000ee70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:29:05.440 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.451 7f598ffff700 1 -- 172.21.15.102:0/3334905244 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f59981abcb0 con 0x7f59980dd6c0 2024-04-17T22:29:05.440 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.451 7f598ffff700 1 -- 172.21.15.102:0/3334905244 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f599000c8e0 con 0x7f59980dd6c0 2024-04-17T22:29:05.440 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.451 7f599d499700 1 -- 172.21.15.102:0/3334905244 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59980daa30 con 0x7f59980dd6c0 2024-04-17T22:29:05.440 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.451 7f598ffff700 1 -- 172.21.15.102:0/3334905244 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f599000f040 con 0x7f59980dd6c0 2024-04-17T22:29:05.440 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.451 7f599d499700 1 -- 172.21.15.102:0/3334905244 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59980daf90 con 0x7f59980dd6c0 2024-04-17T22:29:05.440 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.451 7f598ffff700 1 -- 172.21.15.102:0/3334905244 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7f5990022060 con 0x7f59980dd6c0 2024-04-17T22:29:05.440 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.451 7f598ffff700 1 -- 172.21.15.102:0/3334905244 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(12..12 src has 1..12) v4 ==== 2374+0+0 (secure 0 0 0) 0x7f5990010040 con 0x7f59980dd6c0 2024-04-17T22:29:05.440 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.452 7f599d499700 1 -- 172.21.15.102:0/3334905244 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59980f0820 con 0x7f59980dd6c0 2024-04-17T22:29:05.442 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.455 7f598ffff700 1 -- 172.21.15.102:0/3334905244 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+110455 (secure 0 0 0) 0x7f599000c6d0 con 0x7f59980dd6c0 2024-04-17T22:29:05.548 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.561 7f599d499700 1 -- 172.21.15.102:0/3334905244 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f59981ac290 con 0x7f59980dd6c0 2024-04-17T22:29:05.548 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.562 7f598ffff700 1 -- 172.21.15.102:0/3334905244 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v12) v1 ==== 84+0+64 (secure 0 0 0) 0x7f5998114440 con 0x7f59980dd6c0 2024-04-17T22:29:05.548 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-17T22:29:05.549 INFO:teuthology.orchestra.run.smithi102.stdout:[ 2024-04-17T22:29:05.549 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-04-17T22:29:05.549 INFO:teuthology.orchestra.run.smithi102.stdout: "poolnum": 1, 2024-04-17T22:29:05.549 INFO:teuthology.orchestra.run.smithi102.stdout: "poolname": "rbd" 2024-04-17T22:29:05.549 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-17T22:29:05.549 INFO:teuthology.orchestra.run.smithi102.stdout:] 2024-04-17T22:29:05.550 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.563 7f599d499700 1 -- 172.21.15.102:0/3334905244 >> v1:172.21.15.102:6816/42360 conn(0x7f597c0145a0 legacy=0x7f597c016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:05.550 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.563 7f599d499700 1 -- 172.21.15.102:0/3334905244 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f59980dd6c0 msgr2=0x7f59981ab680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:05.550 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.563 7f599d499700 1 --2- 172.21.15.102:0/3334905244 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f59980dd6c0 0x7f59981ab680 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7f5990006e10 tx=0x7f599000ee70).stop 2024-04-17T22:29:05.550 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.563 7f599d499700 1 -- 172.21.15.102:0/3334905244 shutdown_connections 2024-04-17T22:29:05.550 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.563 7f599d499700 1 --2- 172.21.15.102:0/3334905244 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f59980dd6c0 0x7f59981ab680 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:05.550 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.563 7f599d499700 1 --2- 172.21.15.102:0/3334905244 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f59980d1d80 0x7f59980dcc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:05.551 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.563 7f599d499700 1 --2- 172.21.15.102:0/3334905244 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f59980dd270 0x7f59981ab050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:05.551 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.563 7f599d499700 1 -- 172.21.15.102:0/3334905244 >> 172.21.15.102:0/3334905244 conn(0x7f59981a3e90 msgr2=0x7f59980d6810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:29:05.551 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.563 7f599d499700 1 -- 172.21.15.102:0/3334905244 shutdown_connections 2024-04-17T22:29:05.551 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.563 7f599d499700 1 -- 172.21.15.102:0/3334905244 wait complete. 2024-04-17T22:29:05.560 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-17T22:29:05.560 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-17T22:29:05.690 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.701 7f7514eae700 1 Processor -- start 2024-04-17T22:29:05.690 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.702 7f7514eae700 1 -- start start 2024-04-17T22:29:05.691 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.702 7f7514eae700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f75100f57b0 0x7f751010ae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:05.691 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.702 7f7514eae700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f751010b450 0x7f751019fad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:05.691 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.702 7f7514eae700 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75101a0900 con 0x7f751010b8a0 2024-04-17T22:29:05.691 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.702 7f7514eae700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75101a0b20 con 0x7f751010b450 2024-04-17T22:29:05.691 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.702 7f7514eae700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f75101056b0 con 0x7f75100f57b0 2024-04-17T22:29:05.691 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.702 7f750db64700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f75100f57b0 0x7f751010ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:05.691 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.702 7f750d363700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f751010b450 0x7f751019fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:05.691 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.702 7f750e365700 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f751010b8a0 0x7f75101a0100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:46430/0 (socket says 172.21.15.102:46430) 2024-04-17T22:29:05.691 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.702 7f750e365700 1 -- 172.21.15.102:0/2203744918 learned_addr learned my addr 172.21.15.102:0/2203744918 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-17T22:29:05.691 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.702 7f750cb62700 1 -- 172.21.15.102:0/2203744918 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3313967715 0 0) 0x7f7508001490 con 0x7f751010b8a0 2024-04-17T22:29:05.691 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.702 7f750db64700 1 -- 172.21.15.102:0/2203744918 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f751010b450 msgr2=0x7f751019fad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:29:05.691 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.702 7f750db64700 1 --2- 172.21.15.102:0/2203744918 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f751010b450 0x7f751019fad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:05.692 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.702 7f750db64700 1 -- 172.21.15.102:0/2203744918 >> v1:172.21.15.102:6789/0 conn(0x7f751010b8a0 legacy=0x7f75101a0100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:05.692 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.702 7f750db64700 1 -- 172.21.15.102:0/2203744918 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75101a91e0 con 0x7f75100f57b0 2024-04-17T22:29:05.692 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.702 7f750d363700 1 --2- 172.21.15.102:0/2203744918 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f751010b450 0x7f751019fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:29:05.692 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.703 7f750db64700 1 --2- 172.21.15.102:0/2203744918 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f75100f57b0 0x7f751010ae20 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7f7500002f70 tx=0x7f750000dc00).ready entity=mon.2 client_cookie=9442ca8d88f75906 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:29:05.692 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.703 7f750cb62700 1 -- 172.21.15.102:0/2203744918 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f75101a91e0 con 0x7f75100f57b0 2024-04-17T22:29:05.692 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.703 7f750cb62700 1 -- 172.21.15.102:0/2203744918 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f750000bc00 con 0x7f75100f57b0 2024-04-17T22:29:05.692 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.703 7f750cb62700 1 -- 172.21.15.102:0/2203744918 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f7500015070 con 0x7f75100f57b0 2024-04-17T22:29:05.692 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.703 7f7514eae700 1 -- 172.21.15.102:0/2203744918 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f75100f57b0 msgr2=0x7f751010ae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:05.692 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.703 7f7514eae700 1 --2- 172.21.15.102:0/2203744918 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f75100f57b0 0x7f751010ae20 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7f7500002f70 tx=0x7f750000dc00).stop 2024-04-17T22:29:05.692 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.703 7f7514eae700 1 -- 172.21.15.102:0/2203744918 shutdown_connections 2024-04-17T22:29:05.692 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.703 7f7514eae700 1 --2- 172.21.15.102:0/2203744918 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f75100f57b0 0x7f751010ae20 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:05.692 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.703 7f7514eae700 1 --2- 172.21.15.102:0/2203744918 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f751010b450 0x7f751019fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:05.692 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.703 7f7514eae700 1 -- 172.21.15.102:0/2203744918 >> 172.21.15.102:0/2203744918 conn(0x7f75101138d0 msgr2=0x7f75100f4fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:29:05.692 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.703 7f7514eae700 1 -- 172.21.15.102:0/2203744918 shutdown_connections 2024-04-17T22:29:05.693 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.703 7f7514eae700 1 -- 172.21.15.102:0/2203744918 wait complete. 2024-04-17T22:29:05.715 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.726 7f7514eae700 1 Processor -- start 2024-04-17T22:29:05.725 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.726 7f7514eae700 1 -- start start 2024-04-17T22:29:05.725 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.726 7f7514eae700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f75100d7640 0x7f75100ce290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:05.725 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.726 7f7514eae700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f75100ce8c0 0x7f75101aff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:05.725 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.726 7f7514eae700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f75100ced10 0x7f75101b0570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:05.725 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.726 7f7514eae700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75100d7070 con 0x7f75100d7640 2024-04-17T22:29:05.725 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.726 7f7514eae700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75100d6cf0 con 0x7f75100ced10 2024-04-17T22:29:05.725 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.726 7f7514eae700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f75101b0ba0 con 0x7f75100ce8c0 2024-04-17T22:29:05.725 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.726 7f750db64700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f75100d7640 0x7f75100ce290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:05.725 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.726 7f750d363700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f75100ce8c0 0x7f75101aff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:05.725 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.726 7f750e365700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f75100ced10 0x7f75101b0570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:05.726 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.727 7f750db64700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f75100d7640 0x7f75100ce290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:49128/0 (socket says 172.21.15.102:49128) 2024-04-17T22:29:05.726 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.727 7f750d363700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f75100ce8c0 0x7f75101aff40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:40898/0 (socket says 172.21.15.102:40898) 2024-04-17T22:29:05.726 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.727 7f750db64700 1 -- 172.21.15.102:0/699361729 learned_addr learned my addr 172.21.15.102:0/699361729 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:29:05.726 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.727 7f750e365700 1 -- 172.21.15.102:0/699361729 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f75100ce8c0 msgr2=0x7f75101aff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:05.726 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.727 7f750e365700 1 --2- 172.21.15.102:0/699361729 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f75100ce8c0 0x7f75101aff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:05.726 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.727 7f750e365700 1 -- 172.21.15.102:0/699361729 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f75100d7640 msgr2=0x7f75100ce290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:05.726 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.727 7f750e365700 1 --2- 172.21.15.102:0/699361729 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f75100d7640 0x7f75100ce290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:05.726 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.727 7f750e365700 1 -- 172.21.15.102:0/699361729 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75101b0d80 con 0x7f75100ced10 2024-04-17T22:29:05.726 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.727 7f750db64700 1 --2- 172.21.15.102:0/699361729 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f75100d7640 0x7f75100ce290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:29:05.726 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.727 7f750d363700 1 --2- 172.21.15.102:0/699361729 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f75100ce8c0 0x7f75101aff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:29:05.726 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.727 7f750e365700 1 --2- 172.21.15.102:0/699361729 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f75100ced10 0x7f75101b0570 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7f75100e0a50 tx=0x7f750800ef70).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:29:05.726 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.727 7f74feffd700 1 -- 172.21.15.102:0/699361729 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f75101b0d80 con 0x7f75100ced10 2024-04-17T22:29:05.726 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.727 7f74feffd700 1 -- 172.21.15.102:0/699361729 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7508005b90 con 0x7f75100ced10 2024-04-17T22:29:05.727 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.727 7f7514eae700 1 -- 172.21.15.102:0/699361729 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75101b7740 con 0x7f75100ced10 2024-04-17T22:29:05.727 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.727 7f74feffd700 1 -- 172.21.15.102:0/699361729 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f7508014070 con 0x7f75100ced10 2024-04-17T22:29:05.727 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.727 7f7514eae700 1 -- 172.21.15.102:0/699361729 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75101b7b20 con 0x7f75100ced10 2024-04-17T22:29:05.727 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.728 7f74feffd700 1 -- 172.21.15.102:0/699361729 <== mon.1 v2:172.21.15.102:3301/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7f75080070d0 con 0x7f75100ced10 2024-04-17T22:29:05.727 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.728 7f74feffd700 1 -- 172.21.15.102:0/699361729 <== mon.1 v2:172.21.15.102:3301/0 5 ==== osd_map(12..12 src has 1..12) v4 ==== 2374+0+0 (secure 0 0 0) 0x7f750800db00 con 0x7f75100ced10 2024-04-17T22:29:05.727 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.728 7f7514eae700 1 -- 172.21.15.102:0/699361729 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75100f0860 con 0x7f75100ced10 2024-04-17T22:29:05.727 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.731 7f74feffd700 1 -- 172.21.15.102:0/699361729 <== mon.1 v2:172.21.15.102:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+110455 (secure 0 0 0) 0x7f750800e0d0 con 0x7f75100ced10 2024-04-17T22:29:05.825 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.838 7f7514eae700 1 -- 172.21.15.102:0/699361729 >> v1:172.21.15.102:6816/42360 conn(0x7f74ec0145a0 legacy=0x7f74ec016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:05.825 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.838 7f7514eae700 1 -- 172.21.15.102:0/699361729 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f75100ced10 msgr2=0x7f75101b0570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:05.825 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.838 7f7514eae700 1 --2- 172.21.15.102:0/699361729 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f75100ced10 0x7f75101b0570 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7f75100e0a50 tx=0x7f750800ef70).stop 2024-04-17T22:29:05.826 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.838 7f7514eae700 1 -- 172.21.15.102:0/699361729 shutdown_connections 2024-04-17T22:29:05.826 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.838 7f7514eae700 1 --2- 172.21.15.102:0/699361729 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f75100ced10 0x7f75101b0570 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:05.826 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.838 7f7514eae700 1 --2- 172.21.15.102:0/699361729 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f75100d7640 0x7f75100ce290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:05.826 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.838 7f7514eae700 1 --2- 172.21.15.102:0/699361729 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f75100ce8c0 0x7f75101aff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:05.826 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.838 7f7514eae700 1 -- 172.21.15.102:0/699361729 >> 172.21.15.102:0/699361729 conn(0x7f75100c5b30 msgr2=0x7f75101a3fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:29:05.826 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.838 7f7514eae700 1 -- 172.21.15.102:0/699361729 shutdown_connections 2024-04-17T22:29:05.826 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.838 7f7514eae700 1 -- 172.21.15.102:0/699361729 wait complete. 2024-04-17T22:29:05.826 INFO:teuthology.orchestra.run.smithi102.stderr:Invalid command: missing required parameter pg_num() 2024-04-17T22:29:05.827 INFO:teuthology.orchestra.run.smithi102.stderr:osd pool create {} {replicated|erasure} {} {} {} {} {} {} {} : create pool 2024-04-17T22:29:05.827 INFO:teuthology.orchestra.run.smithi102.stderr:Error EINVAL: invalid command 2024-04-17T22:29:05.837 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-17T22:29:05.838 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata 64 2024-04-17T22:29:05.968 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.979 7fdced8a2700 1 Processor -- start 2024-04-17T22:29:05.968 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.979 7fdced8a2700 1 -- start start 2024-04-17T22:29:05.969 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.980 7fdced8a2700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fdce80f5840 0x7fdce810b010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:05.969 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.980 7fdced8a2700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fdce810b640 0x7fdce819faf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:05.969 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.980 7fdced8a2700 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdce8103b40 con 0x7fdce810ba90 2024-04-17T22:29:05.969 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.980 7fdced8a2700 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdce810bee0 con 0x7fdce80f5840 2024-04-17T22:29:05.969 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.980 7fdced8a2700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdce80fa370 con 0x7fdce810b640 2024-04-17T22:29:05.969 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.980 7fdce65c5700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fdce80f5840 0x7fdce810b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:05.969 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.980 7fdce6dc6700 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7fdce810ba90 0x7fdce81a0120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:46438/0 (socket says 172.21.15.102:46438) 2024-04-17T22:29:05.969 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.980 7fdce5dc4700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fdce810b640 0x7fdce819faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:05.969 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.980 7fdce65c5700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fdce80f5840 0x7fdce810b010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:49136/0 (socket says 172.21.15.102:49136) 2024-04-17T22:29:05.969 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.980 7fdce6dc6700 1 -- 172.21.15.102:0/260480210 learned_addr learned my addr 172.21.15.102:0/260480210 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-17T22:29:05.969 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.980 7fdce55c3700 1 -- 172.21.15.102:0/260480210 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1164587972 0 0) 0x7fdce0001490 con 0x7fdce810ba90 2024-04-17T22:29:05.969 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.980 7fdce55c3700 1 -- 172.21.15.102:0/260480210 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdccc003270 con 0x7fdce810ba90 2024-04-17T22:29:05.969 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.980 7fdce5dc4700 1 -- 172.21.15.102:0/260480210 >> v2:172.21.15.102:3300/0 conn(0x7fdce80f5840 msgr2=0x7fdce810b010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:29:05.970 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.980 7fdce5dc4700 1 --2- 172.21.15.102:0/260480210 >> v2:172.21.15.102:3300/0 conn(0x7fdce80f5840 0x7fdce810b010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:05.970 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.980 7fdce5dc4700 1 -- 172.21.15.102:0/260480210 >> v1:172.21.15.102:6789/0 conn(0x7fdce810ba90 legacy=0x7fdce81a0120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:05.970 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.980 7fdce5dc4700 1 -- 172.21.15.102:0/260480210 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdce81a0920 con 0x7fdce810b640 2024-04-17T22:29:05.970 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.980 7fdce65c5700 1 --2- 172.21.15.102:0/260480210 >> v2:172.21.15.102:3300/0 conn(0x7fdce80f5840 0x7fdce810b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:29:05.970 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.980 7fdce5dc4700 1 --2- 172.21.15.102:0/260480210 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fdce810b640 0x7fdce819faf0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7fdcd8002f70 tx=0x7fdcd800b970).ready entity=mon.1 client_cookie=b26095a9fc84054d server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:29:05.970 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.980 7fdce55c3700 1 -- 172.21.15.102:0/260480210 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fdce81a0920 con 0x7fdce810b640 2024-04-17T22:29:05.970 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.980 7fdce55c3700 1 -- 172.21.15.102:0/260480210 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcd800d5c0 con 0x7fdce810b640 2024-04-17T22:29:05.970 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.980 7fdce55c3700 1 -- 172.21.15.102:0/260480210 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fdcd800d990 con 0x7fdce810b640 2024-04-17T22:29:05.970 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.980 7fdced8a2700 1 -- 172.21.15.102:0/260480210 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fdce810b640 msgr2=0x7fdce819faf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:05.970 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.980 7fdced8a2700 1 --2- 172.21.15.102:0/260480210 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fdce810b640 0x7fdce819faf0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7fdcd8002f70 tx=0x7fdcd800b970).stop 2024-04-17T22:29:05.970 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.981 7fdced8a2700 1 -- 172.21.15.102:0/260480210 shutdown_connections 2024-04-17T22:29:05.970 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.981 7fdced8a2700 1 --2- 172.21.15.102:0/260480210 >> v2:172.21.15.102:3300/0 conn(0x7fdce80f5840 0x7fdce810b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:05.970 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.981 7fdced8a2700 1 --2- 172.21.15.102:0/260480210 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fdce810b640 0x7fdce819faf0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:05.971 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.981 7fdced8a2700 1 -- 172.21.15.102:0/260480210 >> 172.21.15.102:0/260480210 conn(0x7fdce81138a0 msgr2=0x7fdce80f4f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:29:05.971 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.981 7fdced8a2700 1 -- 172.21.15.102:0/260480210 shutdown_connections 2024-04-17T22:29:05.971 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:05.981 7fdced8a2700 1 -- 172.21.15.102:0/260480210 wait complete. 2024-04-17T22:29:05.992 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:06.003 7fdced8a2700 1 Processor -- start 2024-04-17T22:29:05.992 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:06.003 7fdced8a2700 1 -- start start 2024-04-17T22:29:05.992 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:06.004 7fdced8a2700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fdce80dada0 0x7fdce80d0200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:05.992 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:06.004 7fdced8a2700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdce80d0830 0x7fdce81b4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:05.992 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:06.004 7fdced8a2700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fdce80d0c80 0x7fdce81b4640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:05.992 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:06.004 7fdced8a2700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdce80e2c80 con 0x7fdce80d0830 2024-04-17T22:29:05.992 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:06.004 7fdced8a2700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdce80da6e0 con 0x7fdce80dada0 2024-04-17T22:29:05.993 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:06.004 7fdced8a2700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdce80d3140 con 0x7fdce80d0c80 2024-04-17T22:29:05.993 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:06.004 7fdce65c5700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fdce80dada0 0x7fdce80d0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:05.993 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:06.004 7fdce5dc4700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdce80d0830 0x7fdce81b4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:05.993 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:06.004 7fdce65c5700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fdce80dada0 0x7fdce80d0200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:44084/0 (socket says 172.21.15.102:44084) 2024-04-17T22:29:05.993 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:06.004 7fdce65c5700 1 -- 172.21.15.102:0/482009813 learned_addr learned my addr 172.21.15.102:0/482009813 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:29:05.993 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:06.004 7fdce6dc6700 1 --2- 172.21.15.102:0/482009813 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fdce80d0c80 0x7fdce81b4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:05.993 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:06.004 7fdce65c5700 1 -- 172.21.15.102:0/482009813 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fdce80d0c80 msgr2=0x7fdce81b4640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:05.993 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:06.004 7fdce65c5700 1 --2- 172.21.15.102:0/482009813 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fdce80d0c80 0x7fdce81b4640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:05.993 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:06.004 7fdce65c5700 1 -- 172.21.15.102:0/482009813 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdce80d0830 msgr2=0x7fdce81b4010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:05.993 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:06.004 7fdce65c5700 1 --2- 172.21.15.102:0/482009813 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdce80d0830 0x7fdce81b4010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:05.993 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:06.004 7fdce65c5700 1 -- 172.21.15.102:0/482009813 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdce81b4c70 con 0x7fdce80dada0 2024-04-17T22:29:05.993 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:06.004 7fdce5dc4700 1 --2- 172.21.15.102:0/482009813 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdce80d0830 0x7fdce81b4010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:29:05.993 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:06.004 7fdce65c5700 1 --2- 172.21.15.102:0/482009813 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fdce80dada0 0x7fdce80d0200 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7fdce80c5460 tx=0x7fdcdc00bf70).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:29:05.994 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:06.004 7fdcd77fe700 1 -- 172.21.15.102:0/482009813 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fdce81b4c70 con 0x7fdce80dada0 2024-04-17T22:29:05.994 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:06.004 7fdcd77fe700 1 -- 172.21.15.102:0/482009813 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcdc00c9a0 con 0x7fdce80dada0 2024-04-17T22:29:05.994 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:06.005 7fdcd77fe700 1 -- 172.21.15.102:0/482009813 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fdcdc00f040 con 0x7fdce80dada0 2024-04-17T22:29:05.994 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:06.005 7fdced8a2700 1 -- 172.21.15.102:0/482009813 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdce81bb780 con 0x7fdce80dada0 2024-04-17T22:29:05.994 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:06.005 7fdced8a2700 1 -- 172.21.15.102:0/482009813 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdce81bbb60 con 0x7fdce80dada0 2024-04-17T22:29:05.994 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:06.005 7fdcd77fe700 1 -- 172.21.15.102:0/482009813 <== mon.1 v2:172.21.15.102:3301/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7fdcdc011070 con 0x7fdce80dada0 2024-04-17T22:29:05.994 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:06.005 7fdcd77fe700 1 -- 172.21.15.102:0/482009813 <== mon.1 v2:172.21.15.102:3301/0 5 ==== osd_map(12..12 src has 1..12) v4 ==== 2374+0+0 (secure 0 0 0) 0x7fdcdc010040 con 0x7fdce80dada0 2024-04-17T22:29:05.994 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:06.006 7fdced8a2700 1 -- 172.21.15.102:0/482009813 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdce80f0800 con 0x7fdce80dada0 2024-04-17T22:29:05.995 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:06.009 7fdcd77fe700 1 -- 172.21.15.102:0/482009813 <== mon.1 v2:172.21.15.102:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+110455 (secure 0 0 0) 0x7fdcdc00c5a0 con 0x7fdce80dada0 2024-04-17T22:29:06.098 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:06.112 7fdced8a2700 1 -- 172.21.15.102:0/482009813 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 64} v 0) v1 -- 0x7fdce80efed0 con 0x7fdce80dada0 2024-04-17T22:29:06.420 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:06.433 7fdcd77fe700 1 -- 172.21.15.102:0/482009813 <== mon.1 v2:172.21.15.102:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 64}]=0 pool 'cephfs_metadata' created v13) v1 ==== 134+0+0 (secure 0 0 0) 0x7fdcdc000c30 con 0x7fdce80dada0 2024-04-17T22:29:06.425 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:06.438 7fdced8a2700 1 -- 172.21.15.102:0/482009813 >> v1:172.21.15.102:6816/42360 conn(0x7fdcc40145a0 legacy=0x7fdcc4016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:06.425 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:06.438 7fdced8a2700 1 -- 172.21.15.102:0/482009813 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fdce80dada0 msgr2=0x7fdce80d0200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:06.425 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:06.438 7fdced8a2700 1 --2- 172.21.15.102:0/482009813 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fdce80dada0 0x7fdce80d0200 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7fdce80c5460 tx=0x7fdcdc00bf70).stop 2024-04-17T22:29:06.425 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:06.439 7fdced8a2700 1 -- 172.21.15.102:0/482009813 shutdown_connections 2024-04-17T22:29:06.425 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:06.439 7fdced8a2700 1 --2- 172.21.15.102:0/482009813 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fdce80d0c80 0x7fdce81b4640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:06.426 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:06.439 7fdced8a2700 1 --2- 172.21.15.102:0/482009813 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fdce80dada0 0x7fdce80d0200 secure :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 rx=0x7fdce80c5460 tx=0x7fdcdc00bf70).stop 2024-04-17T22:29:06.426 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:06.439 7fdced8a2700 1 --2- 172.21.15.102:0/482009813 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdce80d0830 0x7fdce81b4010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:06.426 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:06.439 7fdced8a2700 1 -- 172.21.15.102:0/482009813 >> 172.21.15.102:0/482009813 conn(0x7fdce80e05c0 msgr2=0x7fdce80dc700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:29:06.426 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:06.440 7fdced8a2700 1 -- 172.21.15.102:0/482009813 shutdown_connections 2024-04-17T22:29:06.427 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:06.440 7fdced8a2700 1 -- 172.21.15.102:0/482009813 wait complete. 2024-04-17T22:29:06.430 INFO:teuthology.orchestra.run.smithi102.stderr:pool 'cephfs_metadata' created 2024-04-17T22:29:06.441 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 64 2024-04-17T22:29:06.572 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:06.583 7f952ea6e700 1 Processor -- start 2024-04-17T22:29:06.572 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:06.583 7f952ea6e700 1 -- start start 2024-04-17T22:29:06.572 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:06.583 7f952ea6e700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f95281050d0 0x7f952810ac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:06.572 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:06.583 7f952ea6e700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f95281a4ef0 0x7f95281a7350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:06.572 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:06.583 7f952ea6e700 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95281a07a0 con 0x7f95280f54c0 2024-04-17T22:29:06.572 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:06.583 7f952ea6e700 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f952810ba70 con 0x7f95281a4ef0 2024-04-17T22:29:06.572 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:06.583 7f952ea6e700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95281a8990 con 0x7f95281050d0 2024-04-17T22:29:06.572 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:06.583 7f9527fff700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f95281a4ef0 0x7f95281a7350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:06.572 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:06.583 7f95277fe700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f95281050d0 0x7f952810ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:06.572 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:06.583 7f9527fff700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f95281a4ef0 0x7f95281a7350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:49162/0 (socket says 172.21.15.102:49162) 2024-04-17T22:29:06.572 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:06.583 7f9527fff700 1 -- 172.21.15.102:0/2284083846 learned_addr learned my addr 172.21.15.102:0/2284083846 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:29:06.573 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:06.583 7f95277fe700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f95281050d0 0x7f952810ac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:44094/0 (socket says 172.21.15.102:44094) 2024-04-17T22:29:06.573 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:06.583 7f9526ffd700 1 --1- 172.21.15.102:0/2284083846 >> v1:172.21.15.102:6789/0 conn(0x7f95280f54c0 0x7f952810b270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:46442/0 (socket says 172.21.15.102:46442) 2024-04-17T22:29:06.573 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:06.584 7f9527fff700 1 -- 172.21.15.102:0/2284083846 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f95281050d0 msgr2=0x7f952810ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:29:06.573 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:06.584 7f9527fff700 1 --2- 172.21.15.102:0/2284083846 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f95281050d0 0x7f952810ac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:06.573 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:06.584 7f9527fff700 1 -- 172.21.15.102:0/2284083846 >> v1:172.21.15.102:6789/0 conn(0x7f95280f54c0 legacy=0x7f952810b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:06.573 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:06.584 7f9527fff700 1 -- 172.21.15.102:0/2284083846 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95281a8b40 con 0x7f95281a4ef0 2024-04-17T22:29:06.573 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:06.584 7f95267fc700 1 -- 172.21.15.102:0/2284083846 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3556835197 0 0) 0x7f951c001110 con 0x7f95280f54c0 2024-04-17T22:29:06.573 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:06.584 7f95277fe700 1 --2- 172.21.15.102:0/2284083846 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f95281050d0 0x7f952810ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:29:06.573 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:06.584 7f9527fff700 1 --2- 172.21.15.102:0/2284083846 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f95281a4ef0 0x7f95281a7350 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7f95180032f0 tx=0x7f9518009de0).ready entity=mon.0 client_cookie=fd338be59180f858 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:29:06.573 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:06.584 7f95267fc700 1 -- 172.21.15.102:0/2284083846 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f95281a8b40 con 0x7f95281a4ef0 2024-04-17T22:29:06.573 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:06.584 7f95267fc700 1 -- 172.21.15.102:0/2284083846 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f951800b8d0 con 0x7f95281a4ef0 2024-04-17T22:29:06.573 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:06.584 7f95267fc700 1 -- 172.21.15.102:0/2284083846 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f951800bab0 con 0x7f95281a4ef0 2024-04-17T22:29:06.573 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:06.584 7f952ea6e700 1 -- 172.21.15.102:0/2284083846 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f95281a4ef0 msgr2=0x7f95281a7350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:06.573 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:06.584 7f952ea6e700 1 --2- 172.21.15.102:0/2284083846 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f95281a4ef0 0x7f95281a7350 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7f95180032f0 tx=0x7f9518009de0).stop 2024-04-17T22:29:06.573 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:06.584 7f952ea6e700 1 -- 172.21.15.102:0/2284083846 shutdown_connections 2024-04-17T22:29:06.574 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:06.584 7f952ea6e700 1 --2- 172.21.15.102:0/2284083846 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f95281050d0 0x7f952810ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:06.574 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:06.584 7f952ea6e700 1 --2- 172.21.15.102:0/2284083846 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f95281a4ef0 0x7f95281a7350 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:06.574 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:06.584 7f952ea6e700 1 -- 172.21.15.102:0/2284083846 >> 172.21.15.102:0/2284083846 conn(0x7f95281135a0 msgr2=0x7f95280f4c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:29:06.574 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:06.585 7f952ea6e700 1 -- 172.21.15.102:0/2284083846 shutdown_connections 2024-04-17T22:29:06.574 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:06.585 7f952ea6e700 1 -- 172.21.15.102:0/2284083846 wait complete. 2024-04-17T22:29:06.593 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:06.607 7f952ea6e700 1 Processor -- start 2024-04-17T22:29:06.593 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:06.607 7f952ea6e700 1 -- start start 2024-04-17T22:29:06.596 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:06.607 7f952ea6e700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f95280f14b0 0x7f95280f1900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:06.596 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:06.607 7f952ea6e700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f95280f1f30 0x7f95280d2430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:06.597 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:06.607 7f952ea6e700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f95280d2a60 0x7f95281b8e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:06.597 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:06.607 7f952ea6e700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95280bd000 con 0x7f95280d2a60 2024-04-17T22:29:06.597 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:06.607 7f952ea6e700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95280d6b90 con 0x7f95280f14b0 2024-04-17T22:29:06.597 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:06.607 7f952ea6e700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f95280bfc20 con 0x7f95280f1f30 2024-04-17T22:29:06.597 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:06.608 7f95277fe700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f95280f14b0 0x7f95280f1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:06.597 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:06.608 7f9527fff700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f95280d2a60 0x7f95281b8e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:06.597 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:06.608 7f95277fe700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f95280f14b0 0x7f95280f1900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:44104/0 (socket says 172.21.15.102:44104) 2024-04-17T22:29:06.598 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:06.608 7f95277fe700 1 -- 172.21.15.102:0/1715466090 learned_addr learned my addr 172.21.15.102:0/1715466090 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:29:06.598 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:06.608 7f9526ffd700 1 --2- 172.21.15.102:0/1715466090 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f95280f1f30 0x7f95280d2430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:06.598 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:06.608 7f95277fe700 1 -- 172.21.15.102:0/1715466090 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f95280f1f30 msgr2=0x7f95280d2430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:06.598 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:06.608 7f95277fe700 1 --2- 172.21.15.102:0/1715466090 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f95280f1f30 0x7f95280d2430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:06.598 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:06.608 7f95277fe700 1 -- 172.21.15.102:0/1715466090 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f95280d2a60 msgr2=0x7f95281b8e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:06.598 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:06.608 7f95277fe700 1 --2- 172.21.15.102:0/1715466090 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f95280d2a60 0x7f95281b8e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:06.598 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:06.608 7f95277fe700 1 -- 172.21.15.102:0/1715466090 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95280bff10 con 0x7f95280f14b0 2024-04-17T22:29:06.598 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:06.608 7f9526ffd700 1 --2- 172.21.15.102:0/1715466090 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f95280f1f30 0x7f95280d2430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:29:06.599 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:06.608 7f9527fff700 1 --2- 172.21.15.102:0/1715466090 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f95280d2a60 0x7f95281b8e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:29:06.599 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:06.608 7f95277fe700 1 --2- 172.21.15.102:0/1715466090 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f95280f14b0 0x7f95280f1900 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7f95280c0380 tx=0x7f952000bf70).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:29:06.599 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:06.608 7f9524ff9700 1 -- 172.21.15.102:0/1715466090 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f95280bff10 con 0x7f95280f14b0 2024-04-17T22:29:06.599 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:06.608 7f9524ff9700 1 -- 172.21.15.102:0/1715466090 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f952000c940 con 0x7f95280f14b0 2024-04-17T22:29:06.599 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:06.608 7f952ea6e700 1 -- 172.21.15.102:0/1715466090 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95280d2eb0 con 0x7f95280f14b0 2024-04-17T22:29:06.599 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:06.608 7f9524ff9700 1 -- 172.21.15.102:0/1715466090 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f952000f040 con 0x7f95280f14b0 2024-04-17T22:29:06.599 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:06.608 7f952ea6e700 1 -- 172.21.15.102:0/1715466090 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95281b9490 con 0x7f95280f14b0 2024-04-17T22:29:06.599 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:06.609 7f9524ff9700 1 -- 172.21.15.102:0/1715466090 <== mon.1 v2:172.21.15.102:3301/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7f9520011070 con 0x7f95280f14b0 2024-04-17T22:29:06.599 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:06.609 7f9524ff9700 1 -- 172.21.15.102:0/1715466090 <== mon.1 v2:172.21.15.102:3301/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 2728+0+0 (secure 0 0 0) 0x7f9520010040 con 0x7f95280f14b0 2024-04-17T22:29:06.599 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:06.609 7f952ea6e700 1 -- 172.21.15.102:0/1715466090 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95280f0500 con 0x7f95280f14b0 2024-04-17T22:29:06.600 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:06.612 7f9524ff9700 1 -- 172.21.15.102:0/1715466090 <== mon.1 v2:172.21.15.102:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+110455 (secure 0 0 0) 0x7f952000c540 con 0x7f95280f14b0 2024-04-17T22:29:06.703 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:06.716 7f952ea6e700 1 -- 172.21.15.102:0/1715466090 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pgp_num": 64} v 0) v1 -- 0x7f95281abe90 con 0x7f95280f14b0 2024-04-17T22:29:07.432 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:07.445 7f9524ff9700 1 -- 172.21.15.102:0/1715466090 <== mon.1 v2:172.21.15.102:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pgp_num": 64}]=0 pool 'cephfs_data' created v14) v1 ==== 141+0+0 (secure 0 0 0) 0x7f9520000c30 con 0x7f95280f14b0 2024-04-17T22:29:07.458 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:07.472 7f952ea6e700 1 -- 172.21.15.102:0/1715466090 >> v1:172.21.15.102:6816/42360 conn(0x7f95080145a0 legacy=0x7f9508016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:07.459 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:07.472 7f952ea6e700 1 -- 172.21.15.102:0/1715466090 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f95280f14b0 msgr2=0x7f95280f1900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:07.460 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:07.472 7f952ea6e700 1 --2- 172.21.15.102:0/1715466090 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f95280f14b0 0x7f95280f1900 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7f95280c0380 tx=0x7f952000bf70).stop 2024-04-17T22:29:07.460 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:07.473 7f952ea6e700 1 -- 172.21.15.102:0/1715466090 shutdown_connections 2024-04-17T22:29:07.460 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:07.473 7f952ea6e700 1 --2- 172.21.15.102:0/1715466090 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f95280d2a60 0x7f95281b8e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:07.460 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:07.473 7f952ea6e700 1 --2- 172.21.15.102:0/1715466090 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f95280f14b0 0x7f95280f1900 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:07.460 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:07.473 7f952ea6e700 1 --2- 172.21.15.102:0/1715466090 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f95280f1f30 0x7f95280d2430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:07.460 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:07.473 7f952ea6e700 1 -- 172.21.15.102:0/1715466090 >> 172.21.15.102:0/1715466090 conn(0x7f95280c5890 msgr2=0x7f95280d92b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:29:07.461 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:07.474 7f952ea6e700 1 -- 172.21.15.102:0/1715466090 shutdown_connections 2024-04-17T22:29:07.461 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:07.475 7f952ea6e700 1 -- 172.21.15.102:0/1715466090 wait complete. 2024-04-17T22:29:07.467 INFO:teuthology.orchestra.run.smithi102.stderr:pool 'cephfs_data' created 2024-04-17T22:29:07.477 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-17T22:29:07.609 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:07.620 7f85ff5f2700 1 Processor -- start 2024-04-17T22:29:07.609 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:07.620 7f85ff5f2700 1 -- start start 2024-04-17T22:29:07.609 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:07.620 7f85ff5f2700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f85f810e240 0x7f85f810e690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:07.609 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:07.621 7f85ff5f2700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f85f810ecc0 0x7f85f80f1730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:07.609 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:07.621 7f85ff5f2700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f85f80f1d60 0x7f85f81aa4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:07.609 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:07.621 7f85ff5f2700 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85f80f9f10 con 0x7f85f810e240 2024-04-17T22:29:07.609 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:07.621 7f85ff5f2700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85f80f9030 con 0x7f85f810ecc0 2024-04-17T22:29:07.609 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:07.621 7f85ff5f2700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f85f80fa1b0 con 0x7f85f80f1d60 2024-04-17T22:29:07.609 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:07.621 7f85f77fe700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f85f810ecc0 0x7f85f80f1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:07.609 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:07.621 7f85fcbee700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f85f80f1d60 0x7f85f81aa4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:07.610 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:07.621 7f85f77fe700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f85f810ecc0 0x7f85f80f1730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:44118/0 (socket says 172.21.15.102:44118) 2024-04-17T22:29:07.610 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:07.621 7f85fcbee700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f85f80f1d60 0x7f85f81aa4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:40912/0 (socket says 172.21.15.102:40912) 2024-04-17T22:29:07.610 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:07.621 7f85f77fe700 1 -- 172.21.15.102:0/2456312902 learned_addr learned my addr 172.21.15.102:0/2456312902 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:29:07.610 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:07.621 7f85f7fff700 1 --2- 172.21.15.102:0/2456312902 >> v2:172.21.15.102:3300/0 conn(0x7f85f810e240 0x7f85f810e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:07.610 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:07.621 7f85fcbee700 1 -- 172.21.15.102:0/2456312902 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f85f810ecc0 msgr2=0x7f85f80f1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:29:07.610 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:07.621 7f85fcbee700 1 --2- 172.21.15.102:0/2456312902 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f85f810ecc0 0x7f85f80f1730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:07.610 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:07.621 7f85fcbee700 1 -- 172.21.15.102:0/2456312902 >> v2:172.21.15.102:3300/0 conn(0x7f85f810e240 msgr2=0x7f85f810e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:29:07.610 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:07.621 7f85fcbee700 1 --2- 172.21.15.102:0/2456312902 >> v2:172.21.15.102:3300/0 conn(0x7f85f810e240 0x7f85f810e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:07.610 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:07.621 7f85fcbee700 1 -- 172.21.15.102:0/2456312902 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85f80f21b0 con 0x7f85f80f1d60 2024-04-17T22:29:07.610 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:07.621 7f85f77fe700 1 --2- 172.21.15.102:0/2456312902 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f85f810ecc0 0x7f85f80f1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:29:07.610 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:07.621 7f85f7fff700 1 --2- 172.21.15.102:0/2456312902 >> v2:172.21.15.102:3300/0 conn(0x7f85f810e240 0x7f85f810e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:29:07.610 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:07.621 7f85fcbee700 1 --2- 172.21.15.102:0/2456312902 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f85f80f1d60 0x7f85f81aa4d0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7f85f80f46a0 tx=0x7f85f0009cf0).ready entity=mon.2 client_cookie=f3e4e31520612f61 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:29:07.610 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:07.621 7f85f6ffd700 1 -- 172.21.15.102:0/2456312902 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f85f80f21b0 con 0x7f85f80f1d60 2024-04-17T22:29:07.610 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:07.621 7f85f6ffd700 1 -- 172.21.15.102:0/2456312902 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85f000b800 con 0x7f85f80f1d60 2024-04-17T22:29:07.611 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:07.621 7f85f6ffd700 1 -- 172.21.15.102:0/2456312902 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f85f000b9e0 con 0x7f85f80f1d60 2024-04-17T22:29:07.611 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:07.621 7f85ff5f2700 1 -- 172.21.15.102:0/2456312902 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f85f80f1d60 msgr2=0x7f85f81aa4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:07.611 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:07.621 7f85ff5f2700 1 --2- 172.21.15.102:0/2456312902 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f85f80f1d60 0x7f85f81aa4d0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7f85f80f46a0 tx=0x7f85f0009cf0).stop 2024-04-17T22:29:07.611 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:07.622 7f85ff5f2700 1 -- 172.21.15.102:0/2456312902 shutdown_connections 2024-04-17T22:29:07.611 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:07.622 7f85ff5f2700 1 --2- 172.21.15.102:0/2456312902 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f85f80f1d60 0x7f85f81aa4d0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:07.611 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:07.622 7f85ff5f2700 1 --2- 172.21.15.102:0/2456312902 >> v2:172.21.15.102:3300/0 conn(0x7f85f810e240 0x7f85f810e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:07.611 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:07.622 7f85ff5f2700 1 --2- 172.21.15.102:0/2456312902 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f85f810ecc0 0x7f85f80f1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:07.611 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:07.622 7f85ff5f2700 1 -- 172.21.15.102:0/2456312902 >> 172.21.15.102:0/2456312902 conn(0x7f85f81138a0 msgr2=0x7f85f80f4f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:29:07.611 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:07.622 7f85ff5f2700 1 -- 172.21.15.102:0/2456312902 shutdown_connections 2024-04-17T22:29:07.611 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:07.622 7f85ff5f2700 1 -- 172.21.15.102:0/2456312902 wait complete. 2024-04-17T22:29:07.631 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:07.645 7f85ff5f2700 1 Processor -- start 2024-04-17T22:29:07.631 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:07.645 7f85ff5f2700 1 -- start start 2024-04-17T22:29:07.633 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:07.645 7f85ff5f2700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f85f80d60a0 0x7f85f80d64f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:07.633 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:07.645 7f85ff5f2700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f85f80d6b20 0x7f85f81b3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:07.633 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:07.645 7f85ff5f2700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f85f81b4620 0x7f85f81b4a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:07.633 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:07.645 7f85ff5f2700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85f81a8ee0 con 0x7f85f81b4620 2024-04-17T22:29:07.633 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:07.645 7f85ff5f2700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85f80dde20 con 0x7f85f80d60a0 2024-04-17T22:29:07.634 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:07.645 7f85ff5f2700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f85f81b50a0 con 0x7f85f80d6b20 2024-04-17T22:29:07.634 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:07.645 7f85f7fff700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f85f80d60a0 0x7f85f80d64f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:07.634 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:07.645 7f85f7fff700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f85f80d60a0 0x7f85f80d64f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:44132/0 (socket says 172.21.15.102:44132) 2024-04-17T22:29:07.634 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:07.645 7f85f7fff700 1 -- 172.21.15.102:0/339235289 learned_addr learned my addr 172.21.15.102:0/339235289 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:29:07.634 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:07.645 7f85fcbee700 1 --2- 172.21.15.102:0/339235289 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f85f81b4620 0x7f85f81b4a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:07.634 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:07.645 7f85f77fe700 1 --2- 172.21.15.102:0/339235289 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f85f80d6b20 0x7f85f81b3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:07.634 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:07.646 7f85f7fff700 1 -- 172.21.15.102:0/339235289 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f85f80d6b20 msgr2=0x7f85f81b3ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:07.634 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:07.646 7f85f7fff700 1 --2- 172.21.15.102:0/339235289 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f85f80d6b20 0x7f85f81b3ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:07.634 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:07.646 7f85f7fff700 1 -- 172.21.15.102:0/339235289 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f85f81b4620 msgr2=0x7f85f81b4a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:07.634 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:07.646 7f85f7fff700 1 --2- 172.21.15.102:0/339235289 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f85f81b4620 0x7f85f81b4a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:07.634 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:07.646 7f85f7fff700 1 -- 172.21.15.102:0/339235289 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85f81a89a0 con 0x7f85f80d60a0 2024-04-17T22:29:07.634 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:07.646 7f85f77fe700 1 --2- 172.21.15.102:0/339235289 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f85f80d6b20 0x7f85f81b3ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:29:07.634 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:07.646 7f85f7fff700 1 --2- 172.21.15.102:0/339235289 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f85f80d60a0 0x7f85f80d64f0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7f85f80c85f0 tx=0x7f85ec00bf70).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:29:07.634 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:07.646 7f85f57fa700 1 -- 172.21.15.102:0/339235289 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f85f81a89a0 con 0x7f85f80d60a0 2024-04-17T22:29:07.635 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:07.646 7f85f57fa700 1 -- 172.21.15.102:0/339235289 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85ec00c9a0 con 0x7f85f80d60a0 2024-04-17T22:29:07.635 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:07.646 7f85ff5f2700 1 -- 172.21.15.102:0/339235289 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85f80de050 con 0x7f85f80d60a0 2024-04-17T22:29:07.635 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:07.646 7f85f57fa700 1 -- 172.21.15.102:0/339235289 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f85ec00f040 con 0x7f85f80d60a0 2024-04-17T22:29:07.635 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:07.646 7f85ff5f2700 1 -- 172.21.15.102:0/339235289 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85f81bb8c0 con 0x7f85f80d60a0 2024-04-17T22:29:07.635 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:07.646 7f85f57fa700 1 -- 172.21.15.102:0/339235289 <== mon.1 v2:172.21.15.102:3301/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7f85ec011070 con 0x7f85f80d60a0 2024-04-17T22:29:07.635 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:07.646 7f85f57fa700 1 -- 172.21.15.102:0/339235289 <== mon.1 v2:172.21.15.102:3301/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 3078+0+0 (secure 0 0 0) 0x7f85ec010040 con 0x7f85f80d60a0 2024-04-17T22:29:07.635 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:07.647 7f85ff5f2700 1 -- 172.21.15.102:0/339235289 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85f81ae710 con 0x7f85f80d60a0 2024-04-17T22:29:07.636 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:07.650 7f85f57fa700 1 -- 172.21.15.102:0/339235289 <== mon.1 v2:172.21.15.102:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+110455 (secure 0 0 0) 0x7f85ec00c5a0 con 0x7f85f80d60a0 2024-04-17T22:29:07.772 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:07.785 7f85ff5f2700 1 -- 172.21.15.102:0/339235289 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f85f80a64e0 con 0x7f85f80d60a0 2024-04-17T22:29:07.776 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:07.790 7f85f57fa700 1 -- 172.21.15.102:0/339235289 <== mon.1 v2:172.21.15.102:3301/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v3) v1 ==== 172+0+0 (secure 0 0 0) 0x7f85ec000c30 con 0x7f85f80d60a0 2024-04-17T22:29:07.780 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:07.793 7f85ff5f2700 1 -- 172.21.15.102:0/339235289 >> v1:172.21.15.102:6816/42360 conn(0x7f85d80145a0 legacy=0x7f85d8016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:07.780 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:07.793 7f85ff5f2700 1 -- 172.21.15.102:0/339235289 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f85f80d60a0 msgr2=0x7f85f80d64f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:07.780 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:07.793 7f85ff5f2700 1 --2- 172.21.15.102:0/339235289 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f85f80d60a0 0x7f85f80d64f0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7f85f80c85f0 tx=0x7f85ec00bf70).stop 2024-04-17T22:29:07.784 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:07.797 7f85ff5f2700 1 -- 172.21.15.102:0/339235289 shutdown_connections 2024-04-17T22:29:07.784 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:07.797 7f85ff5f2700 1 --2- 172.21.15.102:0/339235289 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f85f81b4620 0x7f85f81b4a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:07.784 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:07.797 7f85ff5f2700 1 --2- 172.21.15.102:0/339235289 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f85f80d60a0 0x7f85f80d64f0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:07.784 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:07.797 7f85ff5f2700 1 --2- 172.21.15.102:0/339235289 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f85f80d6b20 0x7f85f81b3ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:07.784 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:07.798 7f85ff5f2700 1 -- 172.21.15.102:0/339235289 >> 172.21.15.102:0/339235289 conn(0x7f85f80c6a60 msgr2=0x7f85f80c3020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:29:07.785 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:07.798 7f85ff5f2700 1 -- 172.21.15.102:0/339235289 shutdown_connections 2024-04-17T22:29:07.785 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:07.798 7f85ff5f2700 1 -- 172.21.15.102:0/339235289 wait complete. 2024-04-17T22:29:07.788 INFO:teuthology.orchestra.run.smithi102.stderr:new fs with metadata pool 2 and data pool 3 2024-04-17T22:29:07.804 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T22:29:07.937 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:07.948 7fae02f2c700 1 Processor -- start 2024-04-17T22:29:07.938 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:07.948 7fae02f2c700 1 -- start start 2024-04-17T22:29:07.938 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:07.949 7fae02f2c700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fadfc1a64d0 0x7fadfc1a8930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:07.938 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:07.949 7fae02f2c700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fadfc1a8f60 0x7fadfc1ab3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:07.938 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:07.949 7fae02f2c700 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fadfc10eeb0 con 0x7fadfc10e260 2024-04-17T22:29:07.938 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:07.949 7fae02f2c700 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fadfc0f9f30 con 0x7fadfc1a64d0 2024-04-17T22:29:07.938 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:07.949 7fae02f2c700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fadfc0f9050 con 0x7fadfc1a8f60 2024-04-17T22:29:07.938 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:07.949 7fadfaffd700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fadfc1a64d0 0x7fadfc1a8930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:07.938 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:07.949 7fadfaffd700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fadfc1a64d0 0x7fadfc1a8930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:49206/0 (socket says 172.21.15.102:49206) 2024-04-17T22:29:07.938 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:07.949 7fadfaffd700 1 -- 172.21.15.102:0/2632453517 learned_addr learned my addr 172.21.15.102:0/2632453517 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:29:07.938 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:07.949 7fadfb7fe700 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7fadfc10e260 0x7fadfc10e6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:46448/0 (socket says 172.21.15.102:46448) 2024-04-17T22:29:07.938 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:07.949 7fadfbfff700 1 --2- 172.21.15.102:0/2632453517 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fadfc1a8f60 0x7fadfc1ab3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:07.938 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:07.949 7fadfbfff700 1 -- 172.21.15.102:0/2632453517 >> v2:172.21.15.102:3300/0 conn(0x7fadfc1a64d0 msgr2=0x7fadfc1a8930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:29:07.938 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:07.949 7fadfbfff700 1 --2- 172.21.15.102:0/2632453517 >> v2:172.21.15.102:3300/0 conn(0x7fadfc1a64d0 0x7fadfc1a8930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:07.939 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:07.949 7fadfbfff700 1 -- 172.21.15.102:0/2632453517 >> v1:172.21.15.102:6789/0 conn(0x7fadfc10e260 legacy=0x7fadfc10e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:07.939 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:07.949 7fadfbfff700 1 -- 172.21.15.102:0/2632453517 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadfc1aca00 con 0x7fadfc1a8f60 2024-04-17T22:29:07.939 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:07.949 7fadfa7fc700 1 -- 172.21.15.102:0/2632453517 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2001092306 0 0) 0x7fadec001110 con 0x7fadfc10e260 2024-04-17T22:29:07.939 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:07.949 7fadfaffd700 1 --2- 172.21.15.102:0/2632453517 >> v2:172.21.15.102:3300/0 conn(0x7fadfc1a64d0 0x7fadfc1a8930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:29:07.939 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:07.949 7fadfbfff700 1 --2- 172.21.15.102:0/2632453517 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fadfc1a8f60 0x7fadfc1ab3c0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7fadfc0f4490 tx=0x7fadf4009cc0).ready entity=mon.2 client_cookie=2848f309f9750b83 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:29:07.939 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:07.949 7fadfa7fc700 1 -- 172.21.15.102:0/2632453517 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fadfc1aca00 con 0x7fadfc1a8f60 2024-04-17T22:29:07.939 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:07.949 7fadfa7fc700 1 -- 172.21.15.102:0/2632453517 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadf400b840 con 0x7fadfc1a8f60 2024-04-17T22:29:07.939 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:07.949 7fadfa7fc700 1 -- 172.21.15.102:0/2632453517 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fadf400ba20 con 0x7fadfc1a8f60 2024-04-17T22:29:07.939 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:07.949 7fae02f2c700 1 -- 172.21.15.102:0/2632453517 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fadfc1a8f60 msgr2=0x7fadfc1ab3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:07.939 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:07.949 7fae02f2c700 1 --2- 172.21.15.102:0/2632453517 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fadfc1a8f60 0x7fadfc1ab3c0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7fadfc0f4490 tx=0x7fadf4009cc0).stop 2024-04-17T22:29:07.939 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:07.950 7fae02f2c700 1 -- 172.21.15.102:0/2632453517 shutdown_connections 2024-04-17T22:29:07.939 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:07.950 7fae02f2c700 1 --2- 172.21.15.102:0/2632453517 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fadfc1a8f60 0x7fadfc1ab3c0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:07.939 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:07.950 7fae02f2c700 1 --2- 172.21.15.102:0/2632453517 >> v2:172.21.15.102:3300/0 conn(0x7fadfc1a64d0 0x7fadfc1a8930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:07.939 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:07.950 7fae02f2c700 1 -- 172.21.15.102:0/2632453517 >> 172.21.15.102:0/2632453517 conn(0x7fadfc1138c0 msgr2=0x7fadfc0f4f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:29:07.940 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:07.950 7fae02f2c700 1 -- 172.21.15.102:0/2632453517 shutdown_connections 2024-04-17T22:29:07.940 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:07.950 7fae02f2c700 1 -- 172.21.15.102:0/2632453517 wait complete. 2024-04-17T22:29:07.963 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:07.973 7fae02f2c700 1 Processor -- start 2024-04-17T22:29:07.963 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:07.973 7fae02f2c700 1 -- start start 2024-04-17T22:29:07.964 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:07.974 7fae02f2c700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fadfc0cde40 0x7fadfc0d6370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:07.964 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:07.974 7fae02f2c700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fadfc0d69a0 0x7fadfc1b3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:07.964 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:07.974 7fae02f2c700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fadfc1b4520 0x7fadfc1b4970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:07.964 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:07.974 7fae02f2c700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadfc08f2e0 con 0x7fadfc0cde40 2024-04-17T22:29:07.964 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:07.974 7fae02f2c700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadfc0fa1d0 con 0x7fadfc0d69a0 2024-04-17T22:29:07.964 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:07.974 7fae02f2c700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fadfc0d6df0 con 0x7fadfc1b4520 2024-04-17T22:29:07.964 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:07.974 7fadfaffd700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fadfc0d69a0 0x7fadfc1b3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:07.964 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:07.974 7fadfb7fe700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fadfc0cde40 0x7fadfc0d6370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:07.964 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:07.974 7fadfaffd700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fadfc0d69a0 0x7fadfc1b3ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:44138/0 (socket says 172.21.15.102:44138) 2024-04-17T22:29:07.964 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:07.974 7fadfaffd700 1 -- 172.21.15.102:0/1441896588 learned_addr learned my addr 172.21.15.102:0/1441896588 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:29:07.964 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:07.974 7fadfbfff700 1 --2- 172.21.15.102:0/1441896588 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fadfc1b4520 0x7fadfc1b4970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:07.964 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:07.974 7fadfb7fe700 1 -- 172.21.15.102:0/1441896588 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fadfc1b4520 msgr2=0x7fadfc1b4970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:07.964 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:07.974 7fadfb7fe700 1 --2- 172.21.15.102:0/1441896588 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fadfc1b4520 0x7fadfc1b4970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:07.964 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:07.974 7fadfb7fe700 1 -- 172.21.15.102:0/1441896588 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fadfc0d69a0 msgr2=0x7fadfc1b3ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:07.964 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:07.974 7fadfb7fe700 1 --2- 172.21.15.102:0/1441896588 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fadfc0d69a0 0x7fadfc1b3ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:07.964 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:07.974 7fadfb7fe700 1 -- 172.21.15.102:0/1441896588 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadfc0c4470 con 0x7fadfc0cde40 2024-04-17T22:29:07.965 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:07.974 7fadfaffd700 1 --2- 172.21.15.102:0/1441896588 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fadfc0d69a0 0x7fadfc1b3ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:29:07.965 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:07.974 7fadfb7fe700 1 --2- 172.21.15.102:0/1441896588 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fadfc0cde40 0x7fadfc0d6370 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7fadec000bc0 tx=0x7fadec00c800).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:29:07.965 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:07.975 7fadf8ff9700 1 -- 172.21.15.102:0/1441896588 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fadec0070d0 con 0x7fadfc0cde40 2024-04-17T22:29:07.965 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:07.975 7fadf8ff9700 1 -- 172.21.15.102:0/1441896588 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadec0072d0 con 0x7fadfc0cde40 2024-04-17T22:29:07.965 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:07.975 7fae02f2c700 1 -- 172.21.15.102:0/1441896588 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadfc1bb3d0 con 0x7fadfc0cde40 2024-04-17T22:29:07.965 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:07.975 7fadf8ff9700 1 -- 172.21.15.102:0/1441896588 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fadec005750 con 0x7fadfc0cde40 2024-04-17T22:29:07.965 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:07.975 7fae02f2c700 1 -- 172.21.15.102:0/1441896588 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadfc1bb7b0 con 0x7fadfc0cde40 2024-04-17T22:29:07.965 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:07.975 7fadf8ff9700 1 -- 172.21.15.102:0/1441896588 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7fadec005950 con 0x7fadfc0cde40 2024-04-17T22:29:07.965 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:07.975 7fadf8ff9700 1 -- 172.21.15.102:0/1441896588 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 3194+0+0 (secure 0 0 0) 0x7fadec014a10 con 0x7fadfc0cde40 2024-04-17T22:29:07.965 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:07.976 7fae02f2c700 1 -- 172.21.15.102:0/1441896588 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fadfc0f0820 con 0x7fadfc0cde40 2024-04-17T22:29:07.965 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:07.979 7fadf8ff9700 1 -- 172.21.15.102:0/1441896588 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+110455 (secure 0 0 0) 0x7fadfc0c4470 con 0x7fadfc0cde40 2024-04-17T22:29:08.069 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.083 7fae02f2c700 1 -- 172.21.15.102:0/1441896588 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fadfc1ae5f0 con 0x7fadfc0cde40 2024-04-17T22:29:08.070 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.083 7fadf8ff9700 1 -- 172.21.15.102:0/1441896588 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v15) v1 ==== 74+0+8774 (secure 0 0 0) 0x7fadec000c30 con 0x7fadfc0cde40 2024-04-17T22:29:08.070 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-17T22:29:08.070 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":15,"fsid":"cb63e789-091d-49b1-b120-55f9db526871","created":"2024-04-17 22:28:48.209635","modified":"2024-04-17 22:29:07.787507","last_up_change":"2024-04-17 22:28:54.236215","last_in_change":"2024-04-17 22:28:54.236215","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":4,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-04-17 22:28:57.478545","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-17 22:29:06.144708","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-17 22:29:06.748924","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3351ef85-ade4-4b90-a8f4-213f4e00ba01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6812","nonce":42955}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6813","nonce":42955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6815","nonce":42955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6814","nonce":42955}]},"public_addr":"172.21.15.102:6812/42955","cluster_addr":"172.21.15.102:6813/42955","heartbeat_back_addr":"172.21.15.102:6815/42955","heartbeat_front_addr":"172.21.15.102:6814/42955","state":["exists","up"]},{"osd":1,"uuid":"04f67eac-f36b-48f5-982d-4cfc776596d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6808","nonce":42958}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6809","nonce":42958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6811","nonce":42958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6810","nonce":42958}]},"public_addr":"172.21.15.102:6808/42958","cluster_addr":"172.21.15.102:6809/42958","heartbeat_back_addr":"172.21.15.102:6811/42958","heartbeat_front_addr":"172.21.15.102:6810/42958","state":["exists","up"]},{"osd":2,"uuid":"d524204b-70b2-40a8-8f73-f9e7d44318af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6804","nonce":42954}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6805","nonce":42954}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6807","nonce":42954}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6806","nonce":42954}]},"public_addr":"172.21.15.102:6804/42954","cluster_addr":"172.21.15.102:6805/42954","heartbeat_back_addr":"172.21.15.102:6807/42954","heartbeat_front_addr":"172.21.15.102:6806/42954","state":["exists","up"]},{"osd":3,"uuid":"4fc9d20c-1ad0-484f-aafa-2c9a7574245b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6800","nonce":42953}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6801","nonce":42953}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6803","nonce":42953}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6802","nonce":42953}]},"public_addr":"172.21.15.102:6800/42953","cluster_addr":"172.21.15.102:6801/42953","heartbeat_back_addr":"172.21.15.102:6803/42953","heartbeat_front_addr":"172.21.15.102:6802/42953","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2024-04-17T22:29:08.072 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.085 7fae02f2c700 1 -- 172.21.15.102:0/1441896588 >> v1:172.21.15.102:6816/42360 conn(0x7fade00145a0 legacy=0x7fade0016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:08.072 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.085 7fae02f2c700 1 -- 172.21.15.102:0/1441896588 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fadfc0cde40 msgr2=0x7fadfc0d6370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:08.072 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.085 7fae02f2c700 1 --2- 172.21.15.102:0/1441896588 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fadfc0cde40 0x7fadfc0d6370 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7fadec000bc0 tx=0x7fadec00c800).stop 2024-04-17T22:29:08.072 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.085 7fae02f2c700 1 -- 172.21.15.102:0/1441896588 shutdown_connections 2024-04-17T22:29:08.072 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.085 7fae02f2c700 1 --2- 172.21.15.102:0/1441896588 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fadfc1b4520 0x7fadfc1b4970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:08.072 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.085 7fae02f2c700 1 --2- 172.21.15.102:0/1441896588 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fadfc0cde40 0x7fadfc0d6370 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:08.072 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.085 7fae02f2c700 1 --2- 172.21.15.102:0/1441896588 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fadfc0d69a0 0x7fadfc1b3ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:08.072 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.085 7fae02f2c700 1 -- 172.21.15.102:0/1441896588 >> 172.21.15.102:0/1441896588 conn(0x7fadfc0c3840 msgr2=0x7fadfc0dadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:29:08.072 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.085 7fae02f2c700 1 -- 172.21.15.102:0/1441896588 shutdown_connections 2024-04-17T22:29:08.072 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.085 7fae02f2c700 1 -- 172.21.15.102:0/1441896588 wait complete. 2024-04-17T22:29:08.082 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T22:29:08.209 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.221 7f4687a4f700 1 Processor -- start 2024-04-17T22:29:08.209 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.221 7f4687a4f700 1 -- start start 2024-04-17T22:29:08.209 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.221 7f4687a4f700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f468010e260 0x7f468010e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:08.209 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.221 7f4687a4f700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f468010ece0 0x7f46800f1750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:08.210 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.221 7f4687a4f700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f46800f1d80 0x7f46801aa4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:08.210 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.221 7f4687a4f700 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46800f9f30 con 0x7f468010ece0 2024-04-17T22:29:08.210 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.221 7f4687a4f700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46800f9050 con 0x7f46800f1d80 2024-04-17T22:29:08.210 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.221 7f4687a4f700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f46800fa1d0 con 0x7f468010e260 2024-04-17T22:29:08.210 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.221 7f467ffff700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f468010ece0 0x7f46800f1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:08.210 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.221 7f468484a700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f468010e260 0x7f468010e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:08.210 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.221 7f468504b700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f46800f1d80 0x7f46801aa4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:08.210 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.221 7f468484a700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f468010e260 0x7f468010e6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:40942/0 (socket says 172.21.15.102:40942) 2024-04-17T22:29:08.210 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.221 7f467ffff700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f468010ece0 0x7f46800f1750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:49214/0 (socket says 172.21.15.102:49214) 2024-04-17T22:29:08.210 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.221 7f468484a700 1 -- 172.21.15.102:0/2815856542 learned_addr learned my addr 172.21.15.102:0/2815856542 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:29:08.210 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.222 7f467ffff700 1 -- 172.21.15.102:0/2815856542 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f468010e260 msgr2=0x7f468010e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:29:08.210 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.222 7f467ffff700 1 --2- 172.21.15.102:0/2815856542 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f468010e260 0x7f468010e6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:08.210 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.222 7f467ffff700 1 -- 172.21.15.102:0/2815856542 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f46800f1d80 msgr2=0x7f46801aa4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:29:08.211 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.222 7f467ffff700 1 --2- 172.21.15.102:0/2815856542 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f46800f1d80 0x7f46801aa4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:08.211 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.222 7f467ffff700 1 -- 172.21.15.102:0/2815856542 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46800f21d0 con 0x7f468010ece0 2024-04-17T22:29:08.211 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.222 7f468484a700 1 --2- 172.21.15.102:0/2815856542 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f468010e260 0x7f468010e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:29:08.211 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.222 7f468504b700 1 --2- 172.21.15.102:0/2815856542 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f46800f1d80 0x7f46801aa4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:29:08.211 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.222 7f467ffff700 1 --2- 172.21.15.102:0/2815856542 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f468010ece0 0x7f46800f1750 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0x7f46800f46c0 tx=0x7f466c00ba50).ready entity=mon.0 client_cookie=16a8fde786c73450 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:29:08.211 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.222 7f467f7fe700 1 -- 172.21.15.102:0/2815856542 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f46800f21d0 con 0x7f468010ece0 2024-04-17T22:29:08.211 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.222 7f467f7fe700 1 -- 172.21.15.102:0/2815856542 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f466c00d570 con 0x7f468010ece0 2024-04-17T22:29:08.211 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.222 7f467f7fe700 1 -- 172.21.15.102:0/2815856542 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f466c00d940 con 0x7f468010ece0 2024-04-17T22:29:08.211 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.222 7f4687a4f700 1 -- 172.21.15.102:0/2815856542 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f468010ece0 msgr2=0x7f46800f1750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:08.211 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.222 7f4687a4f700 1 --2- 172.21.15.102:0/2815856542 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f468010ece0 0x7f46800f1750 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0x7f46800f46c0 tx=0x7f466c00ba50).stop 2024-04-17T22:29:08.211 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.222 7f4687a4f700 1 -- 172.21.15.102:0/2815856542 shutdown_connections 2024-04-17T22:29:08.211 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.222 7f4687a4f700 1 --2- 172.21.15.102:0/2815856542 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f46800f1d80 0x7f46801aa4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:08.211 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.222 7f4687a4f700 1 --2- 172.21.15.102:0/2815856542 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f468010e260 0x7f468010e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:08.211 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.222 7f4687a4f700 1 --2- 172.21.15.102:0/2815856542 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f468010ece0 0x7f46800f1750 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:08.212 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.222 7f4687a4f700 1 -- 172.21.15.102:0/2815856542 >> 172.21.15.102:0/2815856542 conn(0x7f46801138c0 msgr2=0x7f46800f4f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:29:08.212 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.222 7f4687a4f700 1 -- 172.21.15.102:0/2815856542 shutdown_connections 2024-04-17T22:29:08.212 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.223 7f4687a4f700 1 -- 172.21.15.102:0/2815856542 wait complete. 2024-04-17T22:29:08.232 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.245 7f4687a4f700 1 Processor -- start 2024-04-17T22:29:08.232 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.245 7f4687a4f700 1 -- start start 2024-04-17T22:29:08.234 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.246 7f4687a4f700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f46801ad040 0x7f46801ad490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:08.234 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.246 7f4687a4f700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f46801adac0 0x7f46801b3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:08.234 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.246 7f4687a4f700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f46801b41d0 0x7f46801b4620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:08.234 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.246 7f4687a4f700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f468008f2e0 con 0x7f46801ad040 2024-04-17T22:29:08.234 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.246 7f4687a4f700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46800de1f0 con 0x7f46801b41d0 2024-04-17T22:29:08.234 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.246 7f4687a4f700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f46800c40f0 con 0x7f46801adac0 2024-04-17T22:29:08.234 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.246 7f467ffff700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f46801adac0 0x7f46801b3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:08.235 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.246 7f468484a700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f46801ad040 0x7f46801ad490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:08.235 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.246 7f468504b700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f46801b41d0 0x7f46801b4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:08.235 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.246 7f467ffff700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f46801adac0 0x7f46801b3ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:40950/0 (socket says 172.21.15.102:40950) 2024-04-17T22:29:08.235 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.246 7f467ffff700 1 -- 172.21.15.102:0/2923248310 learned_addr learned my addr 172.21.15.102:0/2923248310 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:29:08.235 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.246 7f468484a700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f46801ad040 0x7f46801ad490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:49218/0 (socket says 172.21.15.102:49218) 2024-04-17T22:29:08.235 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.246 7f467ffff700 1 -- 172.21.15.102:0/2923248310 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f46801b41d0 msgr2=0x7f46801b4620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:08.235 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.246 7f467ffff700 1 --2- 172.21.15.102:0/2923248310 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f46801b41d0 0x7f46801b4620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:08.235 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.246 7f467ffff700 1 -- 172.21.15.102:0/2923248310 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f46801ad040 msgr2=0x7f46801ad490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:08.235 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.246 7f467ffff700 1 --2- 172.21.15.102:0/2923248310 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f46801ad040 0x7f46801ad490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:08.235 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.246 7f467ffff700 1 -- 172.21.15.102:0/2923248310 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46801a6170 con 0x7f46801adac0 2024-04-17T22:29:08.235 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.246 7f468504b700 1 --2- 172.21.15.102:0/2923248310 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f46801b41d0 0x7f46801b4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:29:08.235 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.246 7f467ffff700 1 --2- 172.21.15.102:0/2923248310 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f46801adac0 0x7f46801b3ba0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7f46800c46b0 tx=0x7f466c0068c0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:29:08.235 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.246 7f467dffb700 1 -- 172.21.15.102:0/2923248310 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f46801a6170 con 0x7f46801adac0 2024-04-17T22:29:08.235 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.246 7f467dffb700 1 -- 172.21.15.102:0/2923248310 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f466c011610 con 0x7f46801adac0 2024-04-17T22:29:08.236 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.246 7f4687a4f700 1 -- 172.21.15.102:0/2923248310 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46800dfda0 con 0x7f46801adac0 2024-04-17T22:29:08.236 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.246 7f467dffb700 1 -- 172.21.15.102:0/2923248310 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f466c017040 con 0x7f46801adac0 2024-04-17T22:29:08.236 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.246 7f4687a4f700 1 -- 172.21.15.102:0/2923248310 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46801bb470 con 0x7f46801adac0 2024-04-17T22:29:08.236 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.247 7f467dffb700 1 -- 172.21.15.102:0/2923248310 <== mon.2 v2:172.21.15.102:3302/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7f466c0119e0 con 0x7f46801adac0 2024-04-17T22:29:08.236 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.247 7f467dffb700 1 -- 172.21.15.102:0/2923248310 <== mon.2 v2:172.21.15.102:3302/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 3194+0+0 (secure 0 0 0) 0x7f466c018070 con 0x7f46801adac0 2024-04-17T22:29:08.236 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.247 7f4687a4f700 1 -- 172.21.15.102:0/2923248310 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46800f0820 con 0x7f46801adac0 2024-04-17T22:29:08.237 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.251 7f467dffb700 1 -- 172.21.15.102:0/2923248310 <== mon.2 v2:172.21.15.102:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+110455 (secure 0 0 0) 0x7f466c011400 con 0x7f46801adac0 2024-04-17T22:29:08.342 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.356 7f4687a4f700 1 -- 172.21.15.102:0/2923248310 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f46801ae2c0 con 0x7f46801adac0 2024-04-17T22:29:08.343 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.356 7f467dffb700 1 -- 172.21.15.102:0/2923248310 <== mon.2 v2:172.21.15.102:3302/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v15) v1 ==== 74+0+8774 (secure 0 0 0) 0x7f466c00d360 con 0x7f46801adac0 2024-04-17T22:29:08.343 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-17T22:29:08.343 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":15,"fsid":"cb63e789-091d-49b1-b120-55f9db526871","created":"2024-04-17 22:28:48.209635","modified":"2024-04-17 22:29:07.787507","last_up_change":"2024-04-17 22:28:54.236215","last_in_change":"2024-04-17 22:28:54.236215","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":4,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-04-17 22:28:57.478545","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-17 22:29:06.144708","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-17 22:29:06.748924","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3351ef85-ade4-4b90-a8f4-213f4e00ba01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6812","nonce":42955}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6813","nonce":42955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6815","nonce":42955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6814","nonce":42955}]},"public_addr":"172.21.15.102:6812/42955","cluster_addr":"172.21.15.102:6813/42955","heartbeat_back_addr":"172.21.15.102:6815/42955","heartbeat_front_addr":"172.21.15.102:6814/42955","state":["exists","up"]},{"osd":1,"uuid":"04f67eac-f36b-48f5-982d-4cfc776596d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6808","nonce":42958}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6809","nonce":42958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6811","nonce":42958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6810","nonce":42958}]},"public_addr":"172.21.15.102:6808/42958","cluster_addr":"172.21.15.102:6809/42958","heartbeat_back_addr":"172.21.15.102:6811/42958","heartbeat_front_addr":"172.21.15.102:6810/42958","state":["exists","up"]},{"osd":2,"uuid":"d524204b-70b2-40a8-8f73-f9e7d44318af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6804","nonce":42954}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6805","nonce":42954}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6807","nonce":42954}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6806","nonce":42954}]},"public_addr":"172.21.15.102:6804/42954","cluster_addr":"172.21.15.102:6805/42954","heartbeat_back_addr":"172.21.15.102:6807/42954","heartbeat_front_addr":"172.21.15.102:6806/42954","state":["exists","up"]},{"osd":3,"uuid":"4fc9d20c-1ad0-484f-aafa-2c9a7574245b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6800","nonce":42953}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6801","nonce":42953}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6803","nonce":42953}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6802","nonce":42953}]},"public_addr":"172.21.15.102:6800/42953","cluster_addr":"172.21.15.102:6801/42953","heartbeat_back_addr":"172.21.15.102:6803/42953","heartbeat_front_addr":"172.21.15.102:6802/42953","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2024-04-17T22:29:08.344 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.357 7f4687a4f700 1 -- 172.21.15.102:0/2923248310 >> v1:172.21.15.102:6816/42360 conn(0x7f46680145a0 legacy=0x7f4668016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:08.344 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.357 7f4687a4f700 1 -- 172.21.15.102:0/2923248310 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f46801adac0 msgr2=0x7f46801b3ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:08.344 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.357 7f4687a4f700 1 --2- 172.21.15.102:0/2923248310 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f46801adac0 0x7f46801b3ba0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7f46800c46b0 tx=0x7f466c0068c0).stop 2024-04-17T22:29:08.344 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.357 7f4687a4f700 1 -- 172.21.15.102:0/2923248310 shutdown_connections 2024-04-17T22:29:08.345 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.357 7f4687a4f700 1 --2- 172.21.15.102:0/2923248310 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f46801b41d0 0x7f46801b4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:08.345 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.357 7f4687a4f700 1 --2- 172.21.15.102:0/2923248310 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f46801ad040 0x7f46801ad490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:08.345 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.357 7f4687a4f700 1 --2- 172.21.15.102:0/2923248310 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f46801adac0 0x7f46801b3ba0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:08.345 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.357 7f4687a4f700 1 -- 172.21.15.102:0/2923248310 >> 172.21.15.102:0/2923248310 conn(0x7f46800c5140 msgr2=0x7f46800d6900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:29:08.345 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.358 7f4687a4f700 1 -- 172.21.15.102:0/2923248310 shutdown_connections 2024-04-17T22:29:08.345 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.358 7f4687a4f700 1 -- 172.21.15.102:0/2923248310 wait complete. 2024-04-17T22:29:08.355 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-17T22:29:08.485 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.496 7fd915ad6700 1 Processor -- start 2024-04-17T22:29:08.485 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.496 7fd915ad6700 1 -- start start 2024-04-17T22:29:08.485 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.497 7fd915ad6700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fd91010e240 0x7fd91010e690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:08.485 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.497 7fd915ad6700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd91010ecc0 0x7fd9100f1730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:08.485 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.497 7fd915ad6700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd9100f1d60 0x7fd9101aa4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:08.485 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.497 7fd915ad6700 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9100f9f10 con 0x7fd91010e240 2024-04-17T22:29:08.486 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.497 7fd915ad6700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9100f9030 con 0x7fd9100f1d60 2024-04-17T22:29:08.486 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.497 7fd915ad6700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd9100fa1b0 con 0x7fd91010ecc0 2024-04-17T22:29:08.486 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.497 7fd907fff700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd91010ecc0 0x7fd9100f1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:08.486 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.497 7fd90e85d700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fd91010e240 0x7fd91010e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:08.486 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.497 7fd90f05e700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd9100f1d60 0x7fd9101aa4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:08.486 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.497 7fd907fff700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd91010ecc0 0x7fd9100f1730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:40966/0 (socket says 172.21.15.102:40966) 2024-04-17T22:29:08.486 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.497 7fd907fff700 1 -- 172.21.15.102:0/2988676373 learned_addr learned my addr 172.21.15.102:0/2988676373 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:29:08.486 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.497 7fd90e85d700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fd91010e240 0x7fd91010e690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:49226/0 (socket says 172.21.15.102:49226) 2024-04-17T22:29:08.486 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.497 7fd90f05e700 1 --2- 172.21.15.102:0/2988676373 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd9100f1d60 0x7fd9101aa4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:44162/0 (socket says 172.21.15.102:44162) 2024-04-17T22:29:08.486 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.497 7fd90e85d700 1 -- 172.21.15.102:0/2988676373 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd91010ecc0 msgr2=0x7fd9100f1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:29:08.486 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.497 7fd90e85d700 1 --2- 172.21.15.102:0/2988676373 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd91010ecc0 0x7fd9100f1730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:08.486 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.497 7fd90e85d700 1 -- 172.21.15.102:0/2988676373 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd9100f1d60 msgr2=0x7fd9101aa4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:29:08.486 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.497 7fd90e85d700 1 --2- 172.21.15.102:0/2988676373 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd9100f1d60 0x7fd9101aa4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:08.487 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.497 7fd90e85d700 1 -- 172.21.15.102:0/2988676373 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9100f21b0 con 0x7fd91010e240 2024-04-17T22:29:08.487 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.497 7fd90f05e700 1 --2- 172.21.15.102:0/2988676373 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd9100f1d60 0x7fd9101aa4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:29:08.487 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.497 7fd90e85d700 1 --2- 172.21.15.102:0/2988676373 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd91010e240 0x7fd91010e690 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 rx=0x7fd910103580 tx=0x7fd90000ba50).ready entity=mon.0 client_cookie=30155ae353af64e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:29:08.487 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.497 7fd90e05c700 1 -- 172.21.15.102:0/2988676373 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd9100f21b0 con 0x7fd91010e240 2024-04-17T22:29:08.487 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.497 7fd90e05c700 1 -- 172.21.15.102:0/2988676373 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd90000d570 con 0x7fd91010e240 2024-04-17T22:29:08.487 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.497 7fd90e05c700 1 -- 172.21.15.102:0/2988676373 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd90000d940 con 0x7fd91010e240 2024-04-17T22:29:08.487 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.497 7fd915ad6700 1 -- 172.21.15.102:0/2988676373 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd91010e240 msgr2=0x7fd91010e690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:08.487 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.497 7fd915ad6700 1 --2- 172.21.15.102:0/2988676373 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd91010e240 0x7fd91010e690 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 rx=0x7fd910103580 tx=0x7fd90000ba50).stop 2024-04-17T22:29:08.487 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.498 7fd915ad6700 1 -- 172.21.15.102:0/2988676373 shutdown_connections 2024-04-17T22:29:08.487 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.498 7fd915ad6700 1 --2- 172.21.15.102:0/2988676373 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd9100f1d60 0x7fd9101aa4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:08.487 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.498 7fd915ad6700 1 --2- 172.21.15.102:0/2988676373 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd91010e240 0x7fd91010e690 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:08.487 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.498 7fd915ad6700 1 --2- 172.21.15.102:0/2988676373 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd91010ecc0 0x7fd9100f1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:08.487 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.498 7fd915ad6700 1 -- 172.21.15.102:0/2988676373 >> 172.21.15.102:0/2988676373 conn(0x7fd9101138a0 msgr2=0x7fd9100f4f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:29:08.487 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.498 7fd915ad6700 1 -- 172.21.15.102:0/2988676373 shutdown_connections 2024-04-17T22:29:08.487 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.498 7fd915ad6700 1 -- 172.21.15.102:0/2988676373 wait complete. 2024-04-17T22:29:08.507 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.520 7fd915ad6700 1 Processor -- start 2024-04-17T22:29:08.507 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.521 7fd915ad6700 1 -- start start 2024-04-17T22:29:08.509 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.521 7fd915ad6700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd9100d9ff0 0x7fd9101ad6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:08.509 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.521 7fd915ad6700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd9101add10 0x7fd9101b3e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:08.509 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.521 7fd915ad6700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd9101ae160 0x7fd9101b4440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:08.510 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.521 7fd915ad6700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9100d9990 con 0x7fd9101ae160 2024-04-17T22:29:08.510 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.521 7fd915ad6700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9100da440 con 0x7fd9100d9ff0 2024-04-17T22:29:08.510 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.521 7fd915ad6700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd9101b4a70 con 0x7fd9101add10 2024-04-17T22:29:08.510 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.521 7fd90e85d700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd9100d9ff0 0x7fd9101ad6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:08.510 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.521 7fd907fff700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd9101add10 0x7fd9101b3e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:08.510 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.521 7fd90f05e700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd9101ae160 0x7fd9101b4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:08.510 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.521 7fd90e85d700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd9100d9ff0 0x7fd9101ad6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:44164/0 (socket says 172.21.15.102:44164) 2024-04-17T22:29:08.510 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.521 7fd907fff700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd9101add10 0x7fd9101b3e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:40976/0 (socket says 172.21.15.102:40976) 2024-04-17T22:29:08.510 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.521 7fd90e85d700 1 -- 172.21.15.102:0/3677578059 learned_addr learned my addr 172.21.15.102:0/3677578059 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:29:08.510 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.521 7fd90e85d700 1 -- 172.21.15.102:0/3677578059 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd9101add10 msgr2=0x7fd9101b3e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:08.510 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.521 7fd90e85d700 1 --2- 172.21.15.102:0/3677578059 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd9101add10 0x7fd9101b3e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:08.510 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.521 7fd90e85d700 1 -- 172.21.15.102:0/3677578059 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd9101ae160 msgr2=0x7fd9101b4440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:08.510 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.521 7fd90e85d700 1 --2- 172.21.15.102:0/3677578059 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd9101ae160 0x7fd9101b4440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:08.510 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.521 7fd90e85d700 1 -- 172.21.15.102:0/3677578059 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9101b4ca0 con 0x7fd9100d9ff0 2024-04-17T22:29:08.511 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.521 7fd90f05e700 1 --2- 172.21.15.102:0/3677578059 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd9101ae160 0x7fd9101b4440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:29:08.511 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.521 7fd90e85d700 1 --2- 172.21.15.102:0/3677578059 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd9100d9ff0 0x7fd9101ad6e0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0x7fd9100c56d0 tx=0x7fd9000068c0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:29:08.511 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.522 7fd90c859700 1 -- 172.21.15.102:0/3677578059 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd9101b4ca0 con 0x7fd9100d9ff0 2024-04-17T22:29:08.511 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.522 7fd90c859700 1 -- 172.21.15.102:0/3677578059 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd90000de20 con 0x7fd9100d9ff0 2024-04-17T22:29:08.511 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.522 7fd915ad6700 1 -- 172.21.15.102:0/3677578059 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9101bb500 con 0x7fd9100d9ff0 2024-04-17T22:29:08.511 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.522 7fd90c859700 1 -- 172.21.15.102:0/3677578059 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd900011c10 con 0x7fd9100d9ff0 2024-04-17T22:29:08.511 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.522 7fd915ad6700 1 -- 172.21.15.102:0/3677578059 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9101bb8e0 con 0x7fd9100d9ff0 2024-04-17T22:29:08.511 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.522 7fd90c859700 1 -- 172.21.15.102:0/3677578059 <== mon.1 v2:172.21.15.102:3301/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7fd900027460 con 0x7fd9100d9ff0 2024-04-17T22:29:08.511 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.522 7fd90c859700 1 -- 172.21.15.102:0/3677578059 <== mon.1 v2:172.21.15.102:3301/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 3194+0+0 (secure 0 0 0) 0x7fd900020120 con 0x7fd9100d9ff0 2024-04-17T22:29:08.511 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.523 7fd915ad6700 1 -- 172.21.15.102:0/3677578059 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9100f0800 con 0x7fd9100d9ff0 2024-04-17T22:29:08.512 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.526 7fd90c859700 1 -- 172.21.15.102:0/3677578059 <== mon.1 v2:172.21.15.102:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+110455 (secure 0 0 0) 0x7fd900011810 con 0x7fd9100d9ff0 2024-04-17T22:29:08.648 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.661 7fd915ad6700 1 -- 172.21.15.102:0/3677578059 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fd9100efed0 con 0x7fd9100d9ff0 2024-04-17T22:29:08.778 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.792 7fd90c859700 1 -- 172.21.15.102:0/3677578059 <== mon.1 v2:172.21.15.102:3301/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v4) v1 ==== 118+0+0 (secure 0 0 0) 0x7fd90000d360 con 0x7fd9100d9ff0 2024-04-17T22:29:08.780 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.793 7fd915ad6700 1 -- 172.21.15.102:0/3677578059 >> v1:172.21.15.102:6816/42360 conn(0x7fd8f40145a0 legacy=0x7fd8f4016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:08.781 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.793 7fd915ad6700 1 -- 172.21.15.102:0/3677578059 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd9100d9ff0 msgr2=0x7fd9101ad6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:08.781 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.793 7fd915ad6700 1 --2- 172.21.15.102:0/3677578059 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd9100d9ff0 0x7fd9101ad6e0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0x7fd9100c56d0 tx=0x7fd9000068c0).stop 2024-04-17T22:29:08.781 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.793 7fd915ad6700 1 -- 172.21.15.102:0/3677578059 shutdown_connections 2024-04-17T22:29:08.781 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.794 7fd915ad6700 1 --2- 172.21.15.102:0/3677578059 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd9101ae160 0x7fd9101b4440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:08.781 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.794 7fd915ad6700 1 --2- 172.21.15.102:0/3677578059 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd9100d9ff0 0x7fd9101ad6e0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:08.781 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.794 7fd915ad6700 1 --2- 172.21.15.102:0/3677578059 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd9101add10 0x7fd9101b3e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:08.781 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.794 7fd915ad6700 1 -- 172.21.15.102:0/3677578059 >> 172.21.15.102:0/3677578059 conn(0x7fd9100c6a60 msgr2=0x7fd9101a82e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:29:08.781 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.794 7fd915ad6700 1 -- 172.21.15.102:0/3677578059 shutdown_connections 2024-04-17T22:29:08.781 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.794 7fd915ad6700 1 -- 172.21.15.102:0/3677578059 wait complete. 2024-04-17T22:29:08.794 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-17T22:29:08.794 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-17T22:29:08.921 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.934 7f8350e02700 1 Processor -- start 2024-04-17T22:29:08.921 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.934 7f8350e02700 1 -- start start 2024-04-17T22:29:08.923 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.935 7f8350e02700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f834c0f5840 0x7f834c10b010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:08.923 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.935 7f8350e02700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f834c10ba90 0x7f834c1a02f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:08.923 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.935 7f8350e02700 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f834c103b40 con 0x7f834c10b640 2024-04-17T22:29:08.923 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.935 7f8350e02700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f834c10bee0 con 0x7f834c10ba90 2024-04-17T22:29:08.923 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.935 7f8350e02700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f834c0fa370 con 0x7f834c0f5840 2024-04-17T22:29:08.923 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.935 7f8349b64700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f834c0f5840 0x7f834c10b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:08.923 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.935 7f834a365700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f834c10ba90 0x7f834c1a02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:08.923 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.935 7f8341363700 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f834c10b640 0x7f834c19faf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:46456/0 (socket says 172.21.15.102:46456) 2024-04-17T22:29:08.923 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.935 7f8341363700 1 -- 172.21.15.102:0/2839724334 learned_addr learned my addr 172.21.15.102:0/2839724334 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-17T22:29:08.924 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.935 7f8349b64700 1 -- 172.21.15.102:0/2839724334 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f834c10ba90 msgr2=0x7f834c1a02f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:29:08.924 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.935 7f8349b64700 1 --2- 172.21.15.102:0/2839724334 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f834c10ba90 0x7f834c1a02f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:08.924 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.935 7f8349b64700 1 -- 172.21.15.102:0/2839724334 >> v1:172.21.15.102:6789/0 conn(0x7f834c10b640 legacy=0x7f834c19faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:08.924 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.935 7f8349363700 1 -- 172.21.15.102:0/2839724334 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4287144351 0 0) 0x7f8334001110 con 0x7f834c10b640 2024-04-17T22:29:08.924 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.935 7f8349b64700 1 -- 172.21.15.102:0/2839724334 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f834c1a0920 con 0x7f834c0f5840 2024-04-17T22:29:08.924 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.935 7f834a365700 1 --2- 172.21.15.102:0/2839724334 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f834c10ba90 0x7f834c1a02f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:29:08.924 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.935 7f8349b64700 1 --2- 172.21.15.102:0/2839724334 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f834c0f5840 0x7f834c10b010 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7f834c0f46a0 tx=0x7f833c00b9d0).ready entity=mon.2 client_cookie=aacbaf0799742b0d server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:29:08.924 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.935 7f8349363700 1 -- 172.21.15.102:0/2839724334 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f834c1a0920 con 0x7f834c0f5840 2024-04-17T22:29:08.924 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.935 7f8349363700 1 -- 172.21.15.102:0/2839724334 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f833c00d570 con 0x7f834c0f5840 2024-04-17T22:29:08.924 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.935 7f8349363700 1 -- 172.21.15.102:0/2839724334 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f833c00d940 con 0x7f834c0f5840 2024-04-17T22:29:08.924 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.935 7f8350e02700 1 -- 172.21.15.102:0/2839724334 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f834c0f5840 msgr2=0x7f834c10b010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:08.924 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.935 7f8350e02700 1 --2- 172.21.15.102:0/2839724334 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f834c0f5840 0x7f834c10b010 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7f834c0f46a0 tx=0x7f833c00b9d0).stop 2024-04-17T22:29:08.924 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.936 7f8350e02700 1 -- 172.21.15.102:0/2839724334 shutdown_connections 2024-04-17T22:29:08.924 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.936 7f8350e02700 1 --2- 172.21.15.102:0/2839724334 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f834c10ba90 0x7f834c1a02f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:08.924 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.936 7f8350e02700 1 --2- 172.21.15.102:0/2839724334 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f834c0f5840 0x7f834c10b010 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:08.925 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.936 7f8350e02700 1 -- 172.21.15.102:0/2839724334 >> 172.21.15.102:0/2839724334 conn(0x7f834c1138a0 msgr2=0x7f834c0f4f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:29:08.925 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.936 7f8350e02700 1 -- 172.21.15.102:0/2839724334 shutdown_connections 2024-04-17T22:29:08.925 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.936 7f8350e02700 1 -- 172.21.15.102:0/2839724334 wait complete. 2024-04-17T22:29:08.947 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.959 7f8350e02700 1 Processor -- start 2024-04-17T22:29:08.947 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.959 7f8350e02700 1 -- start start 2024-04-17T22:29:08.947 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.959 7f8350e02700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f834c1a0ff0 0x7f834c0d4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:08.947 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.959 7f8350e02700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f834c0d5560 0x7f834c1afa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:08.947 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.959 7f8350e02700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f834c0d59b0 0x7f834c1b0040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:08.947 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.959 7f8350e02700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f834c0c3390 con 0x7f834c0d59b0 2024-04-17T22:29:08.947 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.959 7f8350e02700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f834c1a1440 con 0x7f834c1a0ff0 2024-04-17T22:29:08.948 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.959 7f8350e02700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f834c1b0670 con 0x7f834c0d5560 2024-04-17T22:29:08.948 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.959 7f8349b64700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f834c1a0ff0 0x7f834c0d4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:08.948 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.959 7f8341363700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f834c0d5560 0x7f834c1afa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:08.948 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.959 7f834a365700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f834c0d59b0 0x7f834c1b0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:08.948 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.959 7f8349b64700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f834c1a0ff0 0x7f834c0d4f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:44190/0 (socket says 172.21.15.102:44190) 2024-04-17T22:29:08.948 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.959 7f8349b64700 1 -- 172.21.15.102:0/1677384303 learned_addr learned my addr 172.21.15.102:0/1677384303 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:29:08.948 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.959 7f8341363700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f834c0d5560 0x7f834c1afa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:40982/0 (socket says 172.21.15.102:40982) 2024-04-17T22:29:08.948 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.959 7f8341363700 1 -- 172.21.15.102:0/1677384303 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f834c1a0ff0 msgr2=0x7f834c0d4f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:08.948 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.959 7f8341363700 1 --2- 172.21.15.102:0/1677384303 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f834c1a0ff0 0x7f834c0d4f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:08.948 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.959 7f8341363700 1 -- 172.21.15.102:0/1677384303 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f834c0d59b0 msgr2=0x7f834c1b0040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:08.948 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.959 7f8341363700 1 --2- 172.21.15.102:0/1677384303 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f834c0d59b0 0x7f834c1b0040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:08.948 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.959 7f8341363700 1 -- 172.21.15.102:0/1677384303 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f834c1b08a0 con 0x7f834c0d5560 2024-04-17T22:29:08.948 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.959 7f8349b64700 1 --2- 172.21.15.102:0/1677384303 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f834c1a0ff0 0x7f834c0d4f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:29:08.948 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.960 7f834a365700 1 --2- 172.21.15.102:0/1677384303 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f834c0d59b0 0x7f834c1b0040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:29:08.948 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.960 7f8341363700 1 --2- 172.21.15.102:0/1677384303 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f834c0d5560 0x7f834c1afa10 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7f8334000bc0 tx=0x7f833400c720).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:29:08.949 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.960 7f83437fe700 1 -- 172.21.15.102:0/1677384303 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f834c1b08a0 con 0x7f834c0d5560 2024-04-17T22:29:08.949 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.960 7f83437fe700 1 -- 172.21.15.102:0/1677384303 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83340070d0 con 0x7f834c0d5560 2024-04-17T22:29:08.949 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.960 7f8350e02700 1 -- 172.21.15.102:0/1677384303 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f834c1b7180 con 0x7f834c0d5560 2024-04-17T22:29:08.949 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.960 7f83437fe700 1 -- 172.21.15.102:0/1677384303 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8334005620 con 0x7f834c0d5560 2024-04-17T22:29:08.949 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.960 7f8350e02700 1 -- 172.21.15.102:0/1677384303 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f834c1b7560 con 0x7f834c0d5560 2024-04-17T22:29:08.949 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.960 7f83437fe700 1 -- 172.21.15.102:0/1677384303 <== mon.2 v2:172.21.15.102:3302/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7f833401b070 con 0x7f834c0d5560 2024-04-17T22:29:08.949 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.960 7f83437fe700 1 -- 172.21.15.102:0/1677384303 <== mon.2 v2:172.21.15.102:3302/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 3194+0+0 (secure 0 0 0) 0x7f8334004a00 con 0x7f834c0d5560 2024-04-17T22:29:08.949 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.961 7f8350e02700 1 -- 172.21.15.102:0/1677384303 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f834c0f0800 con 0x7f834c0d5560 2024-04-17T22:29:08.951 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:08.964 7f83437fe700 1 -- 172.21.15.102:0/1677384303 <== mon.2 v2:172.21.15.102:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+110455 (secure 0 0 0) 0x7f833400cd90 con 0x7f834c0d5560 2024-04-17T22:29:09.085 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:09.098 7f8350e02700 1 -- 172.21.15.102:0/1677384303 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f834c0efed0 con 0x7f834c0d5560 2024-04-17T22:29:09.786 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:09.799 7f83437fe700 1 -- 172.21.15.102:0/1677384303 <== mon.2 v2:172.21.15.102:3302/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v5) v1 ==== 122+0+0 (secure 0 0 0) 0x7f8334000c30 con 0x7f834c0d5560 2024-04-17T22:29:09.788 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:09.801 7f8350e02700 1 -- 172.21.15.102:0/1677384303 >> v1:172.21.15.102:6816/42360 conn(0x7f83300145a0 legacy=0x7f8330016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:09.788 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:09.801 7f8350e02700 1 -- 172.21.15.102:0/1677384303 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f834c0d5560 msgr2=0x7f834c1afa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:09.788 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:09.801 7f8350e02700 1 --2- 172.21.15.102:0/1677384303 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f834c0d5560 0x7f834c1afa10 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7f8334000bc0 tx=0x7f833400c720).stop 2024-04-17T22:29:09.788 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:09.801 7f8350e02700 1 -- 172.21.15.102:0/1677384303 shutdown_connections 2024-04-17T22:29:09.788 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:09.801 7f8350e02700 1 --2- 172.21.15.102:0/1677384303 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f834c0d59b0 0x7f834c1b0040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:09.788 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:09.801 7f8350e02700 1 --2- 172.21.15.102:0/1677384303 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f834c1a0ff0 0x7f834c0d4f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:09.788 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:09.801 7f8350e02700 1 --2- 172.21.15.102:0/1677384303 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f834c0d5560 0x7f834c1afa10 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:09.788 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:09.801 7f8350e02700 1 -- 172.21.15.102:0/1677384303 >> 172.21.15.102:0/1677384303 conn(0x7f834c0dff40 msgr2=0x7f834c0c1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:29:09.788 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:09.801 7f8350e02700 1 -- 172.21.15.102:0/1677384303 shutdown_connections 2024-04-17T22:29:09.788 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:09.801 7f8350e02700 1 -- 172.21.15.102:0/1677384303 wait complete. 2024-04-17T22:29:09.801 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-17T22:29:09.802 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T22:29:09.935 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:09.946 7ffbda1a6700 1 Processor -- start 2024-04-17T22:29:09.935 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:09.946 7ffbda1a6700 1 -- start start 2024-04-17T22:29:09.935 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:09.947 7ffbda1a6700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7ffbd410b830 0x7ffbd41a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:09.935 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:09.947 7ffbda1a6700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7ffbd410bc80 0x7ffbd41a7920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:09.935 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:09.947 7ffbda1a6700 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffbd4103b60 con 0x7ffbd40f5860 2024-04-17T22:29:09.935 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:09.947 7ffbda1a6700 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffbd40fa390 con 0x7ffbd410bc80 2024-04-17T22:29:09.935 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:09.947 7ffbda1a6700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ffbd41a0b20 con 0x7ffbd410b830 2024-04-17T22:29:09.935 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:09.947 7ffbcbfff700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7ffbd410b830 0x7ffbd41a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:09.936 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:09.947 7ffbd2dc6700 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7ffbd40f5860 0x7ffbd410b030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:46472/0 (socket says 172.21.15.102:46472) 2024-04-17T22:29:09.936 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:09.947 7ffbd2dc6700 1 -- 172.21.15.102:0/1844668945 learned_addr learned my addr 172.21.15.102:0/1844668945 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-17T22:29:09.936 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:09.947 7ffbd35c7700 1 --2- 172.21.15.102:0/1844668945 >> v2:172.21.15.102:3300/0 conn(0x7ffbd410bc80 0x7ffbd41a7920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:09.936 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:09.947 7ffbcbfff700 1 -- 172.21.15.102:0/1844668945 >> v2:172.21.15.102:3300/0 conn(0x7ffbd410bc80 msgr2=0x7ffbd41a7920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:29:09.936 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:09.947 7ffbcbfff700 1 --2- 172.21.15.102:0/1844668945 >> v2:172.21.15.102:3300/0 conn(0x7ffbd410bc80 0x7ffbd41a7920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:09.936 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:09.947 7ffbcbfff700 1 -- 172.21.15.102:0/1844668945 >> v1:172.21.15.102:6789/0 conn(0x7ffbd40f5860 legacy=0x7ffbd410b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:09.936 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:09.947 7ffbd25c5700 1 -- 172.21.15.102:0/1844668945 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3274948020 0 0) 0x7ffbc4001110 con 0x7ffbd40f5860 2024-04-17T22:29:09.936 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:09.947 7ffbcbfff700 1 -- 172.21.15.102:0/1844668945 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbd41a8f60 con 0x7ffbd410b830 2024-04-17T22:29:09.936 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:09.947 7ffbcbfff700 1 --2- 172.21.15.102:0/1844668945 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7ffbd410b830 0x7ffbd41a52e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7ffbbc002f70 tx=0x7ffbbc00dbd0).ready entity=mon.2 client_cookie=57c3babf87255a7a server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:29:09.936 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:09.947 7ffbd25c5700 1 -- 172.21.15.102:0/1844668945 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ffbd41a8f60 con 0x7ffbd410b830 2024-04-17T22:29:09.936 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:09.947 7ffbd25c5700 1 -- 172.21.15.102:0/1844668945 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbbc00bbd0 con 0x7ffbd410b830 2024-04-17T22:29:09.936 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:09.947 7ffbd25c5700 1 -- 172.21.15.102:0/1844668945 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ffbbc015070 con 0x7ffbd410b830 2024-04-17T22:29:09.937 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:09.948 7ffbda1a6700 1 -- 172.21.15.102:0/1844668945 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7ffbd410b830 msgr2=0x7ffbd41a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:09.937 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:09.948 7ffbda1a6700 1 --2- 172.21.15.102:0/1844668945 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7ffbd410b830 0x7ffbd41a52e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7ffbbc002f70 tx=0x7ffbbc00dbd0).stop 2024-04-17T22:29:09.937 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:09.948 7ffbda1a6700 1 -- 172.21.15.102:0/1844668945 shutdown_connections 2024-04-17T22:29:09.937 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:09.948 7ffbda1a6700 1 --2- 172.21.15.102:0/1844668945 >> v2:172.21.15.102:3300/0 conn(0x7ffbd410bc80 0x7ffbd41a7920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:09.937 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:09.948 7ffbda1a6700 1 --2- 172.21.15.102:0/1844668945 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7ffbd410b830 0x7ffbd41a52e0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:09.937 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:09.948 7ffbda1a6700 1 -- 172.21.15.102:0/1844668945 >> 172.21.15.102:0/1844668945 conn(0x7ffbd41138c0 msgr2=0x7ffbd40f4f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:29:09.937 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:09.948 7ffbda1a6700 1 -- 172.21.15.102:0/1844668945 shutdown_connections 2024-04-17T22:29:09.937 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:09.948 7ffbda1a6700 1 -- 172.21.15.102:0/1844668945 wait complete. 2024-04-17T22:29:09.960 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:09.971 7ffbda1a6700 1 Processor -- start 2024-04-17T22:29:09.960 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:09.971 7ffbda1a6700 1 -- start start 2024-04-17T22:29:09.960 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:09.971 7ffbda1a6700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ffbd40dd2b0 0x7ffbd40d5660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:09.960 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:09.971 7ffbda1a6700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7ffbd40d5c90 0x7ffbd41b02e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:09.960 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:09.971 7ffbda1a6700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffbd40d60e0 0x7ffbd41b0910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:09.960 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:09.971 7ffbda1a6700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbd40e2ba0 con 0x7ffbd40d60e0 2024-04-17T22:29:09.960 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:09.971 7ffbda1a6700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbd40dc930 con 0x7ffbd40dd2b0 2024-04-17T22:29:09.960 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:09.971 7ffbda1a6700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ffbd40dc510 con 0x7ffbd40d5c90 2024-04-17T22:29:09.960 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:09.971 7ffbd2dc6700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ffbd40dd2b0 0x7ffbd40d5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:09.960 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:09.971 7ffbcbfff700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7ffbd40d5c90 0x7ffbd41b02e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:09.961 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:09.971 7ffbd2dc6700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ffbd40dd2b0 0x7ffbd40d5660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:44198/0 (socket says 172.21.15.102:44198) 2024-04-17T22:29:09.961 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:09.971 7ffbd2dc6700 1 -- 172.21.15.102:0/4286946661 learned_addr learned my addr 172.21.15.102:0/4286946661 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:29:09.961 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:09.971 7ffbcbfff700 1 --2- 172.21.15.102:0/4286946661 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7ffbd40d5c90 0x7ffbd41b02e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:40998/0 (socket says 172.21.15.102:40998) 2024-04-17T22:29:09.961 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:09.971 7ffbd35c7700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffbd40d60e0 0x7ffbd41b0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:09.961 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:09.971 7ffbd35c7700 1 -- 172.21.15.102:0/4286946661 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7ffbd40d5c90 msgr2=0x7ffbd41b02e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:09.961 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:09.971 7ffbd35c7700 1 --2- 172.21.15.102:0/4286946661 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7ffbd40d5c90 0x7ffbd41b02e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:09.961 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:09.971 7ffbd35c7700 1 -- 172.21.15.102:0/4286946661 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ffbd40dd2b0 msgr2=0x7ffbd40d5660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:09.961 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:09.971 7ffbd35c7700 1 --2- 172.21.15.102:0/4286946661 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ffbd40dd2b0 0x7ffbd40d5660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:09.961 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:09.971 7ffbd35c7700 1 -- 172.21.15.102:0/4286946661 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbd41b0f40 con 0x7ffbd40d60e0 2024-04-17T22:29:09.961 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:09.971 7ffbd2dc6700 1 --2- 172.21.15.102:0/4286946661 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ffbd40dd2b0 0x7ffbd40d5660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:29:09.961 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:09.972 7ffbd35c7700 1 --2- 172.21.15.102:0/4286946661 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffbd40d60e0 0x7ffbd41b0910 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7ffbd41acd70 tx=0x7ffbcc00ee30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:29:09.961 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:09.972 7ffbd0dc2700 1 -- 172.21.15.102:0/4286946661 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ffbd41b0f40 con 0x7ffbd40d60e0 2024-04-17T22:29:09.961 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:09.972 7ffbd0dc2700 1 -- 172.21.15.102:0/4286946661 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbcc00c850 con 0x7ffbd40d60e0 2024-04-17T22:29:09.962 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:09.972 7ffbda1a6700 1 -- 172.21.15.102:0/4286946661 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffbd41b1260 con 0x7ffbd40d60e0 2024-04-17T22:29:09.962 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:09.972 7ffbd0dc2700 1 -- 172.21.15.102:0/4286946661 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ffbcc00f040 con 0x7ffbd40d60e0 2024-04-17T22:29:09.962 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:09.972 7ffbda1a6700 1 -- 172.21.15.102:0/4286946661 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbd41b7bb0 con 0x7ffbd40d60e0 2024-04-17T22:29:09.962 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:09.972 7ffbd0dc2700 1 -- 172.21.15.102:0/4286946661 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7ffbcc022060 con 0x7ffbd40d60e0 2024-04-17T22:29:09.962 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:09.972 7ffbd0dc2700 1 -- 172.21.15.102:0/4286946661 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 3194+0+0 (secure 0 0 0) 0x7ffbcc010040 con 0x7ffbd40d60e0 2024-04-17T22:29:09.962 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:09.973 7ffbda1a6700 1 -- 172.21.15.102:0/4286946661 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffbd40f0820 con 0x7ffbd40d60e0 2024-04-17T22:29:09.962 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:09.976 7ffbd0dc2700 1 -- 172.21.15.102:0/4286946661 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+110455 (secure 0 0 0) 0x7ffbcc00c640 con 0x7ffbd40d60e0 2024-04-17T22:29:10.096 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.110 7ffbda1a6700 1 -- 172.21.15.102:0/4286946661 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ffbd41aaa00 con 0x7ffbd40d60e0 2024-04-17T22:29:10.096 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.110 7ffbd0dc2700 1 -- 172.21.15.102:0/4286946661 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) v1 ==== 93+0+1704 (secure 0 0 0) 0x7ffbd4114440 con 0x7ffbd40d60e0 2024-04-17T22:29:10.097 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-17T22:29:10.097 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":5,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[],"filesystems":[{"mdsmap":{"epoch":5,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17 22:29:07.787730","modified":"2024-04-17 22:29:09.795249","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"-1 (unspecified)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4177},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4177":{"gid":4177,"name":"a","rank":0,"incarnation":3,"state":"up:active","state_seq":4,"addr":"172.21.15.102:6817/1944542598","addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6817","nonce":1944542598}]},"export_targets":[],"features":4611087854035861503,"flags":0}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-04-17T22:29:10.098 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.111 7ffbda1a6700 1 -- 172.21.15.102:0/4286946661 >> v1:172.21.15.102:6816/42360 conn(0x7ffbb80145a0 legacy=0x7ffbb8016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:10.098 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.111 7ffbda1a6700 1 -- 172.21.15.102:0/4286946661 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffbd40d60e0 msgr2=0x7ffbd41b0910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:10.098 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.111 7ffbda1a6700 1 --2- 172.21.15.102:0/4286946661 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffbd40d60e0 0x7ffbd41b0910 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7ffbd41acd70 tx=0x7ffbcc00ee30).stop 2024-04-17T22:29:10.098 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.111 7ffbda1a6700 1 -- 172.21.15.102:0/4286946661 shutdown_connections 2024-04-17T22:29:10.098 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.111 7ffbda1a6700 1 --2- 172.21.15.102:0/4286946661 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffbd40d60e0 0x7ffbd41b0910 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:10.099 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.111 7ffbda1a6700 1 --2- 172.21.15.102:0/4286946661 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ffbd40dd2b0 0x7ffbd40d5660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:10.099 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.111 7ffbda1a6700 1 --2- 172.21.15.102:0/4286946661 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7ffbd40d5c90 0x7ffbd41b02e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:10.099 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.111 7ffbda1a6700 1 -- 172.21.15.102:0/4286946661 >> 172.21.15.102:0/4286946661 conn(0x7ffbd40e0a70 msgr2=0x7ffbd40ce8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:29:10.099 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.111 7ffbda1a6700 1 -- 172.21.15.102:0/4286946661 shutdown_connections 2024-04-17T22:29:10.099 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.111 7ffbda1a6700 1 -- 172.21.15.102:0/4286946661 wait complete. 2024-04-17T22:29:10.099 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 5 2024-04-17T22:29:10.109 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T22:29:10.238 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.248 7fe58a5e3700 1 Processor -- start 2024-04-17T22:29:10.239 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.248 7fe58a5e3700 1 -- start start 2024-04-17T22:29:10.239 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.248 7fe58a5e3700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe5840f5860 0x7fe58410b030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:10.239 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.248 7fe58a5e3700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fe58410bab0 0x7fe5841a0310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:10.239 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.248 7fe58a5e3700 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe584103b60 con 0x7fe58410b660 2024-04-17T22:29:10.239 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.248 7fe58a5e3700 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe58410bf00 con 0x7fe58410bab0 2024-04-17T22:29:10.239 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.248 7fe58a5e3700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5840fa390 con 0x7fe5840f5860 2024-04-17T22:29:10.239 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.248 7fe5833b0700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe5840f5860 0x7fe58410b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:10.239 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.248 7fe583bb1700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fe58410bab0 0x7fe5841a0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:10.239 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.248 7fe583bb1700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fe58410bab0 0x7fe5841a0310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:49274/0 (socket says 172.21.15.102:49274) 2024-04-17T22:29:10.240 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.248 7fe5833b0700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe5840f5860 0x7fe58410b030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:44214/0 (socket says 172.21.15.102:44214) 2024-04-17T22:29:10.240 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.248 7fe583bb1700 1 -- 172.21.15.102:0/810684143 learned_addr learned my addr 172.21.15.102:0/810684143 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:29:10.240 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.248 7fe57bfff700 1 --1- 172.21.15.102:0/810684143 >> v1:172.21.15.102:6789/0 conn(0x7fe58410b660 0x7fe58419fb10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:46474/0 (socket says 172.21.15.102:46474) 2024-04-17T22:29:10.240 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.249 7fe5833b0700 1 -- 172.21.15.102:0/810684143 >> v2:172.21.15.102:3300/0 conn(0x7fe58410bab0 msgr2=0x7fe5841a0310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:29:10.240 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.249 7fe5833b0700 1 --2- 172.21.15.102:0/810684143 >> v2:172.21.15.102:3300/0 conn(0x7fe58410bab0 0x7fe5841a0310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:10.240 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.249 7fe5833b0700 1 -- 172.21.15.102:0/810684143 >> v1:172.21.15.102:6789/0 conn(0x7fe58410b660 legacy=0x7fe58419fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:10.241 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.249 7fe5833b0700 1 -- 172.21.15.102:0/810684143 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5841a0940 con 0x7fe5840f5860 2024-04-17T22:29:10.241 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.249 7fe5833b0700 1 --2- 172.21.15.102:0/810684143 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe5840f5860 0x7fe58410b030 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7fe574002f70 tx=0x7fe57400b970).ready entity=mon.1 client_cookie=59ee39306621bc5d server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:29:10.241 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.249 7fe582baf700 1 -- 172.21.15.102:0/810684143 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe5841a0940 con 0x7fe5840f5860 2024-04-17T22:29:10.241 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.249 7fe582baf700 1 -- 172.21.15.102:0/810684143 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe57400d5c0 con 0x7fe5840f5860 2024-04-17T22:29:10.241 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.249 7fe582baf700 1 -- 172.21.15.102:0/810684143 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe57400d990 con 0x7fe5840f5860 2024-04-17T22:29:10.241 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.249 7fe58a5e3700 1 -- 172.21.15.102:0/810684143 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe5840f5860 msgr2=0x7fe58410b030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:10.241 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.249 7fe58a5e3700 1 --2- 172.21.15.102:0/810684143 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe5840f5860 0x7fe58410b030 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7fe574002f70 tx=0x7fe57400b970).stop 2024-04-17T22:29:10.241 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.250 7fe58a5e3700 1 -- 172.21.15.102:0/810684143 shutdown_connections 2024-04-17T22:29:10.241 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.250 7fe58a5e3700 1 --2- 172.21.15.102:0/810684143 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe5840f5860 0x7fe58410b030 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:10.241 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.250 7fe58a5e3700 1 --2- 172.21.15.102:0/810684143 >> v2:172.21.15.102:3300/0 conn(0x7fe58410bab0 0x7fe5841a0310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:10.241 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.250 7fe58a5e3700 1 -- 172.21.15.102:0/810684143 >> 172.21.15.102:0/810684143 conn(0x7fe5841138c0 msgr2=0x7fe5840f4f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:29:10.241 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.250 7fe58a5e3700 1 -- 172.21.15.102:0/810684143 shutdown_connections 2024-04-17T22:29:10.242 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.250 7fe58a5e3700 1 -- 172.21.15.102:0/810684143 wait complete. 2024-04-17T22:29:10.260 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.272 7fe58a5e3700 1 Processor -- start 2024-04-17T22:29:10.261 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.272 7fe58a5e3700 1 -- start start 2024-04-17T22:29:10.261 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.273 7fe58a5e3700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fe5841ad690 0x7fe5841adae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:10.261 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.273 7fe58a5e3700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe5841ae110 0x7fe5841b3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:10.261 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.273 7fe58a5e3700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe5841b45f0 0x7fe5841b4a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:10.261 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.273 7fe58a5e3700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5840f5340 con 0x7fe5841ae110 2024-04-17T22:29:10.261 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.273 7fe58a5e3700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5841b5070 con 0x7fe5841b45f0 2024-04-17T22:29:10.261 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.273 7fe58a5e3700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe5840d5cc0 con 0x7fe5841ad690 2024-04-17T22:29:10.261 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.273 7fe583bb1700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe5841b45f0 0x7fe5841b4a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:10.261 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.273 7fe5833b0700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fe5841ad690 0x7fe5841adae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:10.261 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.273 7fe57bfff700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe5841ae110 0x7fe5841b3fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:10.261 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.273 7fe583bb1700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe5841b45f0 0x7fe5841b4a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:44224/0 (socket says 172.21.15.102:44224) 2024-04-17T22:29:10.261 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.273 7fe583bb1700 1 -- 172.21.15.102:0/3202853501 learned_addr learned my addr 172.21.15.102:0/3202853501 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:29:10.261 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.273 7fe5833b0700 1 --2- 172.21.15.102:0/3202853501 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fe5841ad690 0x7fe5841adae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:41010/0 (socket says 172.21.15.102:41010) 2024-04-17T22:29:10.262 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.273 7fe583bb1700 1 -- 172.21.15.102:0/3202853501 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fe5841ad690 msgr2=0x7fe5841adae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:10.262 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.273 7fe583bb1700 1 --2- 172.21.15.102:0/3202853501 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fe5841ad690 0x7fe5841adae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:10.262 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.273 7fe583bb1700 1 -- 172.21.15.102:0/3202853501 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe5841ae110 msgr2=0x7fe5841b3fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:10.262 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.273 7fe583bb1700 1 --2- 172.21.15.102:0/3202853501 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe5841ae110 0x7fe5841b3fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:10.262 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.273 7fe583bb1700 1 -- 172.21.15.102:0/3202853501 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5840d99d0 con 0x7fe5841b45f0 2024-04-17T22:29:10.262 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.273 7fe57bfff700 1 --2- 172.21.15.102:0/3202853501 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe5841ae110 0x7fe5841b3fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:29:10.262 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.273 7fe583bb1700 1 --2- 172.21.15.102:0/3202853501 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe5841b45f0 0x7fe5841b4a40 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7fe5840c4a00 tx=0x7fe57c00c430).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:29:10.262 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.273 7fe5813ac700 1 -- 172.21.15.102:0/3202853501 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe5840d99d0 con 0x7fe5841b45f0 2024-04-17T22:29:10.262 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.273 7fe5813ac700 1 -- 172.21.15.102:0/3202853501 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe57c00c960 con 0x7fe5841b45f0 2024-04-17T22:29:10.262 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.273 7fe58a5e3700 1 -- 172.21.15.102:0/3202853501 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5840d3830 con 0x7fe5841b45f0 2024-04-17T22:29:10.262 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.273 7fe5813ac700 1 -- 172.21.15.102:0/3202853501 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe57c00f040 con 0x7fe5841b45f0 2024-04-17T22:29:10.262 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.274 7fe58a5e3700 1 -- 172.21.15.102:0/3202853501 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5841bb7c0 con 0x7fe5841b45f0 2024-04-17T22:29:10.262 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.274 7fe5813ac700 1 -- 172.21.15.102:0/3202853501 <== mon.1 v2:172.21.15.102:3301/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7fe57c022060 con 0x7fe5841b45f0 2024-04-17T22:29:10.262 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.274 7fe5813ac700 1 -- 172.21.15.102:0/3202853501 <== mon.1 v2:172.21.15.102:3301/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 3194+0+0 (secure 0 0 0) 0x7fe57c010040 con 0x7fe5841b45f0 2024-04-17T22:29:10.263 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.274 7fe58a5e3700 1 -- 172.21.15.102:0/3202853501 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5840f0820 con 0x7fe5841b45f0 2024-04-17T22:29:10.264 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.277 7fe5813ac700 1 -- 172.21.15.102:0/3202853501 <== mon.1 v2:172.21.15.102:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+110455 (secure 0 0 0) 0x7fe57c00edd0 con 0x7fe5841b45f0 2024-04-17T22:29:10.369 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.382 7fe58a5e3700 1 -- 172.21.15.102:0/3202853501 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe5840abcc0 con 0x7fe5841b45f0 2024-04-17T22:29:10.369 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.383 7fe5813ac700 1 -- 172.21.15.102:0/3202853501 <== mon.1 v2:172.21.15.102:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+8761 (secure 0 0 0) 0x7fe584114440 con 0x7fe5841b45f0 2024-04-17T22:29:10.369 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-17T22:29:10.370 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":16,"fsid":"cb63e789-091d-49b1-b120-55f9db526871","created":"2024-04-17 22:28:48.209635","modified":"2024-04-17 22:29:08.789044","last_up_change":"2024-04-17 22:28:54.236215","last_in_change":"2024-04-17 22:28:54.236215","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":4,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-04-17 22:28:57.478545","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-17 22:29:06.144708","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-17 22:29:06.748924","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3351ef85-ade4-4b90-a8f4-213f4e00ba01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6812","nonce":42955}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6813","nonce":42955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6815","nonce":42955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6814","nonce":42955}]},"public_addr":"172.21.15.102:6812/42955","cluster_addr":"172.21.15.102:6813/42955","heartbeat_back_addr":"172.21.15.102:6815/42955","heartbeat_front_addr":"172.21.15.102:6814/42955","state":["exists","up"]},{"osd":1,"uuid":"04f67eac-f36b-48f5-982d-4cfc776596d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6808","nonce":42958}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6809","nonce":42958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6811","nonce":42958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6810","nonce":42958}]},"public_addr":"172.21.15.102:6808/42958","cluster_addr":"172.21.15.102:6809/42958","heartbeat_back_addr":"172.21.15.102:6811/42958","heartbeat_front_addr":"172.21.15.102:6810/42958","state":["exists","up"]},{"osd":2,"uuid":"d524204b-70b2-40a8-8f73-f9e7d44318af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6804","nonce":42954}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6805","nonce":42954}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6807","nonce":42954}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6806","nonce":42954}]},"public_addr":"172.21.15.102:6804/42954","cluster_addr":"172.21.15.102:6805/42954","heartbeat_back_addr":"172.21.15.102:6807/42954","heartbeat_front_addr":"172.21.15.102:6806/42954","state":["exists","up"]},{"osd":3,"uuid":"4fc9d20c-1ad0-484f-aafa-2c9a7574245b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6800","nonce":42953}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6801","nonce":42953}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6803","nonce":42953}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6802","nonce":42953}]},"public_addr":"172.21.15.102:6800/42953","cluster_addr":"172.21.15.102:6801/42953","heartbeat_back_addr":"172.21.15.102:6803/42953","heartbeat_front_addr":"172.21.15.102:6802/42953","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2024-04-17T22:29:10.371 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.384 7fe58a5e3700 1 -- 172.21.15.102:0/3202853501 >> v1:172.21.15.102:6816/42360 conn(0x7fe5680145a0 legacy=0x7fe568016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:10.371 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.384 7fe58a5e3700 1 -- 172.21.15.102:0/3202853501 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe5841b45f0 msgr2=0x7fe5841b4a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:10.371 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.384 7fe58a5e3700 1 --2- 172.21.15.102:0/3202853501 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe5841b45f0 0x7fe5841b4a40 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7fe5840c4a00 tx=0x7fe57c00c430).stop 2024-04-17T22:29:10.371 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.384 7fe58a5e3700 1 -- 172.21.15.102:0/3202853501 shutdown_connections 2024-04-17T22:29:10.371 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.384 7fe58a5e3700 1 --2- 172.21.15.102:0/3202853501 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe5841b45f0 0x7fe5841b4a40 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:10.371 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.384 7fe58a5e3700 1 --2- 172.21.15.102:0/3202853501 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fe5841ad690 0x7fe5841adae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:10.371 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.384 7fe58a5e3700 1 --2- 172.21.15.102:0/3202853501 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe5841ae110 0x7fe5841b3fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:10.371 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.384 7fe58a5e3700 1 -- 172.21.15.102:0/3202853501 >> 172.21.15.102:0/3202853501 conn(0x7fe5840c4400 msgr2=0x7fe5840c1f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:29:10.371 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.384 7fe58a5e3700 1 -- 172.21.15.102:0/3202853501 shutdown_connections 2024-04-17T22:29:10.372 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.384 7fe58a5e3700 1 -- 172.21.15.102:0/3202853501 wait complete. 2024-04-17T22:29:10.381 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-17T22:29:10.382 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-17T22:29:10.507 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.519 7fc84d9c4700 1 Processor -- start 2024-04-17T22:29:10.507 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.519 7fc84d9c4700 1 -- start start 2024-04-17T22:29:10.507 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.519 7fc84d9c4700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fc8480f5860 0x7fc84810b030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:10.508 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.519 7fc84d9c4700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc84810b660 0x7fc84819fb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:10.508 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.519 7fc84d9c4700 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc848103b60 con 0x7fc84810bab0 2024-04-17T22:29:10.508 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.519 7fc84d9c4700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc84810bf00 con 0x7fc84810b660 2024-04-17T22:29:10.508 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.519 7fc84d9c4700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc8480fa390 con 0x7fc8480f5860 2024-04-17T22:29:10.508 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.519 7fc845dc4700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc84810b660 0x7fc84819fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:10.508 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.519 7fc846dc6700 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7fc84810bab0 0x7fc8481a0140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:46488/0 (socket says 172.21.15.102:46488) 2024-04-17T22:29:10.508 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.519 7fc8465c5700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fc8480f5860 0x7fc84810b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:10.508 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.519 7fc846dc6700 1 -- 172.21.15.102:0/2321090602 learned_addr learned my addr 172.21.15.102:0/2321090602 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-17T22:29:10.508 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.520 7fc8465c5700 1 -- 172.21.15.102:0/2321090602 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc84810b660 msgr2=0x7fc84819fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:29:10.508 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.520 7fc8455c3700 1 -- 172.21.15.102:0/2321090602 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3199541570 0 0) 0x7fc840001490 con 0x7fc84810bab0 2024-04-17T22:29:10.508 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.520 7fc8465c5700 1 --2- 172.21.15.102:0/2321090602 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc84810b660 0x7fc84819fb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:10.508 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.520 7fc8465c5700 1 -- 172.21.15.102:0/2321090602 >> v1:172.21.15.102:6789/0 conn(0x7fc84810bab0 legacy=0x7fc8481a0140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:10.509 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.520 7fc8465c5700 1 -- 172.21.15.102:0/2321090602 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8481a0940 con 0x7fc8480f5860 2024-04-17T22:29:10.509 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.520 7fc845dc4700 1 --2- 172.21.15.102:0/2321090602 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc84810b660 0x7fc84819fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:29:10.509 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.520 7fc8465c5700 1 --2- 172.21.15.102:0/2321090602 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fc8480f5860 0x7fc84810b030 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7fc83c002f70 tx=0x7fc83c00b9a0).ready entity=mon.2 client_cookie=7f0e48560926f714 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:29:10.509 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.520 7fc8455c3700 1 -- 172.21.15.102:0/2321090602 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc8481a0940 con 0x7fc8480f5860 2024-04-17T22:29:10.509 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.520 7fc8455c3700 1 -- 172.21.15.102:0/2321090602 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc83c00d570 con 0x7fc8480f5860 2024-04-17T22:29:10.509 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.520 7fc8455c3700 1 -- 172.21.15.102:0/2321090602 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc83c00d940 con 0x7fc8480f5860 2024-04-17T22:29:10.509 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.520 7fc84d9c4700 1 -- 172.21.15.102:0/2321090602 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fc8480f5860 msgr2=0x7fc84810b030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:10.509 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.520 7fc84d9c4700 1 --2- 172.21.15.102:0/2321090602 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fc8480f5860 0x7fc84810b030 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7fc83c002f70 tx=0x7fc83c00b9a0).stop 2024-04-17T22:29:10.509 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.521 7fc84d9c4700 1 -- 172.21.15.102:0/2321090602 shutdown_connections 2024-04-17T22:29:10.509 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.521 7fc84d9c4700 1 --2- 172.21.15.102:0/2321090602 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fc8480f5860 0x7fc84810b030 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:10.509 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.521 7fc84d9c4700 1 --2- 172.21.15.102:0/2321090602 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc84810b660 0x7fc84819fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:10.509 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.521 7fc84d9c4700 1 -- 172.21.15.102:0/2321090602 >> 172.21.15.102:0/2321090602 conn(0x7fc8481138c0 msgr2=0x7fc8480f4f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:29:10.509 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.521 7fc84d9c4700 1 -- 172.21.15.102:0/2321090602 shutdown_connections 2024-04-17T22:29:10.509 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.521 7fc84d9c4700 1 -- 172.21.15.102:0/2321090602 wait complete. 2024-04-17T22:29:10.532 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.543 7fc84d9c4700 1 Processor -- start 2024-04-17T22:29:10.532 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.543 7fc84d9c4700 1 -- start start 2024-04-17T22:29:10.532 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.543 7fc84d9c4700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc8480ce6f0 0x7fc8480f17d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:10.532 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.543 7fc84d9c4700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fc8480f1e00 0x7fc8481ab770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:10.532 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.543 7fc84d9c4700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc8480f2250 0x7fc8481abda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:10.532 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.543 7fc84d9c4700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8480cdd70 con 0x7fc8480f2250 2024-04-17T22:29:10.532 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.543 7fc84d9c4700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8481ac3d0 con 0x7fc8480ce6f0 2024-04-17T22:29:10.533 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.543 7fc84d9c4700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc8481b9960 con 0x7fc8480f1e00 2024-04-17T22:29:10.533 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.544 7fc8465c5700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc8480ce6f0 0x7fc8480f17d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:10.533 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.544 7fc845dc4700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fc8480f1e00 0x7fc8481ab770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:10.533 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.544 7fc846dc6700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc8480f2250 0x7fc8481abda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:10.533 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.544 7fc8465c5700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc8480ce6f0 0x7fc8480f17d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:44238/0 (socket says 172.21.15.102:44238) 2024-04-17T22:29:10.533 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.544 7fc845dc4700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fc8480f1e00 0x7fc8481ab770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:41030/0 (socket says 172.21.15.102:41030) 2024-04-17T22:29:10.533 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.544 7fc8465c5700 1 -- 172.21.15.102:0/4188557705 learned_addr learned my addr 172.21.15.102:0/4188557705 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:29:10.533 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.544 7fc845dc4700 1 -- 172.21.15.102:0/4188557705 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc8480ce6f0 msgr2=0x7fc8480f17d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:10.533 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.544 7fc845dc4700 1 --2- 172.21.15.102:0/4188557705 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc8480ce6f0 0x7fc8480f17d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:10.533 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.544 7fc845dc4700 1 -- 172.21.15.102:0/4188557705 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc8480f2250 msgr2=0x7fc8481abda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:10.533 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.544 7fc845dc4700 1 --2- 172.21.15.102:0/4188557705 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc8480f2250 0x7fc8481abda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:10.533 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.544 7fc845dc4700 1 -- 172.21.15.102:0/4188557705 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8481b9b10 con 0x7fc8480f1e00 2024-04-17T22:29:10.533 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.544 7fc846dc6700 1 --2- 172.21.15.102:0/4188557705 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc8480f2250 0x7fc8481abda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:29:10.534 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.544 7fc845dc4700 1 --2- 172.21.15.102:0/4188557705 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fc8480f1e00 0x7fc8481ab770 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0x7fc83800c000 tx=0x7fc83800ebc0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:29:10.534 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.544 7fc8377fe700 1 -- 172.21.15.102:0/4188557705 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc8481b9b10 con 0x7fc8480f1e00 2024-04-17T22:29:10.534 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.544 7fc8377fe700 1 -- 172.21.15.102:0/4188557705 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc838010780 con 0x7fc8480f1e00 2024-04-17T22:29:10.534 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.544 7fc84d9c4700 1 -- 172.21.15.102:0/4188557705 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8481b9d10 con 0x7fc8480f1e00 2024-04-17T22:29:10.534 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.544 7fc8377fe700 1 -- 172.21.15.102:0/4188557705 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc83800f040 con 0x7fc8480f1e00 2024-04-17T22:29:10.534 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.544 7fc84d9c4700 1 -- 172.21.15.102:0/4188557705 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8481ba0f0 con 0x7fc8480f1e00 2024-04-17T22:29:10.534 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.545 7fc8377fe700 1 -- 172.21.15.102:0/4188557705 <== mon.2 v2:172.21.15.102:3302/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7fc838011070 con 0x7fc8480f1e00 2024-04-17T22:29:10.534 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.545 7fc8377fe700 1 -- 172.21.15.102:0/4188557705 <== mon.2 v2:172.21.15.102:3302/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 3194+0+0 (secure 0 0 0) 0x7fc838010040 con 0x7fc8480f1e00 2024-04-17T22:29:10.534 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.545 7fc84d9c4700 1 -- 172.21.15.102:0/4188557705 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8481ac600 con 0x7fc8480f1e00 2024-04-17T22:29:10.535 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.548 7fc8377fe700 1 -- 172.21.15.102:0/4188557705 <== mon.2 v2:172.21.15.102:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+110455 (secure 0 0 0) 0x7fc838010380 con 0x7fc8480f1e00 2024-04-17T22:29:10.635 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.649 7fc84d9c4700 1 -- 172.21.15.102:0/4188557705 --> v1:172.21.15.102:6816/42360 -- command(tid 0: {"prefix": "pg dump", "target": ["mgr", ""], "format": "json"}) v1 -- 0x7fc84808a300 con 0x7fc8240145a0 2024-04-17T22:29:10.643 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.657 7fc8377fe700 1 -- 172.21.15.102:0/4188557705 <== mgr.4104 v1:172.21.15.102:6816/42360 1 ==== command_reply(tid 0: 0 dumped all) v1 ==== 18+0+299651 (unknown 2965378022 0 3595174762) 0x7fc83c006950 con 0x7fc8240145a0 2024-04-17T22:29:10.648 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-17T22:29:10.651 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.659 7fc84d9c4700 1 -- 172.21.15.102:0/4188557705 >> v1:172.21.15.102:6816/42360 conn(0x7fc8240145a0 legacy=0x7fc824016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:10.651 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.659 7fc84d9c4700 1 -- 172.21.15.102:0/4188557705 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fc8480f1e00 msgr2=0x7fc8481ab770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:10.651 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.659 7fc84d9c4700 1 --2- 172.21.15.102:0/4188557705 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fc8480f1e00 0x7fc8481ab770 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0x7fc83800c000 tx=0x7fc83800ebc0).stop 2024-04-17T22:29:10.652 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.659 7fc84d9c4700 1 -- 172.21.15.102:0/4188557705 shutdown_connections 2024-04-17T22:29:10.652 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.659 7fc84d9c4700 1 --2- 172.21.15.102:0/4188557705 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc8480f2250 0x7fc8481abda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:10.652 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.659 7fc84d9c4700 1 --2- 172.21.15.102:0/4188557705 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc8480ce6f0 0x7fc8480f17d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:10.652 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.659 7fc84d9c4700 1 --2- 172.21.15.102:0/4188557705 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fc8480f1e00 0x7fc8481ab770 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:10.652 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.659 7fc84d9c4700 1 -- 172.21.15.102:0/4188557705 >> 172.21.15.102:0/4188557705 conn(0x7fc8480c6a60 msgr2=0x7fc8480c7290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:29:10.652 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.659 7fc84d9c4700 1 -- 172.21.15.102:0/4188557705 shutdown_connections 2024-04-17T22:29:10.652 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.659 7fc84d9c4700 1 -- 172.21.15.102:0/4188557705 wait complete. 2024-04-17T22:29:10.652 INFO:teuthology.orchestra.run.smithi102.stderr:dumped all 2024-04-17T22:29:10.657 INFO:teuthology.orchestra.run.smithi102.stdout:{"pg_ready":true,"pg_map":{"version":20,"stamp":"2024-04-17 22:29:09.187086","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":4,"num_per_pool_osds":4,"kb":377487360,"kb_used":4194752,"kb_used_data":416,"kb_used_omap":0,"kb_used_meta":4194304,"kb_avail":373292608,"statfs":{"total":386547056640,"available":382251630592,"internally_reserved":4294967296,"allocated":425984,"data_stored":97524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4294967296},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"3.1a","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:07.435832","last_change":"2024-04-17 22:29:07.435832","last_active":"2024-04-17 22:29:07.435832","last_peered":"2024-04-17 22:29:07.435832","last_clean":"2024-04-17 22:29:07.435832","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:07.435832","last_undegraded":"2024-04-17 22:29:07.435832","last_fullsized":"2024-04-17 22:29:07.435832","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:06.425872","last_change":"2024-04-17 22:29:06.425872","last_active":"2024-04-17 22:29:06.425872","last_peered":"2024-04-17 22:29:06.425872","last_clean":"2024-04-17 22:29:06.425872","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:06.425872","last_undegraded":"2024-04-17 22:29:06.425872","last_fullsized":"2024-04-17 22:29:06.425872","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:07.435832","last_change":"2024-04-17 22:29:07.435832","last_active":"2024-04-17 22:29:07.435832","last_peered":"2024-04-17 22:29:07.435832","last_clean":"2024-04-17 22:29:07.435832","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:07.435832","last_undegraded":"2024-04-17 22:29:07.435832","last_fullsized":"2024-04-17 22:29:07.435832","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:06.425872","last_change":"2024-04-17 22:29:06.425872","last_active":"2024-04-17 22:29:06.425872","last_peered":"2024-04-17 22:29:06.425872","last_clean":"2024-04-17 22:29:06.425872","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:06.425872","last_undegraded":"2024-04-17 22:29:06.425872","last_fullsized":"2024-04-17 22:29:06.425872","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:07.435832","last_change":"2024-04-17 22:29:07.435832","last_active":"2024-04-17 22:29:07.435832","last_peered":"2024-04-17 22:29:07.435832","last_clean":"2024-04-17 22:29:07.435832","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:07.435832","last_undegraded":"2024-04-17 22:29:07.435832","last_fullsized":"2024-04-17 22:29:07.435832","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:06.425872","last_change":"2024-04-17 22:29:06.425872","last_active":"2024-04-17 22:29:06.425872","last_peered":"2024-04-17 22:29:06.425872","last_clean":"2024-04-17 22:29:06.425872","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:06.425872","last_undegraded":"2024-04-17 22:29:06.425872","last_fullsized":"2024-04-17 22:29:06.425872","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:07.435832","last_change":"2024-04-17 22:29:07.435832","last_active":"2024-04-17 22:29:07.435832","last_peered":"2024-04-17 22:29:07.435832","last_clean":"2024-04-17 22:29:07.435832","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:07.435832","last_undegraded":"2024-04-17 22:29:07.435832","last_fullsized":"2024-04-17 22:29:07.435832","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:06.425872","last_change":"2024-04-17 22:29:06.425872","last_active":"2024-04-17 22:29:06.425872","last_peered":"2024-04-17 22:29:06.425872","last_clean":"2024-04-17 22:29:06.425872","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:06.425872","last_undegraded":"2024-04-17 22:29:06.425872","last_fullsized":"2024-04-17 22:29:06.425872","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:07.435832","last_change":"2024-04-17 22:29:07.435832","last_active":"2024-04-17 22:29:07.435832","last_peered":"2024-04-17 22:29:07.435832","last_clean":"2024-04-17 22:29:07.435832","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:07.435832","last_undegraded":"2024-04-17 22:29:07.435832","last_fullsized":"2024-04-17 22:29:07.435832","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:06.425872","last_change":"2024-04-17 22:29:06.425872","last_active":"2024-04-17 22:29:06.425872","last_peered":"2024-04-17 22:29:06.425872","last_clean":"2024-04-17 22:29:06.425872","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:06.425872","last_undegraded":"2024-04-17 22:29:06.425872","last_fullsized":"2024-04-17 22:29:06.425872","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:07.435832","last_change":"2024-04-17 22:29:07.435832","last_active":"2024-04-17 22:29:07.435832","last_peered":"2024-04-17 22:29:07.435832","last_clean":"2024-04-17 22:29:07.435832","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:07.435832","last_undegraded":"2024-04-17 22:29:07.435832","last_fullsized":"2024-04-17 22:29:07.435832","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:06.425872","last_change":"2024-04-17 22:29:06.425872","last_active":"2024-04-17 22:29:06.425872","last_peered":"2024-04-17 22:29:06.425872","last_clean":"2024-04-17 22:29:06.425872","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:06.425872","last_undegraded":"2024-04-17 22:29:06.425872","last_fullsized":"2024-04-17 22:29:06.425872","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:07.435832","last_change":"2024-04-17 22:29:07.435832","last_active":"2024-04-17 22:29:07.435832","last_peered":"2024-04-17 22:29:07.435832","last_clean":"2024-04-17 22:29:07.435832","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:07.435832","last_undegraded":"2024-04-17 22:29:07.435832","last_fullsized":"2024-04-17 22:29:07.435832","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:06.425872","last_change":"2024-04-17 22:29:06.425872","last_active":"2024-04-17 22:29:06.425872","last_peered":"2024-04-17 22:29:06.425872","last_clean":"2024-04-17 22:29:06.425872","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:06.425872","last_undegraded":"2024-04-17 22:29:06.425872","last_fullsized":"2024-04-17 22:29:06.425872","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:07.435832","last_change":"2024-04-17 22:29:07.435832","last_active":"2024-04-17 22:29:07.435832","last_peered":"2024-04-17 22:29:07.435832","last_clean":"2024-04-17 22:29:07.435832","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:07.435832","last_undegraded":"2024-04-17 22:29:07.435832","last_fullsized":"2024-04-17 22:29:07.435832","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:06.425872","last_change":"2024-04-17 22:29:06.425872","last_active":"2024-04-17 22:29:06.425872","last_peered":"2024-04-17 22:29:06.425872","last_clean":"2024-04-17 22:29:06.425872","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:06.425872","last_undegraded":"2024-04-17 22:29:06.425872","last_fullsized":"2024-04-17 22:29:06.425872","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:07.435832","last_change":"2024-04-17 22:29:07.435832","last_active":"2024-04-17 22:29:07.435832","last_peered":"2024-04-17 22:29:07.435832","last_clean":"2024-04-17 22:29:07.435832","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:07.435832","last_undegraded":"2024-04-17 22:29:07.435832","last_fullsized":"2024-04-17 22:29:07.435832","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:06.425872","last_change":"2024-04-17 22:29:06.425872","last_active":"2024-04-17 22:29:06.425872","last_peered":"2024-04-17 22:29:06.425872","last_clean":"2024-04-17 22:29:06.425872","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:06.425872","last_undegraded":"2024-04-17 22:29:06.425872","last_fullsized":"2024-04-17 22:29:06.425872","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:07.435832","last_change":"2024-04-17 22:29:07.435832","last_active":"2024-04-17 22:29:07.435832","last_peered":"2024-04-17 22:29:07.435832","last_clean":"2024-04-17 22:29:07.435832","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:07.435832","last_undegraded":"2024-04-17 22:29:07.435832","last_fullsized":"2024-04-17 22:29:07.435832","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:06.425872","last_change":"2024-04-17 22:29:06.425872","last_active":"2024-04-17 22:29:06.425872","last_peered":"2024-04-17 22:29:06.425872","last_clean":"2024-04-17 22:29:06.425872","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:06.425872","last_undegraded":"2024-04-17 22:29:06.425872","last_fullsized":"2024-04-17 22:29:06.425872","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:07.435832","last_change":"2024-04-17 22:29:07.435832","last_active":"2024-04-17 22:29:07.435832","last_peered":"2024-04-17 22:29:07.435832","last_clean":"2024-04-17 22:29:07.435832","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:07.435832","last_undegraded":"2024-04-17 22:29:07.435832","last_fullsized":"2024-04-17 22:29:07.435832","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:06.425872","last_change":"2024-04-17 22:29:06.425872","last_active":"2024-04-17 22:29:06.425872","last_peered":"2024-04-17 22:29:06.425872","last_clean":"2024-04-17 22:29:06.425872","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:06.425872","last_undegraded":"2024-04-17 22:29:06.425872","last_fullsized":"2024-04-17 22:29:06.425872","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:07.435832","last_change":"2024-04-17 22:29:07.435832","last_active":"2024-04-17 22:29:07.435832","last_peered":"2024-04-17 22:29:07.435832","last_clean":"2024-04-17 22:29:07.435832","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:07.435832","last_undegraded":"2024-04-17 22:29:07.435832","last_fullsized":"2024-04-17 22:29:07.435832","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:06.425872","last_change":"2024-04-17 22:29:06.425872","last_active":"2024-04-17 22:29:06.425872","last_peered":"2024-04-17 22:29:06.425872","last_clean":"2024-04-17 22:29:06.425872","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:06.425872","last_undegraded":"2024-04-17 22:29:06.425872","last_fullsized":"2024-04-17 22:29:06.425872","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:07.435832","last_change":"2024-04-17 22:29:07.435832","last_active":"2024-04-17 22:29:07.435832","last_peered":"2024-04-17 22:29:07.435832","last_clean":"2024-04-17 22:29:07.435832","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:07.435832","last_undegraded":"2024-04-17 22:29:07.435832","last_fullsized":"2024-04-17 22:29:07.435832","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:06.425872","last_change":"2024-04-17 22:29:06.425872","last_active":"2024-04-17 22:29:06.425872","last_peered":"2024-04-17 22:29:06.425872","last_clean":"2024-04-17 22:29:06.425872","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:06.425872","last_undegraded":"2024-04-17 22:29:06.425872","last_fullsized":"2024-04-17 22:29:06.425872","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:07.435832","last_change":"2024-04-17 22:29:07.435832","last_active":"2024-04-17 22:29:07.435832","last_peered":"2024-04-17 22:29:07.435832","last_clean":"2024-04-17 22:29:07.435832","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:07.435832","last_undegraded":"2024-04-17 22:29:07.435832","last_fullsized":"2024-04-17 22:29:07.435832","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:06.425872","last_change":"2024-04-17 22:29:06.425872","last_active":"2024-04-17 22:29:06.425872","last_peered":"2024-04-17 22:29:06.425872","last_clean":"2024-04-17 22:29:06.425872","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:06.425872","last_undegraded":"2024-04-17 22:29:06.425872","last_fullsized":"2024-04-17 22:29:06.425872","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:07.435832","last_change":"2024-04-17 22:29:07.435832","last_active":"2024-04-17 22:29:07.435832","last_peered":"2024-04-17 22:29:07.435832","last_clean":"2024-04-17 22:29:07.435832","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:07.435832","last_undegraded":"2024-04-17 22:29:07.435832","last_fullsized":"2024-04-17 22:29:07.435832","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:06.425872","last_change":"2024-04-17 22:29:06.425872","last_active":"2024-04-17 22:29:06.425872","last_peered":"2024-04-17 22:29:06.425872","last_clean":"2024-04-17 22:29:06.425872","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:06.425872","last_undegraded":"2024-04-17 22:29:06.425872","last_fullsized":"2024-04-17 22:29:06.425872","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:07.435832","last_change":"2024-04-17 22:29:07.435832","last_active":"2024-04-17 22:29:07.435832","last_peered":"2024-04-17 22:29:07.435832","last_clean":"2024-04-17 22:29:07.435832","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:07.435832","last_undegraded":"2024-04-17 22:29:07.435832","last_fullsized":"2024-04-17 22:29:07.435832","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:06.425872","last_change":"2024-04-17 22:29:06.425872","last_active":"2024-04-17 22:29:06.425872","last_peered":"2024-04-17 22:29:06.425872","last_clean":"2024-04-17 22:29:06.425872","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:06.425872","last_undegraded":"2024-04-17 22:29:06.425872","last_fullsized":"2024-04-17 22:29:06.425872","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:07.435832","last_change":"2024-04-17 22:29:07.435832","last_active":"2024-04-17 22:29:07.435832","last_peered":"2024-04-17 22:29:07.435832","last_clean":"2024-04-17 22:29:07.435832","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:07.435832","last_undegraded":"2024-04-17 22:29:07.435832","last_fullsized":"2024-04-17 22:29:07.435832","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:06.425872","last_change":"2024-04-17 22:29:06.425872","last_active":"2024-04-17 22:29:06.425872","last_peered":"2024-04-17 22:29:06.425872","last_clean":"2024-04-17 22:29:06.425872","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:06.425872","last_undegraded":"2024-04-17 22:29:06.425872","last_fullsized":"2024-04-17 22:29:06.425872","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:07.435832","last_change":"2024-04-17 22:29:07.435832","last_active":"2024-04-17 22:29:07.435832","last_peered":"2024-04-17 22:29:07.435832","last_clean":"2024-04-17 22:29:07.435832","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:07.435832","last_undegraded":"2024-04-17 22:29:07.435832","last_fullsized":"2024-04-17 22:29:07.435832","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:06.425872","last_change":"2024-04-17 22:29:06.425872","last_active":"2024-04-17 22:29:06.425872","last_peered":"2024-04-17 22:29:06.425872","last_clean":"2024-04-17 22:29:06.425872","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:06.425872","last_undegraded":"2024-04-17 22:29:06.425872","last_fullsized":"2024-04-17 22:29:06.425872","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:07.435832","last_change":"2024-04-17 22:29:07.435832","last_active":"2024-04-17 22:29:07.435832","last_peered":"2024-04-17 22:29:07.435832","last_clean":"2024-04-17 22:29:07.435832","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:07.435832","last_undegraded":"2024-04-17 22:29:07.435832","last_fullsized":"2024-04-17 22:29:07.435832","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:06.425872","last_change":"2024-04-17 22:29:06.425872","last_active":"2024-04-17 22:29:06.425872","last_peered":"2024-04-17 22:29:06.425872","last_clean":"2024-04-17 22:29:06.425872","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:06.425872","last_undegraded":"2024-04-17 22:29:06.425872","last_fullsized":"2024-04-17 22:29:06.425872","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:07.435832","last_change":"2024-04-17 22:29:07.435832","last_active":"2024-04-17 22:29:07.435832","last_peered":"2024-04-17 22:29:07.435832","last_clean":"2024-04-17 22:29:07.435832","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:07.435832","last_undegraded":"2024-04-17 22:29:07.435832","last_fullsized":"2024-04-17 22:29:07.435832","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:06.425872","last_change":"2024-04-17 22:29:06.425872","last_active":"2024-04-17 22:29:06.425872","last_peered":"2024-04-17 22:29:06.425872","last_clean":"2024-04-17 22:29:06.425872","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:06.425872","last_undegraded":"2024-04-17 22:29:06.425872","last_fullsized":"2024-04-17 22:29:06.425872","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:07.435832","last_change":"2024-04-17 22:29:07.435832","last_active":"2024-04-17 22:29:07.435832","last_peered":"2024-04-17 22:29:07.435832","last_clean":"2024-04-17 22:29:07.435832","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:07.435832","last_undegraded":"2024-04-17 22:29:07.435832","last_fullsized":"2024-04-17 22:29:07.435832","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":"9","reported_epoch":"11","state":"active+clean","last_fresh":"2024-04-17 22:28:59.256589","last_change":"2024-04-17 22:28:59.253977","last_active":"2024-04-17 22:28:59.256589","last_peered":"2024-04-17 22:28:59.256589","last_clean":"2024-04-17 22:28:59.256589","last_became_active":"2024-04-17 22:28:59.253641","last_became_peered":"2024-04-17 22:28:59.253641","last_unstale":"2024-04-17 22:28:59.256589","last_undegraded":"2024-04-17 22:28:59.256589","last_fullsized":"2024-04-17 22:28:59.256589","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:28:58.239187","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:28:58.239187","last_clean_scrub_stamp":"2024-04-17 22:28:58.239187","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:06.425872","last_change":"2024-04-17 22:29:06.425872","last_active":"2024-04-17 22:29:06.425872","last_peered":"2024-04-17 22:29:06.425872","last_clean":"2024-04-17 22:29:06.425872","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:06.425872","last_undegraded":"2024-04-17 22:29:06.425872","last_fullsized":"2024-04-17 22:29:06.425872","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:07.435832","last_change":"2024-04-17 22:29:07.435832","last_active":"2024-04-17 22:29:07.435832","last_peered":"2024-04-17 22:29:07.435832","last_clean":"2024-04-17 22:29:07.435832","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:07.435832","last_undegraded":"2024-04-17 22:29:07.435832","last_fullsized":"2024-04-17 22:29:07.435832","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":"9","reported_epoch":"11","state":"active+clean","last_fresh":"2024-04-17 22:28:59.257288","last_change":"2024-04-17 22:28:59.255733","last_active":"2024-04-17 22:28:59.257288","last_peered":"2024-04-17 22:28:59.257288","last_clean":"2024-04-17 22:28:59.257288","last_became_active":"2024-04-17 22:28:59.255411","last_became_peered":"2024-04-17 22:28:59.255411","last_unstale":"2024-04-17 22:28:59.257288","last_undegraded":"2024-04-17 22:28:59.257288","last_fullsized":"2024-04-17 22:28:59.257288","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:28:58.239187","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:28:58.239187","last_clean_scrub_stamp":"2024-04-17 22:28:58.239187","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:06.425872","last_change":"2024-04-17 22:29:06.425872","last_active":"2024-04-17 22:29:06.425872","last_peered":"2024-04-17 22:29:06.425872","last_clean":"2024-04-17 22:29:06.425872","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:06.425872","last_undegraded":"2024-04-17 22:29:06.425872","last_fullsized":"2024-04-17 22:29:06.425872","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":"9","reported_epoch":"11","state":"active+clean","last_fresh":"2024-04-17 22:28:59.254418","last_change":"2024-04-17 22:28:59.252728","last_active":"2024-04-17 22:28:59.254418","last_peered":"2024-04-17 22:28:59.254418","last_clean":"2024-04-17 22:28:59.254418","last_became_active":"2024-04-17 22:28:59.252345","last_became_peered":"2024-04-17 22:28:59.252345","last_unstale":"2024-04-17 22:28:59.254418","last_undegraded":"2024-04-17 22:28:59.254418","last_fullsized":"2024-04-17 22:28:59.254418","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:28:58.239187","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:28:58.239187","last_clean_scrub_stamp":"2024-04-17 22:28:58.239187","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:06.425872","last_change":"2024-04-17 22:29:06.425872","last_active":"2024-04-17 22:29:06.425872","last_peered":"2024-04-17 22:29:06.425872","last_clean":"2024-04-17 22:29:06.425872","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:06.425872","last_undegraded":"2024-04-17 22:29:06.425872","last_fullsized":"2024-04-17 22:29:06.425872","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:07.435832","last_change":"2024-04-17 22:29:07.435832","last_active":"2024-04-17 22:29:07.435832","last_peered":"2024-04-17 22:29:07.435832","last_clean":"2024-04-17 22:29:07.435832","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:07.435832","last_undegraded":"2024-04-17 22:29:07.435832","last_fullsized":"2024-04-17 22:29:07.435832","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":"9","reported_epoch":"11","state":"active+clean","last_fresh":"2024-04-17 22:28:59.254870","last_change":"2024-04-17 22:28:59.253392","last_active":"2024-04-17 22:28:59.254870","last_peered":"2024-04-17 22:28:59.254870","last_clean":"2024-04-17 22:28:59.254870","last_became_active":"2024-04-17 22:28:59.253049","last_became_peered":"2024-04-17 22:28:59.253049","last_unstale":"2024-04-17 22:28:59.254870","last_undegraded":"2024-04-17 22:28:59.254870","last_fullsized":"2024-04-17 22:28:59.254870","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:28:58.239187","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:28:58.239187","last_clean_scrub_stamp":"2024-04-17 22:28:58.239187","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:06.425872","last_change":"2024-04-17 22:29:06.425872","last_active":"2024-04-17 22:29:06.425872","last_peered":"2024-04-17 22:29:06.425872","last_clean":"2024-04-17 22:29:06.425872","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:06.425872","last_undegraded":"2024-04-17 22:29:06.425872","last_fullsized":"2024-04-17 22:29:06.425872","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:07.435832","last_change":"2024-04-17 22:29:07.435832","last_active":"2024-04-17 22:29:07.435832","last_peered":"2024-04-17 22:29:07.435832","last_clean":"2024-04-17 22:29:07.435832","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:07.435832","last_undegraded":"2024-04-17 22:29:07.435832","last_fullsized":"2024-04-17 22:29:07.435832","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":"9","reported_epoch":"11","state":"active+clean","last_fresh":"2024-04-17 22:28:59.255890","last_change":"2024-04-17 22:28:59.254907","last_active":"2024-04-17 22:28:59.255890","last_peered":"2024-04-17 22:28:59.255890","last_clean":"2024-04-17 22:28:59.255890","last_became_active":"2024-04-17 22:28:59.254633","last_became_peered":"2024-04-17 22:28:59.254633","last_unstale":"2024-04-17 22:28:59.255890","last_undegraded":"2024-04-17 22:28:59.255890","last_fullsized":"2024-04-17 22:28:59.255890","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:28:58.239187","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:28:58.239187","last_clean_scrub_stamp":"2024-04-17 22:28:58.239187","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:06.425872","last_change":"2024-04-17 22:29:06.425872","last_active":"2024-04-17 22:29:06.425872","last_peered":"2024-04-17 22:29:06.425872","last_clean":"2024-04-17 22:29:06.425872","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:06.425872","last_undegraded":"2024-04-17 22:29:06.425872","last_fullsized":"2024-04-17 22:29:06.425872","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:07.435832","last_change":"2024-04-17 22:29:07.435832","last_active":"2024-04-17 22:29:07.435832","last_peered":"2024-04-17 22:29:07.435832","last_clean":"2024-04-17 22:29:07.435832","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:07.435832","last_undegraded":"2024-04-17 22:29:07.435832","last_fullsized":"2024-04-17 22:29:07.435832","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":"9","reported_epoch":"11","state":"active+clean","last_fresh":"2024-04-17 22:28:59.255467","last_change":"2024-04-17 22:28:59.254217","last_active":"2024-04-17 22:28:59.255467","last_peered":"2024-04-17 22:28:59.255467","last_clean":"2024-04-17 22:28:59.255467","last_became_active":"2024-04-17 22:28:59.253877","last_became_peered":"2024-04-17 22:28:59.253877","last_unstale":"2024-04-17 22:28:59.255467","last_undegraded":"2024-04-17 22:28:59.255467","last_fullsized":"2024-04-17 22:28:59.255467","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:28:58.239187","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:28:58.239187","last_clean_scrub_stamp":"2024-04-17 22:28:58.239187","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:06.425872","last_change":"2024-04-17 22:29:06.425872","last_active":"2024-04-17 22:29:06.425872","last_peered":"2024-04-17 22:29:06.425872","last_clean":"2024-04-17 22:29:06.425872","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:06.425872","last_undegraded":"2024-04-17 22:29:06.425872","last_fullsized":"2024-04-17 22:29:06.425872","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:07.435832","last_change":"2024-04-17 22:29:07.435832","last_active":"2024-04-17 22:29:07.435832","last_peered":"2024-04-17 22:29:07.435832","last_clean":"2024-04-17 22:29:07.435832","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:07.435832","last_undegraded":"2024-04-17 22:29:07.435832","last_fullsized":"2024-04-17 22:29:07.435832","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":"9","reported_epoch":"11","state":"active+clean","last_fresh":"2024-04-17 22:28:59.255985","last_change":"2024-04-17 22:28:59.254830","last_active":"2024-04-17 22:28:59.255985","last_peered":"2024-04-17 22:28:59.255985","last_clean":"2024-04-17 22:28:59.255985","last_became_active":"2024-04-17 22:28:59.254505","last_became_peered":"2024-04-17 22:28:59.254505","last_unstale":"2024-04-17 22:28:59.255985","last_undegraded":"2024-04-17 22:28:59.255985","last_fullsized":"2024-04-17 22:28:59.255985","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:28:58.239187","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:28:58.239187","last_clean_scrub_stamp":"2024-04-17 22:28:58.239187","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:06.425872","last_change":"2024-04-17 22:29:06.425872","last_active":"2024-04-17 22:29:06.425872","last_peered":"2024-04-17 22:29:06.425872","last_clean":"2024-04-17 22:29:06.425872","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:06.425872","last_undegraded":"2024-04-17 22:29:06.425872","last_fullsized":"2024-04-17 22:29:06.425872","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:07.435832","last_change":"2024-04-17 22:29:07.435832","last_active":"2024-04-17 22:29:07.435832","last_peered":"2024-04-17 22:29:07.435832","last_clean":"2024-04-17 22:29:07.435832","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:07.435832","last_undegraded":"2024-04-17 22:29:07.435832","last_fullsized":"2024-04-17 22:29:07.435832","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":"9","reported_epoch":"11","state":"active+clean","last_fresh":"2024-04-17 22:28:59.257265","last_change":"2024-04-17 22:28:59.254055","last_active":"2024-04-17 22:28:59.257265","last_peered":"2024-04-17 22:28:59.257265","last_clean":"2024-04-17 22:28:59.257265","last_became_active":"2024-04-17 22:28:59.253750","last_became_peered":"2024-04-17 22:28:59.253750","last_unstale":"2024-04-17 22:28:59.257265","last_undegraded":"2024-04-17 22:28:59.257265","last_fullsized":"2024-04-17 22:28:59.257265","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:28:58.239187","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:28:58.239187","last_clean_scrub_stamp":"2024-04-17 22:28:58.239187","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:06.425872","last_change":"2024-04-17 22:29:06.425872","last_active":"2024-04-17 22:29:06.425872","last_peered":"2024-04-17 22:29:06.425872","last_clean":"2024-04-17 22:29:06.425872","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:06.425872","last_undegraded":"2024-04-17 22:29:06.425872","last_fullsized":"2024-04-17 22:29:06.425872","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:07.435832","last_change":"2024-04-17 22:29:07.435832","last_active":"2024-04-17 22:29:07.435832","last_peered":"2024-04-17 22:29:07.435832","last_clean":"2024-04-17 22:29:07.435832","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:07.435832","last_undegraded":"2024-04-17 22:29:07.435832","last_fullsized":"2024-04-17 22:29:07.435832","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:07.435832","last_change":"2024-04-17 22:29:07.435832","last_active":"2024-04-17 22:29:07.435832","last_peered":"2024-04-17 22:29:07.435832","last_clean":"2024-04-17 22:29:07.435832","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:07.435832","last_undegraded":"2024-04-17 22:29:07.435832","last_fullsized":"2024-04-17 22:29:07.435832","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:06.425872","last_change":"2024-04-17 22:29:06.425872","last_active":"2024-04-17 22:29:06.425872","last_peered":"2024-04-17 22:29:06.425872","last_clean":"2024-04-17 22:29:06.425872","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:06.425872","last_undegraded":"2024-04-17 22:29:06.425872","last_fullsized":"2024-04-17 22:29:06.425872","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:07.435832","last_change":"2024-04-17 22:29:07.435832","last_active":"2024-04-17 22:29:07.435832","last_peered":"2024-04-17 22:29:07.435832","last_clean":"2024-04-17 22:29:07.435832","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:07.435832","last_undegraded":"2024-04-17 22:29:07.435832","last_fullsized":"2024-04-17 22:29:07.435832","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:06.425872","last_change":"2024-04-17 22:29:06.425872","last_active":"2024-04-17 22:29:06.425872","last_peered":"2024-04-17 22:29:06.425872","last_clean":"2024-04-17 22:29:06.425872","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:06.425872","last_undegraded":"2024-04-17 22:29:06.425872","last_fullsized":"2024-04-17 22:29:06.425872","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:07.435832","last_change":"2024-04-17 22:29:07.435832","last_active":"2024-04-17 22:29:07.435832","last_peered":"2024-04-17 22:29:07.435832","last_clean":"2024-04-17 22:29:07.435832","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:07.435832","last_undegraded":"2024-04-17 22:29:07.435832","last_fullsized":"2024-04-17 22:29:07.435832","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:06.425872","last_change":"2024-04-17 22:29:06.425872","last_active":"2024-04-17 22:29:06.425872","last_peered":"2024-04-17 22:29:06.425872","last_clean":"2024-04-17 22:29:06.425872","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:06.425872","last_undegraded":"2024-04-17 22:29:06.425872","last_fullsized":"2024-04-17 22:29:06.425872","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:07.435832","last_change":"2024-04-17 22:29:07.435832","last_active":"2024-04-17 22:29:07.435832","last_peered":"2024-04-17 22:29:07.435832","last_clean":"2024-04-17 22:29:07.435832","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:07.435832","last_undegraded":"2024-04-17 22:29:07.435832","last_fullsized":"2024-04-17 22:29:07.435832","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:06.425872","last_change":"2024-04-17 22:29:06.425872","last_active":"2024-04-17 22:29:06.425872","last_peered":"2024-04-17 22:29:06.425872","last_clean":"2024-04-17 22:29:06.425872","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:06.425872","last_undegraded":"2024-04-17 22:29:06.425872","last_fullsized":"2024-04-17 22:29:06.425872","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:07.435832","last_change":"2024-04-17 22:29:07.435832","last_active":"2024-04-17 22:29:07.435832","last_peered":"2024-04-17 22:29:07.435832","last_clean":"2024-04-17 22:29:07.435832","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:07.435832","last_undegraded":"2024-04-17 22:29:07.435832","last_fullsized":"2024-04-17 22:29:07.435832","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.20","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:06.425872","last_change":"2024-04-17 22:29:06.425872","last_active":"2024-04-17 22:29:06.425872","last_peered":"2024-04-17 22:29:06.425872","last_clean":"2024-04-17 22:29:06.425872","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:06.425872","last_undegraded":"2024-04-17 22:29:06.425872","last_fullsized":"2024-04-17 22:29:06.425872","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:07.435832","last_change":"2024-04-17 22:29:07.435832","last_active":"2024-04-17 22:29:07.435832","last_peered":"2024-04-17 22:29:07.435832","last_clean":"2024-04-17 22:29:07.435832","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:07.435832","last_undegraded":"2024-04-17 22:29:07.435832","last_fullsized":"2024-04-17 22:29:07.435832","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.21","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:06.425872","last_change":"2024-04-17 22:29:06.425872","last_active":"2024-04-17 22:29:06.425872","last_peered":"2024-04-17 22:29:06.425872","last_clean":"2024-04-17 22:29:06.425872","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:06.425872","last_undegraded":"2024-04-17 22:29:06.425872","last_fullsized":"2024-04-17 22:29:06.425872","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:07.435832","last_change":"2024-04-17 22:29:07.435832","last_active":"2024-04-17 22:29:07.435832","last_peered":"2024-04-17 22:29:07.435832","last_clean":"2024-04-17 22:29:07.435832","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:07.435832","last_undegraded":"2024-04-17 22:29:07.435832","last_fullsized":"2024-04-17 22:29:07.435832","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.22","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:06.425872","last_change":"2024-04-17 22:29:06.425872","last_active":"2024-04-17 22:29:06.425872","last_peered":"2024-04-17 22:29:06.425872","last_clean":"2024-04-17 22:29:06.425872","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:06.425872","last_undegraded":"2024-04-17 22:29:06.425872","last_fullsized":"2024-04-17 22:29:06.425872","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:07.435832","last_change":"2024-04-17 22:29:07.435832","last_active":"2024-04-17 22:29:07.435832","last_peered":"2024-04-17 22:29:07.435832","last_clean":"2024-04-17 22:29:07.435832","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:07.435832","last_undegraded":"2024-04-17 22:29:07.435832","last_fullsized":"2024-04-17 22:29:07.435832","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.23","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:06.425872","last_change":"2024-04-17 22:29:06.425872","last_active":"2024-04-17 22:29:06.425872","last_peered":"2024-04-17 22:29:06.425872","last_clean":"2024-04-17 22:29:06.425872","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:06.425872","last_undegraded":"2024-04-17 22:29:06.425872","last_fullsized":"2024-04-17 22:29:06.425872","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:07.435832","last_change":"2024-04-17 22:29:07.435832","last_active":"2024-04-17 22:29:07.435832","last_peered":"2024-04-17 22:29:07.435832","last_clean":"2024-04-17 22:29:07.435832","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:07.435832","last_undegraded":"2024-04-17 22:29:07.435832","last_fullsized":"2024-04-17 22:29:07.435832","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.24","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:06.425872","last_change":"2024-04-17 22:29:06.425872","last_active":"2024-04-17 22:29:06.425872","last_peered":"2024-04-17 22:29:06.425872","last_clean":"2024-04-17 22:29:06.425872","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:06.425872","last_undegraded":"2024-04-17 22:29:06.425872","last_fullsized":"2024-04-17 22:29:06.425872","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:07.435832","last_change":"2024-04-17 22:29:07.435832","last_active":"2024-04-17 22:29:07.435832","last_peered":"2024-04-17 22:29:07.435832","last_clean":"2024-04-17 22:29:07.435832","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:07.435832","last_undegraded":"2024-04-17 22:29:07.435832","last_fullsized":"2024-04-17 22:29:07.435832","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.25","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:06.425872","last_change":"2024-04-17 22:29:06.425872","last_active":"2024-04-17 22:29:06.425872","last_peered":"2024-04-17 22:29:06.425872","last_clean":"2024-04-17 22:29:06.425872","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:06.425872","last_undegraded":"2024-04-17 22:29:06.425872","last_fullsized":"2024-04-17 22:29:06.425872","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:07.435832","last_change":"2024-04-17 22:29:07.435832","last_active":"2024-04-17 22:29:07.435832","last_peered":"2024-04-17 22:29:07.435832","last_clean":"2024-04-17 22:29:07.435832","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:07.435832","last_undegraded":"2024-04-17 22:29:07.435832","last_fullsized":"2024-04-17 22:29:07.435832","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.26","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:06.425872","last_change":"2024-04-17 22:29:06.425872","last_active":"2024-04-17 22:29:06.425872","last_peered":"2024-04-17 22:29:06.425872","last_clean":"2024-04-17 22:29:06.425872","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:06.425872","last_undegraded":"2024-04-17 22:29:06.425872","last_fullsized":"2024-04-17 22:29:06.425872","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:07.435832","last_change":"2024-04-17 22:29:07.435832","last_active":"2024-04-17 22:29:07.435832","last_peered":"2024-04-17 22:29:07.435832","last_clean":"2024-04-17 22:29:07.435832","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:07.435832","last_undegraded":"2024-04-17 22:29:07.435832","last_fullsized":"2024-04-17 22:29:07.435832","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.27","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:06.425872","last_change":"2024-04-17 22:29:06.425872","last_active":"2024-04-17 22:29:06.425872","last_peered":"2024-04-17 22:29:06.425872","last_clean":"2024-04-17 22:29:06.425872","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:06.425872","last_undegraded":"2024-04-17 22:29:06.425872","last_fullsized":"2024-04-17 22:29:06.425872","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:07.435832","last_change":"2024-04-17 22:29:07.435832","last_active":"2024-04-17 22:29:07.435832","last_peered":"2024-04-17 22:29:07.435832","last_clean":"2024-04-17 22:29:07.435832","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:07.435832","last_undegraded":"2024-04-17 22:29:07.435832","last_fullsized":"2024-04-17 22:29:07.435832","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.28","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:06.425872","last_change":"2024-04-17 22:29:06.425872","last_active":"2024-04-17 22:29:06.425872","last_peered":"2024-04-17 22:29:06.425872","last_clean":"2024-04-17 22:29:06.425872","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:06.425872","last_undegraded":"2024-04-17 22:29:06.425872","last_fullsized":"2024-04-17 22:29:06.425872","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:07.435832","last_change":"2024-04-17 22:29:07.435832","last_active":"2024-04-17 22:29:07.435832","last_peered":"2024-04-17 22:29:07.435832","last_clean":"2024-04-17 22:29:07.435832","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:07.435832","last_undegraded":"2024-04-17 22:29:07.435832","last_fullsized":"2024-04-17 22:29:07.435832","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.29","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:06.425872","last_change":"2024-04-17 22:29:06.425872","last_active":"2024-04-17 22:29:06.425872","last_peered":"2024-04-17 22:29:06.425872","last_clean":"2024-04-17 22:29:06.425872","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:06.425872","last_undegraded":"2024-04-17 22:29:06.425872","last_fullsized":"2024-04-17 22:29:06.425872","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:07.435832","last_change":"2024-04-17 22:29:07.435832","last_active":"2024-04-17 22:29:07.435832","last_peered":"2024-04-17 22:29:07.435832","last_clean":"2024-04-17 22:29:07.435832","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:07.435832","last_undegraded":"2024-04-17 22:29:07.435832","last_fullsized":"2024-04-17 22:29:07.435832","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2a","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:06.425872","last_change":"2024-04-17 22:29:06.425872","last_active":"2024-04-17 22:29:06.425872","last_peered":"2024-04-17 22:29:06.425872","last_clean":"2024-04-17 22:29:06.425872","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:06.425872","last_undegraded":"2024-04-17 22:29:06.425872","last_fullsized":"2024-04-17 22:29:06.425872","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:07.435832","last_change":"2024-04-17 22:29:07.435832","last_active":"2024-04-17 22:29:07.435832","last_peered":"2024-04-17 22:29:07.435832","last_clean":"2024-04-17 22:29:07.435832","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:07.435832","last_undegraded":"2024-04-17 22:29:07.435832","last_fullsized":"2024-04-17 22:29:07.435832","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2b","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:06.425872","last_change":"2024-04-17 22:29:06.425872","last_active":"2024-04-17 22:29:06.425872","last_peered":"2024-04-17 22:29:06.425872","last_clean":"2024-04-17 22:29:06.425872","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:06.425872","last_undegraded":"2024-04-17 22:29:06.425872","last_fullsized":"2024-04-17 22:29:06.425872","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:07.435832","last_change":"2024-04-17 22:29:07.435832","last_active":"2024-04-17 22:29:07.435832","last_peered":"2024-04-17 22:29:07.435832","last_clean":"2024-04-17 22:29:07.435832","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:07.435832","last_undegraded":"2024-04-17 22:29:07.435832","last_fullsized":"2024-04-17 22:29:07.435832","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2c","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:06.425872","last_change":"2024-04-17 22:29:06.425872","last_active":"2024-04-17 22:29:06.425872","last_peered":"2024-04-17 22:29:06.425872","last_clean":"2024-04-17 22:29:06.425872","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:06.425872","last_undegraded":"2024-04-17 22:29:06.425872","last_fullsized":"2024-04-17 22:29:06.425872","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:07.435832","last_change":"2024-04-17 22:29:07.435832","last_active":"2024-04-17 22:29:07.435832","last_peered":"2024-04-17 22:29:07.435832","last_clean":"2024-04-17 22:29:07.435832","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:07.435832","last_undegraded":"2024-04-17 22:29:07.435832","last_fullsized":"2024-04-17 22:29:07.435832","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2d","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:06.425872","last_change":"2024-04-17 22:29:06.425872","last_active":"2024-04-17 22:29:06.425872","last_peered":"2024-04-17 22:29:06.425872","last_clean":"2024-04-17 22:29:06.425872","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:06.425872","last_undegraded":"2024-04-17 22:29:06.425872","last_fullsized":"2024-04-17 22:29:06.425872","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:07.435832","last_change":"2024-04-17 22:29:07.435832","last_active":"2024-04-17 22:29:07.435832","last_peered":"2024-04-17 22:29:07.435832","last_clean":"2024-04-17 22:29:07.435832","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:07.435832","last_undegraded":"2024-04-17 22:29:07.435832","last_fullsized":"2024-04-17 22:29:07.435832","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2e","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:06.425872","last_change":"2024-04-17 22:29:06.425872","last_active":"2024-04-17 22:29:06.425872","last_peered":"2024-04-17 22:29:06.425872","last_clean":"2024-04-17 22:29:06.425872","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:06.425872","last_undegraded":"2024-04-17 22:29:06.425872","last_fullsized":"2024-04-17 22:29:06.425872","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:07.435832","last_change":"2024-04-17 22:29:07.435832","last_active":"2024-04-17 22:29:07.435832","last_peered":"2024-04-17 22:29:07.435832","last_clean":"2024-04-17 22:29:07.435832","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:07.435832","last_undegraded":"2024-04-17 22:29:07.435832","last_fullsized":"2024-04-17 22:29:07.435832","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2f","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:06.425872","last_change":"2024-04-17 22:29:06.425872","last_active":"2024-04-17 22:29:06.425872","last_peered":"2024-04-17 22:29:06.425872","last_clean":"2024-04-17 22:29:06.425872","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:06.425872","last_undegraded":"2024-04-17 22:29:06.425872","last_fullsized":"2024-04-17 22:29:06.425872","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:07.435832","last_change":"2024-04-17 22:29:07.435832","last_active":"2024-04-17 22:29:07.435832","last_peered":"2024-04-17 22:29:07.435832","last_clean":"2024-04-17 22:29:07.435832","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:07.435832","last_undegraded":"2024-04-17 22:29:07.435832","last_fullsized":"2024-04-17 22:29:07.435832","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.30","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:06.425872","last_change":"2024-04-17 22:29:06.425872","last_active":"2024-04-17 22:29:06.425872","last_peered":"2024-04-17 22:29:06.425872","last_clean":"2024-04-17 22:29:06.425872","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:06.425872","last_undegraded":"2024-04-17 22:29:06.425872","last_fullsized":"2024-04-17 22:29:06.425872","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:07.435832","last_change":"2024-04-17 22:29:07.435832","last_active":"2024-04-17 22:29:07.435832","last_peered":"2024-04-17 22:29:07.435832","last_clean":"2024-04-17 22:29:07.435832","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:07.435832","last_undegraded":"2024-04-17 22:29:07.435832","last_fullsized":"2024-04-17 22:29:07.435832","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.31","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:06.425872","last_change":"2024-04-17 22:29:06.425872","last_active":"2024-04-17 22:29:06.425872","last_peered":"2024-04-17 22:29:06.425872","last_clean":"2024-04-17 22:29:06.425872","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:06.425872","last_undegraded":"2024-04-17 22:29:06.425872","last_fullsized":"2024-04-17 22:29:06.425872","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:07.435832","last_change":"2024-04-17 22:29:07.435832","last_active":"2024-04-17 22:29:07.435832","last_peered":"2024-04-17 22:29:07.435832","last_clean":"2024-04-17 22:29:07.435832","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:07.435832","last_undegraded":"2024-04-17 22:29:07.435832","last_fullsized":"2024-04-17 22:29:07.435832","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.32","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:06.425872","last_change":"2024-04-17 22:29:06.425872","last_active":"2024-04-17 22:29:06.425872","last_peered":"2024-04-17 22:29:06.425872","last_clean":"2024-04-17 22:29:06.425872","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:06.425872","last_undegraded":"2024-04-17 22:29:06.425872","last_fullsized":"2024-04-17 22:29:06.425872","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:07.435832","last_change":"2024-04-17 22:29:07.435832","last_active":"2024-04-17 22:29:07.435832","last_peered":"2024-04-17 22:29:07.435832","last_clean":"2024-04-17 22:29:07.435832","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:07.435832","last_undegraded":"2024-04-17 22:29:07.435832","last_fullsized":"2024-04-17 22:29:07.435832","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.33","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:06.425872","last_change":"2024-04-17 22:29:06.425872","last_active":"2024-04-17 22:29:06.425872","last_peered":"2024-04-17 22:29:06.425872","last_clean":"2024-04-17 22:29:06.425872","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:06.425872","last_undegraded":"2024-04-17 22:29:06.425872","last_fullsized":"2024-04-17 22:29:06.425872","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:07.435832","last_change":"2024-04-17 22:29:07.435832","last_active":"2024-04-17 22:29:07.435832","last_peered":"2024-04-17 22:29:07.435832","last_clean":"2024-04-17 22:29:07.435832","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:07.435832","last_undegraded":"2024-04-17 22:29:07.435832","last_fullsized":"2024-04-17 22:29:07.435832","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.34","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:06.425872","last_change":"2024-04-17 22:29:06.425872","last_active":"2024-04-17 22:29:06.425872","last_peered":"2024-04-17 22:29:06.425872","last_clean":"2024-04-17 22:29:06.425872","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:06.425872","last_undegraded":"2024-04-17 22:29:06.425872","last_fullsized":"2024-04-17 22:29:06.425872","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:07.435832","last_change":"2024-04-17 22:29:07.435832","last_active":"2024-04-17 22:29:07.435832","last_peered":"2024-04-17 22:29:07.435832","last_clean":"2024-04-17 22:29:07.435832","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:07.435832","last_undegraded":"2024-04-17 22:29:07.435832","last_fullsized":"2024-04-17 22:29:07.435832","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.35","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:06.425872","last_change":"2024-04-17 22:29:06.425872","last_active":"2024-04-17 22:29:06.425872","last_peered":"2024-04-17 22:29:06.425872","last_clean":"2024-04-17 22:29:06.425872","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:06.425872","last_undegraded":"2024-04-17 22:29:06.425872","last_fullsized":"2024-04-17 22:29:06.425872","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:07.435832","last_change":"2024-04-17 22:29:07.435832","last_active":"2024-04-17 22:29:07.435832","last_peered":"2024-04-17 22:29:07.435832","last_clean":"2024-04-17 22:29:07.435832","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:07.435832","last_undegraded":"2024-04-17 22:29:07.435832","last_fullsized":"2024-04-17 22:29:07.435832","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.36","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:06.425872","last_change":"2024-04-17 22:29:06.425872","last_active":"2024-04-17 22:29:06.425872","last_peered":"2024-04-17 22:29:06.425872","last_clean":"2024-04-17 22:29:06.425872","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:06.425872","last_undegraded":"2024-04-17 22:29:06.425872","last_fullsized":"2024-04-17 22:29:06.425872","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:07.435832","last_change":"2024-04-17 22:29:07.435832","last_active":"2024-04-17 22:29:07.435832","last_peered":"2024-04-17 22:29:07.435832","last_clean":"2024-04-17 22:29:07.435832","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:07.435832","last_undegraded":"2024-04-17 22:29:07.435832","last_fullsized":"2024-04-17 22:29:07.435832","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.37","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:06.425872","last_change":"2024-04-17 22:29:06.425872","last_active":"2024-04-17 22:29:06.425872","last_peered":"2024-04-17 22:29:06.425872","last_clean":"2024-04-17 22:29:06.425872","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:06.425872","last_undegraded":"2024-04-17 22:29:06.425872","last_fullsized":"2024-04-17 22:29:06.425872","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:07.435832","last_change":"2024-04-17 22:29:07.435832","last_active":"2024-04-17 22:29:07.435832","last_peered":"2024-04-17 22:29:07.435832","last_clean":"2024-04-17 22:29:07.435832","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:07.435832","last_undegraded":"2024-04-17 22:29:07.435832","last_fullsized":"2024-04-17 22:29:07.435832","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.38","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:06.425872","last_change":"2024-04-17 22:29:06.425872","last_active":"2024-04-17 22:29:06.425872","last_peered":"2024-04-17 22:29:06.425872","last_clean":"2024-04-17 22:29:06.425872","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:06.425872","last_undegraded":"2024-04-17 22:29:06.425872","last_fullsized":"2024-04-17 22:29:06.425872","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:07.435832","last_change":"2024-04-17 22:29:07.435832","last_active":"2024-04-17 22:29:07.435832","last_peered":"2024-04-17 22:29:07.435832","last_clean":"2024-04-17 22:29:07.435832","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:07.435832","last_undegraded":"2024-04-17 22:29:07.435832","last_fullsized":"2024-04-17 22:29:07.435832","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.39","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:06.425872","last_change":"2024-04-17 22:29:06.425872","last_active":"2024-04-17 22:29:06.425872","last_peered":"2024-04-17 22:29:06.425872","last_clean":"2024-04-17 22:29:06.425872","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:06.425872","last_undegraded":"2024-04-17 22:29:06.425872","last_fullsized":"2024-04-17 22:29:06.425872","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:07.435832","last_change":"2024-04-17 22:29:07.435832","last_active":"2024-04-17 22:29:07.435832","last_peered":"2024-04-17 22:29:07.435832","last_clean":"2024-04-17 22:29:07.435832","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:07.435832","last_undegraded":"2024-04-17 22:29:07.435832","last_fullsized":"2024-04-17 22:29:07.435832","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.3a","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:06.425872","last_change":"2024-04-17 22:29:06.425872","last_active":"2024-04-17 22:29:06.425872","last_peered":"2024-04-17 22:29:06.425872","last_clean":"2024-04-17 22:29:06.425872","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:06.425872","last_undegraded":"2024-04-17 22:29:06.425872","last_fullsized":"2024-04-17 22:29:06.425872","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:07.435832","last_change":"2024-04-17 22:29:07.435832","last_active":"2024-04-17 22:29:07.435832","last_peered":"2024-04-17 22:29:07.435832","last_clean":"2024-04-17 22:29:07.435832","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:07.435832","last_undegraded":"2024-04-17 22:29:07.435832","last_fullsized":"2024-04-17 22:29:07.435832","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.3b","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:06.425872","last_change":"2024-04-17 22:29:06.425872","last_active":"2024-04-17 22:29:06.425872","last_peered":"2024-04-17 22:29:06.425872","last_clean":"2024-04-17 22:29:06.425872","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:06.425872","last_undegraded":"2024-04-17 22:29:06.425872","last_fullsized":"2024-04-17 22:29:06.425872","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:07.435832","last_change":"2024-04-17 22:29:07.435832","last_active":"2024-04-17 22:29:07.435832","last_peered":"2024-04-17 22:29:07.435832","last_clean":"2024-04-17 22:29:07.435832","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:07.435832","last_undegraded":"2024-04-17 22:29:07.435832","last_fullsized":"2024-04-17 22:29:07.435832","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.3c","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:06.425872","last_change":"2024-04-17 22:29:06.425872","last_active":"2024-04-17 22:29:06.425872","last_peered":"2024-04-17 22:29:06.425872","last_clean":"2024-04-17 22:29:06.425872","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:06.425872","last_undegraded":"2024-04-17 22:29:06.425872","last_fullsized":"2024-04-17 22:29:06.425872","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:07.435832","last_change":"2024-04-17 22:29:07.435832","last_active":"2024-04-17 22:29:07.435832","last_peered":"2024-04-17 22:29:07.435832","last_clean":"2024-04-17 22:29:07.435832","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:07.435832","last_undegraded":"2024-04-17 22:29:07.435832","last_fullsized":"2024-04-17 22:29:07.435832","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.3d","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:06.425872","last_change":"2024-04-17 22:29:06.425872","last_active":"2024-04-17 22:29:06.425872","last_peered":"2024-04-17 22:29:06.425872","last_clean":"2024-04-17 22:29:06.425872","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:06.425872","last_undegraded":"2024-04-17 22:29:06.425872","last_fullsized":"2024-04-17 22:29:06.425872","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:07.435832","last_change":"2024-04-17 22:29:07.435832","last_active":"2024-04-17 22:29:07.435832","last_peered":"2024-04-17 22:29:07.435832","last_clean":"2024-04-17 22:29:07.435832","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:07.435832","last_undegraded":"2024-04-17 22:29:07.435832","last_fullsized":"2024-04-17 22:29:07.435832","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.3e","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:06.425872","last_change":"2024-04-17 22:29:06.425872","last_active":"2024-04-17 22:29:06.425872","last_peered":"2024-04-17 22:29:06.425872","last_clean":"2024-04-17 22:29:06.425872","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:06.425872","last_undegraded":"2024-04-17 22:29:06.425872","last_fullsized":"2024-04-17 22:29:06.425872","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:07.435832","last_change":"2024-04-17 22:29:07.435832","last_active":"2024-04-17 22:29:07.435832","last_peered":"2024-04-17 22:29:07.435832","last_clean":"2024-04-17 22:29:07.435832","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:07.435832","last_undegraded":"2024-04-17 22:29:07.435832","last_fullsized":"2024-04-17 22:29:07.435832","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.3f","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:06.425872","last_change":"2024-04-17 22:29:06.425872","last_active":"2024-04-17 22:29:06.425872","last_peered":"2024-04-17 22:29:06.425872","last_clean":"2024-04-17 22:29:06.425872","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:06.425872","last_undegraded":"2024-04-17 22:29:06.425872","last_fullsized":"2024-04-17 22:29:06.425872","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":4},{"poolid":2,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705668,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"kb":94371840,"kb_used":1048688,"kb_used_data":104,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323152,"statfs":{"total":96636764160,"available":95562907648,"internally_reserved":1073741824,"allocated":106496,"data_stored":24381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.216}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.22500000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.24199999999999999}]}]},{"osd":2,"up_from":9,"seq":38654705668,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"kb":94371840,"kb_used":1048688,"kb_used_data":104,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323152,"statfs":{"total":96636764160,"available":95562907648,"internally_reserved":1073741824,"allocated":106496,"data_stored":24381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.129}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.217}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.20100000000000001}]}]},{"osd":0,"up_from":9,"seq":38654705668,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"kb":94371840,"kb_used":1048688,"kb_used_data":104,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323152,"statfs":{"total":96636764160,"available":95562907648,"internally_reserved":1073741824,"allocated":106496,"data_stored":24381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29399999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.31}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.307}]}]},{"osd":1,"up_from":9,"seq":38654705667,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"kb":94371840,"kb_used":1048688,"kb_used_data":104,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323152,"statfs":{"total":96636764160,"available":95562907648,"internally_reserved":1073741824,"allocated":106496,"data_stored":24381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.21099999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.23499999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.218}]}]}]}} 2024-04-17T22:29:10.663 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-17T22:29:10.825 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.836 7f955564f700 1 Processor -- start 2024-04-17T22:29:10.826 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.836 7f955564f700 1 -- start start 2024-04-17T22:29:10.826 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.837 7f955564f700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f95501a6440 0x7f95501a88a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:10.826 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.837 7f955564f700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f95501a8ed0 0x7f95501ab330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:10.826 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.837 7f955564f700 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f955010ee20 con 0x7f955010e1d0 2024-04-17T22:29:10.826 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.837 7f955564f700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95500f9ea0 con 0x7f95501a6440 2024-04-17T22:29:10.827 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.837 7f955564f700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f95500f8fc0 con 0x7f95501a8ed0 2024-04-17T22:29:10.827 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.837 7f9547fff700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f95501a6440 0x7f95501a88a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:10.827 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.837 7f954e2b1700 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f955010e1d0 0x7f955010e620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:46498/0 (socket says 172.21.15.102:46498) 2024-04-17T22:29:10.827 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.837 7f954e2b1700 1 -- 172.21.15.102:0/2743749611 learned_addr learned my addr 172.21.15.102:0/2743749611 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-17T22:29:10.827 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.837 7f954eab2700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f95501a8ed0 0x7f95501ab330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:10.827 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.837 7f9547fff700 1 -- 172.21.15.102:0/2743749611 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f95501a8ed0 msgr2=0x7f95501ab330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:29:10.827 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.837 7f9547fff700 1 --2- 172.21.15.102:0/2743749611 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f95501a8ed0 0x7f95501ab330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:10.827 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.837 7f9547fff700 1 -- 172.21.15.102:0/2743749611 >> v1:172.21.15.102:6789/0 conn(0x7f955010e1d0 legacy=0x7f955010e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:10.827 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.837 7f954dab0700 1 -- 172.21.15.102:0/2743749611 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 211801803 0 0) 0x7f9540001110 con 0x7f955010e1d0 2024-04-17T22:29:10.827 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.837 7f9547fff700 1 -- 172.21.15.102:0/2743749611 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95501ac970 con 0x7f95501a6440 2024-04-17T22:29:10.827 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.837 7f954eab2700 1 --2- 172.21.15.102:0/2743749611 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f95501a8ed0 0x7f95501ab330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:29:10.827 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.837 7f9547fff700 1 --2- 172.21.15.102:0/2743749611 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f95501a6440 0x7f95501a88a0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7f95500f4400 tx=0x7f953800ba50).ready entity=mon.1 client_cookie=bf1e433d3b2ab982 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:29:10.828 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.838 7f954dab0700 1 -- 172.21.15.102:0/2743749611 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f95501ac970 con 0x7f95501a6440 2024-04-17T22:29:10.828 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.838 7f954dab0700 1 -- 172.21.15.102:0/2743749611 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f953800d570 con 0x7f95501a6440 2024-04-17T22:29:10.828 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.838 7f954dab0700 1 -- 172.21.15.102:0/2743749611 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f953800d940 con 0x7f95501a6440 2024-04-17T22:29:10.828 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.838 7f955564f700 1 -- 172.21.15.102:0/2743749611 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f95501a6440 msgr2=0x7f95501a88a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:10.828 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.838 7f955564f700 1 --2- 172.21.15.102:0/2743749611 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f95501a6440 0x7f95501a88a0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7f95500f4400 tx=0x7f953800ba50).stop 2024-04-17T22:29:10.828 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.838 7f955564f700 1 -- 172.21.15.102:0/2743749611 shutdown_connections 2024-04-17T22:29:10.828 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.838 7f955564f700 1 --2- 172.21.15.102:0/2743749611 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f95501a8ed0 0x7f95501ab330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:10.828 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.838 7f955564f700 1 --2- 172.21.15.102:0/2743749611 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f95501a6440 0x7f95501a88a0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:10.828 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.838 7f955564f700 1 -- 172.21.15.102:0/2743749611 >> 172.21.15.102:0/2743749611 conn(0x7f9550113830 msgr2=0x7f95500f4ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:29:10.828 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.838 7f955564f700 1 -- 172.21.15.102:0/2743749611 shutdown_connections 2024-04-17T22:29:10.828 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.838 7f955564f700 1 -- 172.21.15.102:0/2743749611 wait complete. 2024-04-17T22:29:10.848 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.861 7f955564f700 1 Processor -- start 2024-04-17T22:29:10.848 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.861 7f955564f700 1 -- start start 2024-04-17T22:29:10.850 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.862 7f955564f700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f95500d5ff0 0x7f95500d6440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:10.850 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.862 7f955564f700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f95500d6a70 0x7f95500d7770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:10.851 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.862 7f955564f700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f95500d7da0 0x7f95500d81f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:10.851 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.862 7f955564f700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95500fa140 con 0x7f95500d7da0 2024-04-17T22:29:10.851 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.862 7f955564f700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95500ce930 con 0x7f95500d6a70 2024-04-17T22:29:10.851 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.862 7f955564f700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f95500cdd70 con 0x7f95500d5ff0 2024-04-17T22:29:10.851 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.862 7f954e2b1700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f95500d5ff0 0x7f95500d6440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:10.851 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.862 7f954eab2700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f95500d7da0 0x7f95500d81f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:10.851 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.862 7f954e2b1700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f95500d5ff0 0x7f95500d6440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:41042/0 (socket says 172.21.15.102:41042) 2024-04-17T22:29:10.851 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.862 7f954e2b1700 1 -- 172.21.15.102:0/662541991 learned_addr learned my addr 172.21.15.102:0/662541991 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:29:10.851 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.862 7f9547fff700 1 --2- 172.21.15.102:0/662541991 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f95500d6a70 0x7f95500d7770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:10.851 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.862 7f954e2b1700 1 -- 172.21.15.102:0/662541991 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f95500d6a70 msgr2=0x7f95500d7770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:10.851 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.862 7f954e2b1700 1 --2- 172.21.15.102:0/662541991 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f95500d6a70 0x7f95500d7770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:10.851 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.862 7f954e2b1700 1 -- 172.21.15.102:0/662541991 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f95500d7da0 msgr2=0x7f95500d81f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:10.851 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.862 7f954e2b1700 1 --2- 172.21.15.102:0/662541991 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f95500d7da0 0x7f95500d81f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:10.852 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.862 7f954e2b1700 1 -- 172.21.15.102:0/662541991 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95500cd7e0 con 0x7f95500d5ff0 2024-04-17T22:29:10.852 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.862 7f954eab2700 1 --2- 172.21.15.102:0/662541991 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f95500d7da0 0x7f95500d81f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:29:10.852 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.862 7f954e2b1700 1 --2- 172.21.15.102:0/662541991 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f95500d5ff0 0x7f95500d6440 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7f95500c4bb0 tx=0x7f954000c810).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:29:10.852 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.862 7f95477fe700 1 -- 172.21.15.102:0/662541991 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f95500cd7e0 con 0x7f95500d5ff0 2024-04-17T22:29:10.852 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.862 7f95477fe700 1 -- 172.21.15.102:0/662541991 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95400072e0 con 0x7f95500d5ff0 2024-04-17T22:29:10.852 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.862 7f955564f700 1 -- 172.21.15.102:0/662541991 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95500ce580 con 0x7f95500d5ff0 2024-04-17T22:29:10.852 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.862 7f95477fe700 1 -- 172.21.15.102:0/662541991 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9540005810 con 0x7f95500d5ff0 2024-04-17T22:29:10.852 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.862 7f955564f700 1 -- 172.21.15.102:0/662541991 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95501bb910 con 0x7f95500d5ff0 2024-04-17T22:29:10.852 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.863 7f95477fe700 1 -- 172.21.15.102:0/662541991 <== mon.2 v2:172.21.15.102:3302/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7f9540005a10 con 0x7f95500d5ff0 2024-04-17T22:29:10.852 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.863 7f95477fe700 1 -- 172.21.15.102:0/662541991 <== mon.2 v2:172.21.15.102:3302/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 3194+0+0 (secure 0 0 0) 0x7f9540014a10 con 0x7f95500d5ff0 2024-04-17T22:29:10.852 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.864 7f955564f700 1 -- 172.21.15.102:0/662541991 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95500f0790 con 0x7f95500d5ff0 2024-04-17T22:29:10.853 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.867 7f95477fe700 1 -- 172.21.15.102:0/662541991 <== mon.2 v2:172.21.15.102:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+110455 (secure 0 0 0) 0x7f95400070d0 con 0x7f95500d5ff0 2024-04-17T22:29:10.955 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.968 7f955564f700 1 -- 172.21.15.102:0/662541991 --> v1:172.21.15.102:6816/42360 -- command(tid 0: {"prefix": "pg dump", "target": ["mgr", ""], "format": "json"}) v1 -- 0x7f955008a300 con 0x7f95340145a0 2024-04-17T22:29:10.962 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.975 7f95477fe700 1 -- 172.21.15.102:0/662541991 <== mgr.4104 v1:172.21.15.102:6816/42360 1 ==== command_reply(tid 0: 0 dumped all) v1 ==== 18+0+299651 (unknown 2965378022 0 3595174762) 0x7f9538006950 con 0x7f95340145a0 2024-04-17T22:29:10.966 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-17T22:29:10.968 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.977 7f955564f700 1 -- 172.21.15.102:0/662541991 >> v1:172.21.15.102:6816/42360 conn(0x7f95340145a0 legacy=0x7f9534016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:10.968 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.977 7f955564f700 1 -- 172.21.15.102:0/662541991 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f95500d5ff0 msgr2=0x7f95500d6440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:10.969 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.977 7f955564f700 1 --2- 172.21.15.102:0/662541991 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f95500d5ff0 0x7f95500d6440 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7f95500c4bb0 tx=0x7f954000c810).stop 2024-04-17T22:29:10.969 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.977 7f955564f700 1 -- 172.21.15.102:0/662541991 shutdown_connections 2024-04-17T22:29:10.969 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.977 7f955564f700 1 --2- 172.21.15.102:0/662541991 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f95500d7da0 0x7f95500d81f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:10.969 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.977 7f955564f700 1 --2- 172.21.15.102:0/662541991 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f95500d5ff0 0x7f95500d6440 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:10.969 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.977 7f955564f700 1 --2- 172.21.15.102:0/662541991 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f95500d6a70 0x7f95500d7770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:10.969 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.977 7f955564f700 1 -- 172.21.15.102:0/662541991 >> 172.21.15.102:0/662541991 conn(0x7f95500c56d0 msgr2=0x7f95500cf4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:29:10.969 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.977 7f955564f700 1 -- 172.21.15.102:0/662541991 shutdown_connections 2024-04-17T22:29:10.969 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:10.978 7f955564f700 1 -- 172.21.15.102:0/662541991 wait complete. 2024-04-17T22:29:10.969 INFO:teuthology.orchestra.run.smithi102.stderr:dumped all 2024-04-17T22:29:10.975 INFO:teuthology.orchestra.run.smithi102.stdout:{"pg_ready":true,"pg_map":{"version":20,"stamp":"2024-04-17 22:29:09.187086","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":4,"num_per_pool_osds":4,"kb":377487360,"kb_used":4194752,"kb_used_data":416,"kb_used_omap":0,"kb_used_meta":4194304,"kb_avail":373292608,"statfs":{"total":386547056640,"available":382251630592,"internally_reserved":4294967296,"allocated":425984,"data_stored":97524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4294967296},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"3.1a","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:07.435832","last_change":"2024-04-17 22:29:07.435832","last_active":"2024-04-17 22:29:07.435832","last_peered":"2024-04-17 22:29:07.435832","last_clean":"2024-04-17 22:29:07.435832","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:07.435832","last_undegraded":"2024-04-17 22:29:07.435832","last_fullsized":"2024-04-17 22:29:07.435832","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:06.425872","last_change":"2024-04-17 22:29:06.425872","last_active":"2024-04-17 22:29:06.425872","last_peered":"2024-04-17 22:29:06.425872","last_clean":"2024-04-17 22:29:06.425872","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:06.425872","last_undegraded":"2024-04-17 22:29:06.425872","last_fullsized":"2024-04-17 22:29:06.425872","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:07.435832","last_change":"2024-04-17 22:29:07.435832","last_active":"2024-04-17 22:29:07.435832","last_peered":"2024-04-17 22:29:07.435832","last_clean":"2024-04-17 22:29:07.435832","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:07.435832","last_undegraded":"2024-04-17 22:29:07.435832","last_fullsized":"2024-04-17 22:29:07.435832","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:06.425872","last_change":"2024-04-17 22:29:06.425872","last_active":"2024-04-17 22:29:06.425872","last_peered":"2024-04-17 22:29:06.425872","last_clean":"2024-04-17 22:29:06.425872","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:06.425872","last_undegraded":"2024-04-17 22:29:06.425872","last_fullsized":"2024-04-17 22:29:06.425872","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:07.435832","last_change":"2024-04-17 22:29:07.435832","last_active":"2024-04-17 22:29:07.435832","last_peered":"2024-04-17 22:29:07.435832","last_clean":"2024-04-17 22:29:07.435832","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:07.435832","last_undegraded":"2024-04-17 22:29:07.435832","last_fullsized":"2024-04-17 22:29:07.435832","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:06.425872","last_change":"2024-04-17 22:29:06.425872","last_active":"2024-04-17 22:29:06.425872","last_peered":"2024-04-17 22:29:06.425872","last_clean":"2024-04-17 22:29:06.425872","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:06.425872","last_undegraded":"2024-04-17 22:29:06.425872","last_fullsized":"2024-04-17 22:29:06.425872","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:07.435832","last_change":"2024-04-17 22:29:07.435832","last_active":"2024-04-17 22:29:07.435832","last_peered":"2024-04-17 22:29:07.435832","last_clean":"2024-04-17 22:29:07.435832","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:07.435832","last_undegraded":"2024-04-17 22:29:07.435832","last_fullsized":"2024-04-17 22:29:07.435832","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:06.425872","last_change":"2024-04-17 22:29:06.425872","last_active":"2024-04-17 22:29:06.425872","last_peered":"2024-04-17 22:29:06.425872","last_clean":"2024-04-17 22:29:06.425872","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:06.425872","last_undegraded":"2024-04-17 22:29:06.425872","last_fullsized":"2024-04-17 22:29:06.425872","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:07.435832","last_change":"2024-04-17 22:29:07.435832","last_active":"2024-04-17 22:29:07.435832","last_peered":"2024-04-17 22:29:07.435832","last_clean":"2024-04-17 22:29:07.435832","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:07.435832","last_undegraded":"2024-04-17 22:29:07.435832","last_fullsized":"2024-04-17 22:29:07.435832","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:06.425872","last_change":"2024-04-17 22:29:06.425872","last_active":"2024-04-17 22:29:06.425872","last_peered":"2024-04-17 22:29:06.425872","last_clean":"2024-04-17 22:29:06.425872","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:06.425872","last_undegraded":"2024-04-17 22:29:06.425872","last_fullsized":"2024-04-17 22:29:06.425872","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:07.435832","last_change":"2024-04-17 22:29:07.435832","last_active":"2024-04-17 22:29:07.435832","last_peered":"2024-04-17 22:29:07.435832","last_clean":"2024-04-17 22:29:07.435832","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:07.435832","last_undegraded":"2024-04-17 22:29:07.435832","last_fullsized":"2024-04-17 22:29:07.435832","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:06.425872","last_change":"2024-04-17 22:29:06.425872","last_active":"2024-04-17 22:29:06.425872","last_peered":"2024-04-17 22:29:06.425872","last_clean":"2024-04-17 22:29:06.425872","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:06.425872","last_undegraded":"2024-04-17 22:29:06.425872","last_fullsized":"2024-04-17 22:29:06.425872","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:07.435832","last_change":"2024-04-17 22:29:07.435832","last_active":"2024-04-17 22:29:07.435832","last_peered":"2024-04-17 22:29:07.435832","last_clean":"2024-04-17 22:29:07.435832","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:07.435832","last_undegraded":"2024-04-17 22:29:07.435832","last_fullsized":"2024-04-17 22:29:07.435832","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:06.425872","last_change":"2024-04-17 22:29:06.425872","last_active":"2024-04-17 22:29:06.425872","last_peered":"2024-04-17 22:29:06.425872","last_clean":"2024-04-17 22:29:06.425872","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:06.425872","last_undegraded":"2024-04-17 22:29:06.425872","last_fullsized":"2024-04-17 22:29:06.425872","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:07.435832","last_change":"2024-04-17 22:29:07.435832","last_active":"2024-04-17 22:29:07.435832","last_peered":"2024-04-17 22:29:07.435832","last_clean":"2024-04-17 22:29:07.435832","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:07.435832","last_undegraded":"2024-04-17 22:29:07.435832","last_fullsized":"2024-04-17 22:29:07.435832","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:06.425872","last_change":"2024-04-17 22:29:06.425872","last_active":"2024-04-17 22:29:06.425872","last_peered":"2024-04-17 22:29:06.425872","last_clean":"2024-04-17 22:29:06.425872","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:06.425872","last_undegraded":"2024-04-17 22:29:06.425872","last_fullsized":"2024-04-17 22:29:06.425872","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:07.435832","last_change":"2024-04-17 22:29:07.435832","last_active":"2024-04-17 22:29:07.435832","last_peered":"2024-04-17 22:29:07.435832","last_clean":"2024-04-17 22:29:07.435832","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:07.435832","last_undegraded":"2024-04-17 22:29:07.435832","last_fullsized":"2024-04-17 22:29:07.435832","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:06.425872","last_change":"2024-04-17 22:29:06.425872","last_active":"2024-04-17 22:29:06.425872","last_peered":"2024-04-17 22:29:06.425872","last_clean":"2024-04-17 22:29:06.425872","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:06.425872","last_undegraded":"2024-04-17 22:29:06.425872","last_fullsized":"2024-04-17 22:29:06.425872","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:07.435832","last_change":"2024-04-17 22:29:07.435832","last_active":"2024-04-17 22:29:07.435832","last_peered":"2024-04-17 22:29:07.435832","last_clean":"2024-04-17 22:29:07.435832","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:07.435832","last_undegraded":"2024-04-17 22:29:07.435832","last_fullsized":"2024-04-17 22:29:07.435832","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:06.425872","last_change":"2024-04-17 22:29:06.425872","last_active":"2024-04-17 22:29:06.425872","last_peered":"2024-04-17 22:29:06.425872","last_clean":"2024-04-17 22:29:06.425872","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:06.425872","last_undegraded":"2024-04-17 22:29:06.425872","last_fullsized":"2024-04-17 22:29:06.425872","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:07.435832","last_change":"2024-04-17 22:29:07.435832","last_active":"2024-04-17 22:29:07.435832","last_peered":"2024-04-17 22:29:07.435832","last_clean":"2024-04-17 22:29:07.435832","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:07.435832","last_undegraded":"2024-04-17 22:29:07.435832","last_fullsized":"2024-04-17 22:29:07.435832","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:06.425872","last_change":"2024-04-17 22:29:06.425872","last_active":"2024-04-17 22:29:06.425872","last_peered":"2024-04-17 22:29:06.425872","last_clean":"2024-04-17 22:29:06.425872","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:06.425872","last_undegraded":"2024-04-17 22:29:06.425872","last_fullsized":"2024-04-17 22:29:06.425872","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:07.435832","last_change":"2024-04-17 22:29:07.435832","last_active":"2024-04-17 22:29:07.435832","last_peered":"2024-04-17 22:29:07.435832","last_clean":"2024-04-17 22:29:07.435832","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:07.435832","last_undegraded":"2024-04-17 22:29:07.435832","last_fullsized":"2024-04-17 22:29:07.435832","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:06.425872","last_change":"2024-04-17 22:29:06.425872","last_active":"2024-04-17 22:29:06.425872","last_peered":"2024-04-17 22:29:06.425872","last_clean":"2024-04-17 22:29:06.425872","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:06.425872","last_undegraded":"2024-04-17 22:29:06.425872","last_fullsized":"2024-04-17 22:29:06.425872","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:07.435832","last_change":"2024-04-17 22:29:07.435832","last_active":"2024-04-17 22:29:07.435832","last_peered":"2024-04-17 22:29:07.435832","last_clean":"2024-04-17 22:29:07.435832","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:07.435832","last_undegraded":"2024-04-17 22:29:07.435832","last_fullsized":"2024-04-17 22:29:07.435832","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:06.425872","last_change":"2024-04-17 22:29:06.425872","last_active":"2024-04-17 22:29:06.425872","last_peered":"2024-04-17 22:29:06.425872","last_clean":"2024-04-17 22:29:06.425872","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:06.425872","last_undegraded":"2024-04-17 22:29:06.425872","last_fullsized":"2024-04-17 22:29:06.425872","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:07.435832","last_change":"2024-04-17 22:29:07.435832","last_active":"2024-04-17 22:29:07.435832","last_peered":"2024-04-17 22:29:07.435832","last_clean":"2024-04-17 22:29:07.435832","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:07.435832","last_undegraded":"2024-04-17 22:29:07.435832","last_fullsized":"2024-04-17 22:29:07.435832","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:06.425872","last_change":"2024-04-17 22:29:06.425872","last_active":"2024-04-17 22:29:06.425872","last_peered":"2024-04-17 22:29:06.425872","last_clean":"2024-04-17 22:29:06.425872","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:06.425872","last_undegraded":"2024-04-17 22:29:06.425872","last_fullsized":"2024-04-17 22:29:06.425872","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:07.435832","last_change":"2024-04-17 22:29:07.435832","last_active":"2024-04-17 22:29:07.435832","last_peered":"2024-04-17 22:29:07.435832","last_clean":"2024-04-17 22:29:07.435832","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:07.435832","last_undegraded":"2024-04-17 22:29:07.435832","last_fullsized":"2024-04-17 22:29:07.435832","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:06.425872","last_change":"2024-04-17 22:29:06.425872","last_active":"2024-04-17 22:29:06.425872","last_peered":"2024-04-17 22:29:06.425872","last_clean":"2024-04-17 22:29:06.425872","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:06.425872","last_undegraded":"2024-04-17 22:29:06.425872","last_fullsized":"2024-04-17 22:29:06.425872","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:07.435832","last_change":"2024-04-17 22:29:07.435832","last_active":"2024-04-17 22:29:07.435832","last_peered":"2024-04-17 22:29:07.435832","last_clean":"2024-04-17 22:29:07.435832","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:07.435832","last_undegraded":"2024-04-17 22:29:07.435832","last_fullsized":"2024-04-17 22:29:07.435832","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:06.425872","last_change":"2024-04-17 22:29:06.425872","last_active":"2024-04-17 22:29:06.425872","last_peered":"2024-04-17 22:29:06.425872","last_clean":"2024-04-17 22:29:06.425872","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:06.425872","last_undegraded":"2024-04-17 22:29:06.425872","last_fullsized":"2024-04-17 22:29:06.425872","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:07.435832","last_change":"2024-04-17 22:29:07.435832","last_active":"2024-04-17 22:29:07.435832","last_peered":"2024-04-17 22:29:07.435832","last_clean":"2024-04-17 22:29:07.435832","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:07.435832","last_undegraded":"2024-04-17 22:29:07.435832","last_fullsized":"2024-04-17 22:29:07.435832","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:06.425872","last_change":"2024-04-17 22:29:06.425872","last_active":"2024-04-17 22:29:06.425872","last_peered":"2024-04-17 22:29:06.425872","last_clean":"2024-04-17 22:29:06.425872","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:06.425872","last_undegraded":"2024-04-17 22:29:06.425872","last_fullsized":"2024-04-17 22:29:06.425872","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:07.435832","last_change":"2024-04-17 22:29:07.435832","last_active":"2024-04-17 22:29:07.435832","last_peered":"2024-04-17 22:29:07.435832","last_clean":"2024-04-17 22:29:07.435832","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:07.435832","last_undegraded":"2024-04-17 22:29:07.435832","last_fullsized":"2024-04-17 22:29:07.435832","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:06.425872","last_change":"2024-04-17 22:29:06.425872","last_active":"2024-04-17 22:29:06.425872","last_peered":"2024-04-17 22:29:06.425872","last_clean":"2024-04-17 22:29:06.425872","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:06.425872","last_undegraded":"2024-04-17 22:29:06.425872","last_fullsized":"2024-04-17 22:29:06.425872","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:07.435832","last_change":"2024-04-17 22:29:07.435832","last_active":"2024-04-17 22:29:07.435832","last_peered":"2024-04-17 22:29:07.435832","last_clean":"2024-04-17 22:29:07.435832","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:07.435832","last_undegraded":"2024-04-17 22:29:07.435832","last_fullsized":"2024-04-17 22:29:07.435832","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:06.425872","last_change":"2024-04-17 22:29:06.425872","last_active":"2024-04-17 22:29:06.425872","last_peered":"2024-04-17 22:29:06.425872","last_clean":"2024-04-17 22:29:06.425872","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:06.425872","last_undegraded":"2024-04-17 22:29:06.425872","last_fullsized":"2024-04-17 22:29:06.425872","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:07.435832","last_change":"2024-04-17 22:29:07.435832","last_active":"2024-04-17 22:29:07.435832","last_peered":"2024-04-17 22:29:07.435832","last_clean":"2024-04-17 22:29:07.435832","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:07.435832","last_undegraded":"2024-04-17 22:29:07.435832","last_fullsized":"2024-04-17 22:29:07.435832","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:06.425872","last_change":"2024-04-17 22:29:06.425872","last_active":"2024-04-17 22:29:06.425872","last_peered":"2024-04-17 22:29:06.425872","last_clean":"2024-04-17 22:29:06.425872","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:06.425872","last_undegraded":"2024-04-17 22:29:06.425872","last_fullsized":"2024-04-17 22:29:06.425872","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:07.435832","last_change":"2024-04-17 22:29:07.435832","last_active":"2024-04-17 22:29:07.435832","last_peered":"2024-04-17 22:29:07.435832","last_clean":"2024-04-17 22:29:07.435832","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:07.435832","last_undegraded":"2024-04-17 22:29:07.435832","last_fullsized":"2024-04-17 22:29:07.435832","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":"9","reported_epoch":"11","state":"active+clean","last_fresh":"2024-04-17 22:28:59.256589","last_change":"2024-04-17 22:28:59.253977","last_active":"2024-04-17 22:28:59.256589","last_peered":"2024-04-17 22:28:59.256589","last_clean":"2024-04-17 22:28:59.256589","last_became_active":"2024-04-17 22:28:59.253641","last_became_peered":"2024-04-17 22:28:59.253641","last_unstale":"2024-04-17 22:28:59.256589","last_undegraded":"2024-04-17 22:28:59.256589","last_fullsized":"2024-04-17 22:28:59.256589","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:28:58.239187","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:28:58.239187","last_clean_scrub_stamp":"2024-04-17 22:28:58.239187","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:06.425872","last_change":"2024-04-17 22:29:06.425872","last_active":"2024-04-17 22:29:06.425872","last_peered":"2024-04-17 22:29:06.425872","last_clean":"2024-04-17 22:29:06.425872","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:06.425872","last_undegraded":"2024-04-17 22:29:06.425872","last_fullsized":"2024-04-17 22:29:06.425872","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:07.435832","last_change":"2024-04-17 22:29:07.435832","last_active":"2024-04-17 22:29:07.435832","last_peered":"2024-04-17 22:29:07.435832","last_clean":"2024-04-17 22:29:07.435832","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:07.435832","last_undegraded":"2024-04-17 22:29:07.435832","last_fullsized":"2024-04-17 22:29:07.435832","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":"9","reported_epoch":"11","state":"active+clean","last_fresh":"2024-04-17 22:28:59.257288","last_change":"2024-04-17 22:28:59.255733","last_active":"2024-04-17 22:28:59.257288","last_peered":"2024-04-17 22:28:59.257288","last_clean":"2024-04-17 22:28:59.257288","last_became_active":"2024-04-17 22:28:59.255411","last_became_peered":"2024-04-17 22:28:59.255411","last_unstale":"2024-04-17 22:28:59.257288","last_undegraded":"2024-04-17 22:28:59.257288","last_fullsized":"2024-04-17 22:28:59.257288","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:28:58.239187","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:28:58.239187","last_clean_scrub_stamp":"2024-04-17 22:28:58.239187","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:06.425872","last_change":"2024-04-17 22:29:06.425872","last_active":"2024-04-17 22:29:06.425872","last_peered":"2024-04-17 22:29:06.425872","last_clean":"2024-04-17 22:29:06.425872","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:06.425872","last_undegraded":"2024-04-17 22:29:06.425872","last_fullsized":"2024-04-17 22:29:06.425872","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":"9","reported_epoch":"11","state":"active+clean","last_fresh":"2024-04-17 22:28:59.254418","last_change":"2024-04-17 22:28:59.252728","last_active":"2024-04-17 22:28:59.254418","last_peered":"2024-04-17 22:28:59.254418","last_clean":"2024-04-17 22:28:59.254418","last_became_active":"2024-04-17 22:28:59.252345","last_became_peered":"2024-04-17 22:28:59.252345","last_unstale":"2024-04-17 22:28:59.254418","last_undegraded":"2024-04-17 22:28:59.254418","last_fullsized":"2024-04-17 22:28:59.254418","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:28:58.239187","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:28:58.239187","last_clean_scrub_stamp":"2024-04-17 22:28:58.239187","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:06.425872","last_change":"2024-04-17 22:29:06.425872","last_active":"2024-04-17 22:29:06.425872","last_peered":"2024-04-17 22:29:06.425872","last_clean":"2024-04-17 22:29:06.425872","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:06.425872","last_undegraded":"2024-04-17 22:29:06.425872","last_fullsized":"2024-04-17 22:29:06.425872","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:07.435832","last_change":"2024-04-17 22:29:07.435832","last_active":"2024-04-17 22:29:07.435832","last_peered":"2024-04-17 22:29:07.435832","last_clean":"2024-04-17 22:29:07.435832","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:07.435832","last_undegraded":"2024-04-17 22:29:07.435832","last_fullsized":"2024-04-17 22:29:07.435832","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":"9","reported_epoch":"11","state":"active+clean","last_fresh":"2024-04-17 22:28:59.254870","last_change":"2024-04-17 22:28:59.253392","last_active":"2024-04-17 22:28:59.254870","last_peered":"2024-04-17 22:28:59.254870","last_clean":"2024-04-17 22:28:59.254870","last_became_active":"2024-04-17 22:28:59.253049","last_became_peered":"2024-04-17 22:28:59.253049","last_unstale":"2024-04-17 22:28:59.254870","last_undegraded":"2024-04-17 22:28:59.254870","last_fullsized":"2024-04-17 22:28:59.254870","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:28:58.239187","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:28:58.239187","last_clean_scrub_stamp":"2024-04-17 22:28:58.239187","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:06.425872","last_change":"2024-04-17 22:29:06.425872","last_active":"2024-04-17 22:29:06.425872","last_peered":"2024-04-17 22:29:06.425872","last_clean":"2024-04-17 22:29:06.425872","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:06.425872","last_undegraded":"2024-04-17 22:29:06.425872","last_fullsized":"2024-04-17 22:29:06.425872","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:07.435832","last_change":"2024-04-17 22:29:07.435832","last_active":"2024-04-17 22:29:07.435832","last_peered":"2024-04-17 22:29:07.435832","last_clean":"2024-04-17 22:29:07.435832","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:07.435832","last_undegraded":"2024-04-17 22:29:07.435832","last_fullsized":"2024-04-17 22:29:07.435832","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":"9","reported_epoch":"11","state":"active+clean","last_fresh":"2024-04-17 22:28:59.255890","last_change":"2024-04-17 22:28:59.254907","last_active":"2024-04-17 22:28:59.255890","last_peered":"2024-04-17 22:28:59.255890","last_clean":"2024-04-17 22:28:59.255890","last_became_active":"2024-04-17 22:28:59.254633","last_became_peered":"2024-04-17 22:28:59.254633","last_unstale":"2024-04-17 22:28:59.255890","last_undegraded":"2024-04-17 22:28:59.255890","last_fullsized":"2024-04-17 22:28:59.255890","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:28:58.239187","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:28:58.239187","last_clean_scrub_stamp":"2024-04-17 22:28:58.239187","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:06.425872","last_change":"2024-04-17 22:29:06.425872","last_active":"2024-04-17 22:29:06.425872","last_peered":"2024-04-17 22:29:06.425872","last_clean":"2024-04-17 22:29:06.425872","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:06.425872","last_undegraded":"2024-04-17 22:29:06.425872","last_fullsized":"2024-04-17 22:29:06.425872","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:07.435832","last_change":"2024-04-17 22:29:07.435832","last_active":"2024-04-17 22:29:07.435832","last_peered":"2024-04-17 22:29:07.435832","last_clean":"2024-04-17 22:29:07.435832","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:07.435832","last_undegraded":"2024-04-17 22:29:07.435832","last_fullsized":"2024-04-17 22:29:07.435832","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":"9","reported_epoch":"11","state":"active+clean","last_fresh":"2024-04-17 22:28:59.255467","last_change":"2024-04-17 22:28:59.254217","last_active":"2024-04-17 22:28:59.255467","last_peered":"2024-04-17 22:28:59.255467","last_clean":"2024-04-17 22:28:59.255467","last_became_active":"2024-04-17 22:28:59.253877","last_became_peered":"2024-04-17 22:28:59.253877","last_unstale":"2024-04-17 22:28:59.255467","last_undegraded":"2024-04-17 22:28:59.255467","last_fullsized":"2024-04-17 22:28:59.255467","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:28:58.239187","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:28:58.239187","last_clean_scrub_stamp":"2024-04-17 22:28:58.239187","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:06.425872","last_change":"2024-04-17 22:29:06.425872","last_active":"2024-04-17 22:29:06.425872","last_peered":"2024-04-17 22:29:06.425872","last_clean":"2024-04-17 22:29:06.425872","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:06.425872","last_undegraded":"2024-04-17 22:29:06.425872","last_fullsized":"2024-04-17 22:29:06.425872","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:07.435832","last_change":"2024-04-17 22:29:07.435832","last_active":"2024-04-17 22:29:07.435832","last_peered":"2024-04-17 22:29:07.435832","last_clean":"2024-04-17 22:29:07.435832","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:07.435832","last_undegraded":"2024-04-17 22:29:07.435832","last_fullsized":"2024-04-17 22:29:07.435832","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":"9","reported_epoch":"11","state":"active+clean","last_fresh":"2024-04-17 22:28:59.255985","last_change":"2024-04-17 22:28:59.254830","last_active":"2024-04-17 22:28:59.255985","last_peered":"2024-04-17 22:28:59.255985","last_clean":"2024-04-17 22:28:59.255985","last_became_active":"2024-04-17 22:28:59.254505","last_became_peered":"2024-04-17 22:28:59.254505","last_unstale":"2024-04-17 22:28:59.255985","last_undegraded":"2024-04-17 22:28:59.255985","last_fullsized":"2024-04-17 22:28:59.255985","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:28:58.239187","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:28:58.239187","last_clean_scrub_stamp":"2024-04-17 22:28:58.239187","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:06.425872","last_change":"2024-04-17 22:29:06.425872","last_active":"2024-04-17 22:29:06.425872","last_peered":"2024-04-17 22:29:06.425872","last_clean":"2024-04-17 22:29:06.425872","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:06.425872","last_undegraded":"2024-04-17 22:29:06.425872","last_fullsized":"2024-04-17 22:29:06.425872","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:07.435832","last_change":"2024-04-17 22:29:07.435832","last_active":"2024-04-17 22:29:07.435832","last_peered":"2024-04-17 22:29:07.435832","last_clean":"2024-04-17 22:29:07.435832","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:07.435832","last_undegraded":"2024-04-17 22:29:07.435832","last_fullsized":"2024-04-17 22:29:07.435832","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":"9","reported_epoch":"11","state":"active+clean","last_fresh":"2024-04-17 22:28:59.257265","last_change":"2024-04-17 22:28:59.254055","last_active":"2024-04-17 22:28:59.257265","last_peered":"2024-04-17 22:28:59.257265","last_clean":"2024-04-17 22:28:59.257265","last_became_active":"2024-04-17 22:28:59.253750","last_became_peered":"2024-04-17 22:28:59.253750","last_unstale":"2024-04-17 22:28:59.257265","last_undegraded":"2024-04-17 22:28:59.257265","last_fullsized":"2024-04-17 22:28:59.257265","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:28:58.239187","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:28:58.239187","last_clean_scrub_stamp":"2024-04-17 22:28:58.239187","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:06.425872","last_change":"2024-04-17 22:29:06.425872","last_active":"2024-04-17 22:29:06.425872","last_peered":"2024-04-17 22:29:06.425872","last_clean":"2024-04-17 22:29:06.425872","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:06.425872","last_undegraded":"2024-04-17 22:29:06.425872","last_fullsized":"2024-04-17 22:29:06.425872","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:07.435832","last_change":"2024-04-17 22:29:07.435832","last_active":"2024-04-17 22:29:07.435832","last_peered":"2024-04-17 22:29:07.435832","last_clean":"2024-04-17 22:29:07.435832","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:07.435832","last_undegraded":"2024-04-17 22:29:07.435832","last_fullsized":"2024-04-17 22:29:07.435832","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:07.435832","last_change":"2024-04-17 22:29:07.435832","last_active":"2024-04-17 22:29:07.435832","last_peered":"2024-04-17 22:29:07.435832","last_clean":"2024-04-17 22:29:07.435832","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:07.435832","last_undegraded":"2024-04-17 22:29:07.435832","last_fullsized":"2024-04-17 22:29:07.435832","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:06.425872","last_change":"2024-04-17 22:29:06.425872","last_active":"2024-04-17 22:29:06.425872","last_peered":"2024-04-17 22:29:06.425872","last_clean":"2024-04-17 22:29:06.425872","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:06.425872","last_undegraded":"2024-04-17 22:29:06.425872","last_fullsized":"2024-04-17 22:29:06.425872","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:07.435832","last_change":"2024-04-17 22:29:07.435832","last_active":"2024-04-17 22:29:07.435832","last_peered":"2024-04-17 22:29:07.435832","last_clean":"2024-04-17 22:29:07.435832","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:07.435832","last_undegraded":"2024-04-17 22:29:07.435832","last_fullsized":"2024-04-17 22:29:07.435832","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:06.425872","last_change":"2024-04-17 22:29:06.425872","last_active":"2024-04-17 22:29:06.425872","last_peered":"2024-04-17 22:29:06.425872","last_clean":"2024-04-17 22:29:06.425872","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:06.425872","last_undegraded":"2024-04-17 22:29:06.425872","last_fullsized":"2024-04-17 22:29:06.425872","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:07.435832","last_change":"2024-04-17 22:29:07.435832","last_active":"2024-04-17 22:29:07.435832","last_peered":"2024-04-17 22:29:07.435832","last_clean":"2024-04-17 22:29:07.435832","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:07.435832","last_undegraded":"2024-04-17 22:29:07.435832","last_fullsized":"2024-04-17 22:29:07.435832","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:06.425872","last_change":"2024-04-17 22:29:06.425872","last_active":"2024-04-17 22:29:06.425872","last_peered":"2024-04-17 22:29:06.425872","last_clean":"2024-04-17 22:29:06.425872","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:06.425872","last_undegraded":"2024-04-17 22:29:06.425872","last_fullsized":"2024-04-17 22:29:06.425872","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:07.435832","last_change":"2024-04-17 22:29:07.435832","last_active":"2024-04-17 22:29:07.435832","last_peered":"2024-04-17 22:29:07.435832","last_clean":"2024-04-17 22:29:07.435832","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:07.435832","last_undegraded":"2024-04-17 22:29:07.435832","last_fullsized":"2024-04-17 22:29:07.435832","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:06.425872","last_change":"2024-04-17 22:29:06.425872","last_active":"2024-04-17 22:29:06.425872","last_peered":"2024-04-17 22:29:06.425872","last_clean":"2024-04-17 22:29:06.425872","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:06.425872","last_undegraded":"2024-04-17 22:29:06.425872","last_fullsized":"2024-04-17 22:29:06.425872","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:07.435832","last_change":"2024-04-17 22:29:07.435832","last_active":"2024-04-17 22:29:07.435832","last_peered":"2024-04-17 22:29:07.435832","last_clean":"2024-04-17 22:29:07.435832","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:07.435832","last_undegraded":"2024-04-17 22:29:07.435832","last_fullsized":"2024-04-17 22:29:07.435832","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.20","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:06.425872","last_change":"2024-04-17 22:29:06.425872","last_active":"2024-04-17 22:29:06.425872","last_peered":"2024-04-17 22:29:06.425872","last_clean":"2024-04-17 22:29:06.425872","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:06.425872","last_undegraded":"2024-04-17 22:29:06.425872","last_fullsized":"2024-04-17 22:29:06.425872","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:07.435832","last_change":"2024-04-17 22:29:07.435832","last_active":"2024-04-17 22:29:07.435832","last_peered":"2024-04-17 22:29:07.435832","last_clean":"2024-04-17 22:29:07.435832","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:07.435832","last_undegraded":"2024-04-17 22:29:07.435832","last_fullsized":"2024-04-17 22:29:07.435832","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.21","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:06.425872","last_change":"2024-04-17 22:29:06.425872","last_active":"2024-04-17 22:29:06.425872","last_peered":"2024-04-17 22:29:06.425872","last_clean":"2024-04-17 22:29:06.425872","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:06.425872","last_undegraded":"2024-04-17 22:29:06.425872","last_fullsized":"2024-04-17 22:29:06.425872","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:07.435832","last_change":"2024-04-17 22:29:07.435832","last_active":"2024-04-17 22:29:07.435832","last_peered":"2024-04-17 22:29:07.435832","last_clean":"2024-04-17 22:29:07.435832","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:07.435832","last_undegraded":"2024-04-17 22:29:07.435832","last_fullsized":"2024-04-17 22:29:07.435832","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.22","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:06.425872","last_change":"2024-04-17 22:29:06.425872","last_active":"2024-04-17 22:29:06.425872","last_peered":"2024-04-17 22:29:06.425872","last_clean":"2024-04-17 22:29:06.425872","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:06.425872","last_undegraded":"2024-04-17 22:29:06.425872","last_fullsized":"2024-04-17 22:29:06.425872","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:07.435832","last_change":"2024-04-17 22:29:07.435832","last_active":"2024-04-17 22:29:07.435832","last_peered":"2024-04-17 22:29:07.435832","last_clean":"2024-04-17 22:29:07.435832","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:07.435832","last_undegraded":"2024-04-17 22:29:07.435832","last_fullsized":"2024-04-17 22:29:07.435832","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.23","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:06.425872","last_change":"2024-04-17 22:29:06.425872","last_active":"2024-04-17 22:29:06.425872","last_peered":"2024-04-17 22:29:06.425872","last_clean":"2024-04-17 22:29:06.425872","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:06.425872","last_undegraded":"2024-04-17 22:29:06.425872","last_fullsized":"2024-04-17 22:29:06.425872","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:07.435832","last_change":"2024-04-17 22:29:07.435832","last_active":"2024-04-17 22:29:07.435832","last_peered":"2024-04-17 22:29:07.435832","last_clean":"2024-04-17 22:29:07.435832","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:07.435832","last_undegraded":"2024-04-17 22:29:07.435832","last_fullsized":"2024-04-17 22:29:07.435832","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.24","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:06.425872","last_change":"2024-04-17 22:29:06.425872","last_active":"2024-04-17 22:29:06.425872","last_peered":"2024-04-17 22:29:06.425872","last_clean":"2024-04-17 22:29:06.425872","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:06.425872","last_undegraded":"2024-04-17 22:29:06.425872","last_fullsized":"2024-04-17 22:29:06.425872","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:07.435832","last_change":"2024-04-17 22:29:07.435832","last_active":"2024-04-17 22:29:07.435832","last_peered":"2024-04-17 22:29:07.435832","last_clean":"2024-04-17 22:29:07.435832","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:07.435832","last_undegraded":"2024-04-17 22:29:07.435832","last_fullsized":"2024-04-17 22:29:07.435832","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.25","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:06.425872","last_change":"2024-04-17 22:29:06.425872","last_active":"2024-04-17 22:29:06.425872","last_peered":"2024-04-17 22:29:06.425872","last_clean":"2024-04-17 22:29:06.425872","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:06.425872","last_undegraded":"2024-04-17 22:29:06.425872","last_fullsized":"2024-04-17 22:29:06.425872","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:07.435832","last_change":"2024-04-17 22:29:07.435832","last_active":"2024-04-17 22:29:07.435832","last_peered":"2024-04-17 22:29:07.435832","last_clean":"2024-04-17 22:29:07.435832","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:07.435832","last_undegraded":"2024-04-17 22:29:07.435832","last_fullsized":"2024-04-17 22:29:07.435832","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.26","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:06.425872","last_change":"2024-04-17 22:29:06.425872","last_active":"2024-04-17 22:29:06.425872","last_peered":"2024-04-17 22:29:06.425872","last_clean":"2024-04-17 22:29:06.425872","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:06.425872","last_undegraded":"2024-04-17 22:29:06.425872","last_fullsized":"2024-04-17 22:29:06.425872","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:07.435832","last_change":"2024-04-17 22:29:07.435832","last_active":"2024-04-17 22:29:07.435832","last_peered":"2024-04-17 22:29:07.435832","last_clean":"2024-04-17 22:29:07.435832","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:07.435832","last_undegraded":"2024-04-17 22:29:07.435832","last_fullsized":"2024-04-17 22:29:07.435832","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.27","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:06.425872","last_change":"2024-04-17 22:29:06.425872","last_active":"2024-04-17 22:29:06.425872","last_peered":"2024-04-17 22:29:06.425872","last_clean":"2024-04-17 22:29:06.425872","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:06.425872","last_undegraded":"2024-04-17 22:29:06.425872","last_fullsized":"2024-04-17 22:29:06.425872","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:07.435832","last_change":"2024-04-17 22:29:07.435832","last_active":"2024-04-17 22:29:07.435832","last_peered":"2024-04-17 22:29:07.435832","last_clean":"2024-04-17 22:29:07.435832","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:07.435832","last_undegraded":"2024-04-17 22:29:07.435832","last_fullsized":"2024-04-17 22:29:07.435832","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.28","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:06.425872","last_change":"2024-04-17 22:29:06.425872","last_active":"2024-04-17 22:29:06.425872","last_peered":"2024-04-17 22:29:06.425872","last_clean":"2024-04-17 22:29:06.425872","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:06.425872","last_undegraded":"2024-04-17 22:29:06.425872","last_fullsized":"2024-04-17 22:29:06.425872","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:07.435832","last_change":"2024-04-17 22:29:07.435832","last_active":"2024-04-17 22:29:07.435832","last_peered":"2024-04-17 22:29:07.435832","last_clean":"2024-04-17 22:29:07.435832","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:07.435832","last_undegraded":"2024-04-17 22:29:07.435832","last_fullsized":"2024-04-17 22:29:07.435832","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.29","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:06.425872","last_change":"2024-04-17 22:29:06.425872","last_active":"2024-04-17 22:29:06.425872","last_peered":"2024-04-17 22:29:06.425872","last_clean":"2024-04-17 22:29:06.425872","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:06.425872","last_undegraded":"2024-04-17 22:29:06.425872","last_fullsized":"2024-04-17 22:29:06.425872","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:07.435832","last_change":"2024-04-17 22:29:07.435832","last_active":"2024-04-17 22:29:07.435832","last_peered":"2024-04-17 22:29:07.435832","last_clean":"2024-04-17 22:29:07.435832","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:07.435832","last_undegraded":"2024-04-17 22:29:07.435832","last_fullsized":"2024-04-17 22:29:07.435832","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2a","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:06.425872","last_change":"2024-04-17 22:29:06.425872","last_active":"2024-04-17 22:29:06.425872","last_peered":"2024-04-17 22:29:06.425872","last_clean":"2024-04-17 22:29:06.425872","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:06.425872","last_undegraded":"2024-04-17 22:29:06.425872","last_fullsized":"2024-04-17 22:29:06.425872","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:07.435832","last_change":"2024-04-17 22:29:07.435832","last_active":"2024-04-17 22:29:07.435832","last_peered":"2024-04-17 22:29:07.435832","last_clean":"2024-04-17 22:29:07.435832","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:07.435832","last_undegraded":"2024-04-17 22:29:07.435832","last_fullsized":"2024-04-17 22:29:07.435832","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2b","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:06.425872","last_change":"2024-04-17 22:29:06.425872","last_active":"2024-04-17 22:29:06.425872","last_peered":"2024-04-17 22:29:06.425872","last_clean":"2024-04-17 22:29:06.425872","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:06.425872","last_undegraded":"2024-04-17 22:29:06.425872","last_fullsized":"2024-04-17 22:29:06.425872","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:07.435832","last_change":"2024-04-17 22:29:07.435832","last_active":"2024-04-17 22:29:07.435832","last_peered":"2024-04-17 22:29:07.435832","last_clean":"2024-04-17 22:29:07.435832","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:07.435832","last_undegraded":"2024-04-17 22:29:07.435832","last_fullsized":"2024-04-17 22:29:07.435832","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2c","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:06.425872","last_change":"2024-04-17 22:29:06.425872","last_active":"2024-04-17 22:29:06.425872","last_peered":"2024-04-17 22:29:06.425872","last_clean":"2024-04-17 22:29:06.425872","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:06.425872","last_undegraded":"2024-04-17 22:29:06.425872","last_fullsized":"2024-04-17 22:29:06.425872","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:07.435832","last_change":"2024-04-17 22:29:07.435832","last_active":"2024-04-17 22:29:07.435832","last_peered":"2024-04-17 22:29:07.435832","last_clean":"2024-04-17 22:29:07.435832","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:07.435832","last_undegraded":"2024-04-17 22:29:07.435832","last_fullsized":"2024-04-17 22:29:07.435832","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2d","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:06.425872","last_change":"2024-04-17 22:29:06.425872","last_active":"2024-04-17 22:29:06.425872","last_peered":"2024-04-17 22:29:06.425872","last_clean":"2024-04-17 22:29:06.425872","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:06.425872","last_undegraded":"2024-04-17 22:29:06.425872","last_fullsized":"2024-04-17 22:29:06.425872","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:07.435832","last_change":"2024-04-17 22:29:07.435832","last_active":"2024-04-17 22:29:07.435832","last_peered":"2024-04-17 22:29:07.435832","last_clean":"2024-04-17 22:29:07.435832","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:07.435832","last_undegraded":"2024-04-17 22:29:07.435832","last_fullsized":"2024-04-17 22:29:07.435832","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2e","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:06.425872","last_change":"2024-04-17 22:29:06.425872","last_active":"2024-04-17 22:29:06.425872","last_peered":"2024-04-17 22:29:06.425872","last_clean":"2024-04-17 22:29:06.425872","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:06.425872","last_undegraded":"2024-04-17 22:29:06.425872","last_fullsized":"2024-04-17 22:29:06.425872","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:07.435832","last_change":"2024-04-17 22:29:07.435832","last_active":"2024-04-17 22:29:07.435832","last_peered":"2024-04-17 22:29:07.435832","last_clean":"2024-04-17 22:29:07.435832","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:07.435832","last_undegraded":"2024-04-17 22:29:07.435832","last_fullsized":"2024-04-17 22:29:07.435832","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2f","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:06.425872","last_change":"2024-04-17 22:29:06.425872","last_active":"2024-04-17 22:29:06.425872","last_peered":"2024-04-17 22:29:06.425872","last_clean":"2024-04-17 22:29:06.425872","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:06.425872","last_undegraded":"2024-04-17 22:29:06.425872","last_fullsized":"2024-04-17 22:29:06.425872","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:07.435832","last_change":"2024-04-17 22:29:07.435832","last_active":"2024-04-17 22:29:07.435832","last_peered":"2024-04-17 22:29:07.435832","last_clean":"2024-04-17 22:29:07.435832","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:07.435832","last_undegraded":"2024-04-17 22:29:07.435832","last_fullsized":"2024-04-17 22:29:07.435832","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.30","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:06.425872","last_change":"2024-04-17 22:29:06.425872","last_active":"2024-04-17 22:29:06.425872","last_peered":"2024-04-17 22:29:06.425872","last_clean":"2024-04-17 22:29:06.425872","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:06.425872","last_undegraded":"2024-04-17 22:29:06.425872","last_fullsized":"2024-04-17 22:29:06.425872","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:07.435832","last_change":"2024-04-17 22:29:07.435832","last_active":"2024-04-17 22:29:07.435832","last_peered":"2024-04-17 22:29:07.435832","last_clean":"2024-04-17 22:29:07.435832","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:07.435832","last_undegraded":"2024-04-17 22:29:07.435832","last_fullsized":"2024-04-17 22:29:07.435832","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.31","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:06.425872","last_change":"2024-04-17 22:29:06.425872","last_active":"2024-04-17 22:29:06.425872","last_peered":"2024-04-17 22:29:06.425872","last_clean":"2024-04-17 22:29:06.425872","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:06.425872","last_undegraded":"2024-04-17 22:29:06.425872","last_fullsized":"2024-04-17 22:29:06.425872","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:07.435832","last_change":"2024-04-17 22:29:07.435832","last_active":"2024-04-17 22:29:07.435832","last_peered":"2024-04-17 22:29:07.435832","last_clean":"2024-04-17 22:29:07.435832","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:07.435832","last_undegraded":"2024-04-17 22:29:07.435832","last_fullsized":"2024-04-17 22:29:07.435832","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.32","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:06.425872","last_change":"2024-04-17 22:29:06.425872","last_active":"2024-04-17 22:29:06.425872","last_peered":"2024-04-17 22:29:06.425872","last_clean":"2024-04-17 22:29:06.425872","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:06.425872","last_undegraded":"2024-04-17 22:29:06.425872","last_fullsized":"2024-04-17 22:29:06.425872","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:07.435832","last_change":"2024-04-17 22:29:07.435832","last_active":"2024-04-17 22:29:07.435832","last_peered":"2024-04-17 22:29:07.435832","last_clean":"2024-04-17 22:29:07.435832","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:07.435832","last_undegraded":"2024-04-17 22:29:07.435832","last_fullsized":"2024-04-17 22:29:07.435832","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.33","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:06.425872","last_change":"2024-04-17 22:29:06.425872","last_active":"2024-04-17 22:29:06.425872","last_peered":"2024-04-17 22:29:06.425872","last_clean":"2024-04-17 22:29:06.425872","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:06.425872","last_undegraded":"2024-04-17 22:29:06.425872","last_fullsized":"2024-04-17 22:29:06.425872","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:07.435832","last_change":"2024-04-17 22:29:07.435832","last_active":"2024-04-17 22:29:07.435832","last_peered":"2024-04-17 22:29:07.435832","last_clean":"2024-04-17 22:29:07.435832","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:07.435832","last_undegraded":"2024-04-17 22:29:07.435832","last_fullsized":"2024-04-17 22:29:07.435832","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.34","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:06.425872","last_change":"2024-04-17 22:29:06.425872","last_active":"2024-04-17 22:29:06.425872","last_peered":"2024-04-17 22:29:06.425872","last_clean":"2024-04-17 22:29:06.425872","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:06.425872","last_undegraded":"2024-04-17 22:29:06.425872","last_fullsized":"2024-04-17 22:29:06.425872","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:07.435832","last_change":"2024-04-17 22:29:07.435832","last_active":"2024-04-17 22:29:07.435832","last_peered":"2024-04-17 22:29:07.435832","last_clean":"2024-04-17 22:29:07.435832","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:07.435832","last_undegraded":"2024-04-17 22:29:07.435832","last_fullsized":"2024-04-17 22:29:07.435832","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.35","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:06.425872","last_change":"2024-04-17 22:29:06.425872","last_active":"2024-04-17 22:29:06.425872","last_peered":"2024-04-17 22:29:06.425872","last_clean":"2024-04-17 22:29:06.425872","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:06.425872","last_undegraded":"2024-04-17 22:29:06.425872","last_fullsized":"2024-04-17 22:29:06.425872","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:07.435832","last_change":"2024-04-17 22:29:07.435832","last_active":"2024-04-17 22:29:07.435832","last_peered":"2024-04-17 22:29:07.435832","last_clean":"2024-04-17 22:29:07.435832","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:07.435832","last_undegraded":"2024-04-17 22:29:07.435832","last_fullsized":"2024-04-17 22:29:07.435832","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.36","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:06.425872","last_change":"2024-04-17 22:29:06.425872","last_active":"2024-04-17 22:29:06.425872","last_peered":"2024-04-17 22:29:06.425872","last_clean":"2024-04-17 22:29:06.425872","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:06.425872","last_undegraded":"2024-04-17 22:29:06.425872","last_fullsized":"2024-04-17 22:29:06.425872","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:07.435832","last_change":"2024-04-17 22:29:07.435832","last_active":"2024-04-17 22:29:07.435832","last_peered":"2024-04-17 22:29:07.435832","last_clean":"2024-04-17 22:29:07.435832","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:07.435832","last_undegraded":"2024-04-17 22:29:07.435832","last_fullsized":"2024-04-17 22:29:07.435832","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.37","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:06.425872","last_change":"2024-04-17 22:29:06.425872","last_active":"2024-04-17 22:29:06.425872","last_peered":"2024-04-17 22:29:06.425872","last_clean":"2024-04-17 22:29:06.425872","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:06.425872","last_undegraded":"2024-04-17 22:29:06.425872","last_fullsized":"2024-04-17 22:29:06.425872","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:07.435832","last_change":"2024-04-17 22:29:07.435832","last_active":"2024-04-17 22:29:07.435832","last_peered":"2024-04-17 22:29:07.435832","last_clean":"2024-04-17 22:29:07.435832","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:07.435832","last_undegraded":"2024-04-17 22:29:07.435832","last_fullsized":"2024-04-17 22:29:07.435832","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.38","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:06.425872","last_change":"2024-04-17 22:29:06.425872","last_active":"2024-04-17 22:29:06.425872","last_peered":"2024-04-17 22:29:06.425872","last_clean":"2024-04-17 22:29:06.425872","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:06.425872","last_undegraded":"2024-04-17 22:29:06.425872","last_fullsized":"2024-04-17 22:29:06.425872","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:07.435832","last_change":"2024-04-17 22:29:07.435832","last_active":"2024-04-17 22:29:07.435832","last_peered":"2024-04-17 22:29:07.435832","last_clean":"2024-04-17 22:29:07.435832","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:07.435832","last_undegraded":"2024-04-17 22:29:07.435832","last_fullsized":"2024-04-17 22:29:07.435832","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.39","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:06.425872","last_change":"2024-04-17 22:29:06.425872","last_active":"2024-04-17 22:29:06.425872","last_peered":"2024-04-17 22:29:06.425872","last_clean":"2024-04-17 22:29:06.425872","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:06.425872","last_undegraded":"2024-04-17 22:29:06.425872","last_fullsized":"2024-04-17 22:29:06.425872","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:07.435832","last_change":"2024-04-17 22:29:07.435832","last_active":"2024-04-17 22:29:07.435832","last_peered":"2024-04-17 22:29:07.435832","last_clean":"2024-04-17 22:29:07.435832","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:07.435832","last_undegraded":"2024-04-17 22:29:07.435832","last_fullsized":"2024-04-17 22:29:07.435832","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.3a","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:06.425872","last_change":"2024-04-17 22:29:06.425872","last_active":"2024-04-17 22:29:06.425872","last_peered":"2024-04-17 22:29:06.425872","last_clean":"2024-04-17 22:29:06.425872","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:06.425872","last_undegraded":"2024-04-17 22:29:06.425872","last_fullsized":"2024-04-17 22:29:06.425872","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:07.435832","last_change":"2024-04-17 22:29:07.435832","last_active":"2024-04-17 22:29:07.435832","last_peered":"2024-04-17 22:29:07.435832","last_clean":"2024-04-17 22:29:07.435832","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:07.435832","last_undegraded":"2024-04-17 22:29:07.435832","last_fullsized":"2024-04-17 22:29:07.435832","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.3b","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:06.425872","last_change":"2024-04-17 22:29:06.425872","last_active":"2024-04-17 22:29:06.425872","last_peered":"2024-04-17 22:29:06.425872","last_clean":"2024-04-17 22:29:06.425872","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:06.425872","last_undegraded":"2024-04-17 22:29:06.425872","last_fullsized":"2024-04-17 22:29:06.425872","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:07.435832","last_change":"2024-04-17 22:29:07.435832","last_active":"2024-04-17 22:29:07.435832","last_peered":"2024-04-17 22:29:07.435832","last_clean":"2024-04-17 22:29:07.435832","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:07.435832","last_undegraded":"2024-04-17 22:29:07.435832","last_fullsized":"2024-04-17 22:29:07.435832","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.3c","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:06.425872","last_change":"2024-04-17 22:29:06.425872","last_active":"2024-04-17 22:29:06.425872","last_peered":"2024-04-17 22:29:06.425872","last_clean":"2024-04-17 22:29:06.425872","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:06.425872","last_undegraded":"2024-04-17 22:29:06.425872","last_fullsized":"2024-04-17 22:29:06.425872","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:07.435832","last_change":"2024-04-17 22:29:07.435832","last_active":"2024-04-17 22:29:07.435832","last_peered":"2024-04-17 22:29:07.435832","last_clean":"2024-04-17 22:29:07.435832","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:07.435832","last_undegraded":"2024-04-17 22:29:07.435832","last_fullsized":"2024-04-17 22:29:07.435832","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.3d","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:06.425872","last_change":"2024-04-17 22:29:06.425872","last_active":"2024-04-17 22:29:06.425872","last_peered":"2024-04-17 22:29:06.425872","last_clean":"2024-04-17 22:29:06.425872","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:06.425872","last_undegraded":"2024-04-17 22:29:06.425872","last_fullsized":"2024-04-17 22:29:06.425872","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:07.435832","last_change":"2024-04-17 22:29:07.435832","last_active":"2024-04-17 22:29:07.435832","last_peered":"2024-04-17 22:29:07.435832","last_clean":"2024-04-17 22:29:07.435832","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:07.435832","last_undegraded":"2024-04-17 22:29:07.435832","last_fullsized":"2024-04-17 22:29:07.435832","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.3e","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:06.425872","last_change":"2024-04-17 22:29:06.425872","last_active":"2024-04-17 22:29:06.425872","last_peered":"2024-04-17 22:29:06.425872","last_clean":"2024-04-17 22:29:06.425872","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:06.425872","last_undegraded":"2024-04-17 22:29:06.425872","last_fullsized":"2024-04-17 22:29:06.425872","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:07.435832","last_change":"2024-04-17 22:29:07.435832","last_active":"2024-04-17 22:29:07.435832","last_peered":"2024-04-17 22:29:07.435832","last_clean":"2024-04-17 22:29:07.435832","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:07.435832","last_undegraded":"2024-04-17 22:29:07.435832","last_fullsized":"2024-04-17 22:29:07.435832","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.3f","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:29:06.425872","last_change":"2024-04-17 22:29:06.425872","last_active":"2024-04-17 22:29:06.425872","last_peered":"2024-04-17 22:29:06.425872","last_clean":"2024-04-17 22:29:06.425872","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:29:06.425872","last_undegraded":"2024-04-17 22:29:06.425872","last_fullsized":"2024-04-17 22:29:06.425872","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":4},{"poolid":2,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705668,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"kb":94371840,"kb_used":1048688,"kb_used_data":104,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323152,"statfs":{"total":96636764160,"available":95562907648,"internally_reserved":1073741824,"allocated":106496,"data_stored":24381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.216}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.22500000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.24199999999999999}]}]},{"osd":2,"up_from":9,"seq":38654705668,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"kb":94371840,"kb_used":1048688,"kb_used_data":104,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323152,"statfs":{"total":96636764160,"available":95562907648,"internally_reserved":1073741824,"allocated":106496,"data_stored":24381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.129}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.217}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.20100000000000001}]}]},{"osd":0,"up_from":9,"seq":38654705668,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"kb":94371840,"kb_used":1048688,"kb_used_data":104,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323152,"statfs":{"total":96636764160,"available":95562907648,"internally_reserved":1073741824,"allocated":106496,"data_stored":24381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29399999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.31}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.307}]}]},{"osd":1,"up_from":9,"seq":38654705667,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"kb":94371840,"kb_used":1048688,"kb_used_data":104,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323152,"statfs":{"total":96636764160,"available":95562907648,"internally_reserved":1073741824,"allocated":106496,"data_stored":24381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.21099999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.23499999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.218}]}]}]}} 2024-04-17T22:29:10.980 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-17T22:29:11.142 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.153 7f9af7ef5700 1 Processor -- start 2024-04-17T22:29:11.142 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.153 7f9af7ef5700 1 -- start start 2024-04-17T22:29:11.142 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.153 7f9af7ef5700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f9af00f5770 0x7f9af010c110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:11.142 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.153 7f9af7ef5700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f9af010cb90 0x7f9af01a1400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:11.142 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.153 7f9af7ef5700 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9af01a1a30 con 0x7f9af010c740 2024-04-17T22:29:11.143 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.153 7f9af7ef5700 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9af00f93e0 con 0x7f9af010cb90 2024-04-17T22:29:11.143 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.153 7f9af7ef5700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9af01a1c50 con 0x7f9af00f5770 2024-04-17T22:29:11.143 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.153 7f9af4cf0700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f9af00f5770 0x7f9af010c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:11.143 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.153 7f9af54f1700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f9af010cb90 0x7f9af01a1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:11.143 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.153 7f9af4cf0700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f9af00f5770 0x7f9af010c110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:41048/0 (socket says 172.21.15.102:41048) 2024-04-17T22:29:11.143 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.153 7f9af54f1700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f9af010cb90 0x7f9af01a1400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:49306/0 (socket says 172.21.15.102:49306) 2024-04-17T22:29:11.143 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.154 7f9af4cf0700 1 -- 172.21.15.102:0/3150769446 learned_addr learned my addr 172.21.15.102:0/3150769446 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:29:11.143 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.153 7f9aeffff700 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f9af010c740 0x7f9af01a0c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:46504/0 (socket says 172.21.15.102:46504) 2024-04-17T22:29:11.143 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.154 7f9af54f1700 1 -- 172.21.15.102:0/3150769446 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f9af00f5770 msgr2=0x7f9af010c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:29:11.143 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.154 7f9af54f1700 1 --2- 172.21.15.102:0/3150769446 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f9af00f5770 0x7f9af010c110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:11.143 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.154 7f9af54f1700 1 -- 172.21.15.102:0/3150769446 >> v1:172.21.15.102:6789/0 conn(0x7f9af010c740 legacy=0x7f9af01a0c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:11.143 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.154 7f9af54f1700 1 -- 172.21.15.102:0/3150769446 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9af01aa330 con 0x7f9af010cb90 2024-04-17T22:29:11.143 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.154 7f9af54f1700 1 --2- 172.21.15.102:0/3150769446 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9af010cb90 0x7f9af01a1400 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0x7f9af00f9590 tx=0x7f9ae8009d40).ready entity=mon.0 client_cookie=ff1374e0b07388d server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:29:11.143 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.154 7f9aef7fe700 1 -- 172.21.15.102:0/3150769446 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9af01aa330 con 0x7f9af010cb90 2024-04-17T22:29:11.144 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.154 7f9aef7fe700 1 -- 172.21.15.102:0/3150769446 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ae800b7c0 con 0x7f9af010cb90 2024-04-17T22:29:11.144 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.154 7f9aef7fe700 1 -- 172.21.15.102:0/3150769446 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9ae800b9f0 con 0x7f9af010cb90 2024-04-17T22:29:11.144 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.154 7f9af7ef5700 1 -- 172.21.15.102:0/3150769446 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9af010cb90 msgr2=0x7f9af01a1400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:11.144 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.154 7f9af7ef5700 1 --2- 172.21.15.102:0/3150769446 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9af010cb90 0x7f9af01a1400 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0x7f9af00f9590 tx=0x7f9ae8009d40).stop 2024-04-17T22:29:11.144 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.155 7f9af7ef5700 1 -- 172.21.15.102:0/3150769446 shutdown_connections 2024-04-17T22:29:11.144 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.155 7f9af7ef5700 1 --2- 172.21.15.102:0/3150769446 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f9af00f5770 0x7f9af010c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:11.144 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.155 7f9af7ef5700 1 --2- 172.21.15.102:0/3150769446 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9af010cb90 0x7f9af01a1400 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:11.144 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.155 7f9af7ef5700 1 -- 172.21.15.102:0/3150769446 >> 172.21.15.102:0/3150769446 conn(0x7f9af01138c0 msgr2=0x7f9af00f4f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:29:11.144 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.155 7f9af7ef5700 1 -- 172.21.15.102:0/3150769446 shutdown_connections 2024-04-17T22:29:11.144 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.155 7f9af7ef5700 1 -- 172.21.15.102:0/3150769446 wait complete. 2024-04-17T22:29:11.164 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.177 7f9af7ef5700 1 Processor -- start 2024-04-17T22:29:11.164 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.177 7f9af7ef5700 1 -- start start 2024-04-17T22:29:11.166 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.178 7f9af7ef5700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f9af00d5ff0 0x7f9af00d6440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:11.167 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.178 7f9af7ef5700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f9af00d6a70 0x7f9af01b1d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:11.167 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.178 7f9af7ef5700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9af01b2370 0x7f9af01b27c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:11.167 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.178 7f9af7ef5700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9af00cac30 con 0x7f9af01b2370 2024-04-17T22:29:11.167 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.178 7f9af7ef5700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9af01b2df0 con 0x7f9af00d5ff0 2024-04-17T22:29:11.167 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.178 7f9af7ef5700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9af00ca4c0 con 0x7f9af00d6a70 2024-04-17T22:29:11.167 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.178 7f9af54f1700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9af01b2370 0x7f9af01b27c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:11.167 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.178 7f9af4cf0700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f9af00d5ff0 0x7f9af00d6440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:11.167 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.178 7f9aeffff700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f9af00d6a70 0x7f9af01b1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:11.167 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.178 7f9af54f1700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9af01b2370 0x7f9af01b27c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:49312/0 (socket says 172.21.15.102:49312) 2024-04-17T22:29:11.167 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.178 7f9af54f1700 1 -- 172.21.15.102:0/2352434770 learned_addr learned my addr 172.21.15.102:0/2352434770 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:29:11.167 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.178 7f9af4cf0700 1 -- 172.21.15.102:0/2352434770 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f9af00d6a70 msgr2=0x7f9af01b1d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:11.167 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.178 7f9af4cf0700 1 --2- 172.21.15.102:0/2352434770 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f9af00d6a70 0x7f9af01b1d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:11.168 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.178 7f9af4cf0700 1 -- 172.21.15.102:0/2352434770 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9af01b2370 msgr2=0x7f9af01b27c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:11.168 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.178 7f9af4cf0700 1 --2- 172.21.15.102:0/2352434770 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9af01b2370 0x7f9af01b27c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:11.168 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.178 7f9af4cf0700 1 -- 172.21.15.102:0/2352434770 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9af00cb200 con 0x7f9af00d5ff0 2024-04-17T22:29:11.168 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.178 7f9aeffff700 1 --2- 172.21.15.102:0/2352434770 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f9af00d6a70 0x7f9af01b1d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:29:11.168 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.178 7f9af54f1700 1 --2- 172.21.15.102:0/2352434770 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9af01b2370 0x7f9af01b27c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:29:11.168 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.178 7f9af4cf0700 1 --2- 172.21.15.102:0/2352434770 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f9af00d5ff0 0x7f9af00d6440 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7f9af00d75e0 tx=0x7f9ae400eab0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:29:11.168 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.179 7f9aedffb700 1 -- 172.21.15.102:0/2352434770 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9af00cb200 con 0x7f9af00d5ff0 2024-04-17T22:29:11.168 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.179 7f9aedffb700 1 -- 172.21.15.102:0/2352434770 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ae4012600 con 0x7f9af00d5ff0 2024-04-17T22:29:11.168 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.179 7f9af7ef5700 1 -- 172.21.15.102:0/2352434770 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9af01b94b0 con 0x7f9af00d5ff0 2024-04-17T22:29:11.168 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.179 7f9aedffb700 1 -- 172.21.15.102:0/2352434770 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9ae400f040 con 0x7f9af00d5ff0 2024-04-17T22:29:11.168 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.179 7f9af7ef5700 1 -- 172.21.15.102:0/2352434770 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9af01b9890 con 0x7f9af00d5ff0 2024-04-17T22:29:11.168 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.179 7f9aedffb700 1 -- 172.21.15.102:0/2352434770 <== mon.1 v2:172.21.15.102:3301/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7f9ae4022060 con 0x7f9af00d5ff0 2024-04-17T22:29:11.168 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.179 7f9aedffb700 1 -- 172.21.15.102:0/2352434770 <== mon.1 v2:172.21.15.102:3301/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 3194+0+0 (secure 0 0 0) 0x7f9ae4010040 con 0x7f9af00d5ff0 2024-04-17T22:29:11.169 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.180 7f9af7ef5700 1 -- 172.21.15.102:0/2352434770 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9af00f0820 con 0x7f9af00d5ff0 2024-04-17T22:29:11.169 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.183 7f9aedffb700 1 -- 172.21.15.102:0/2352434770 <== mon.1 v2:172.21.15.102:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+110455 (secure 0 0 0) 0x7f9ae40123f0 con 0x7f9af00d5ff0 2024-04-17T22:29:11.270 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.284 7f9af7ef5700 1 -- 172.21.15.102:0/2352434770 --> v1:172.21.15.102:6816/42360 -- command(tid 0: {"prefix": "pg dump", "target": ["mgr", ""], "format": "json"}) v1 -- 0x7f9af008a300 con 0x7f9ad80145a0 2024-04-17T22:29:11.278 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.291 7f9aedffb700 1 -- 172.21.15.102:0/2352434770 <== mgr.4104 v1:172.21.15.102:6816/42360 1 ==== command_reply(tid 0: 0 dumped all) v1 ==== 18+0+306111 (unknown 2965378022 0 992435385) 0x7f9adc001620 con 0x7f9ad80145a0 2024-04-17T22:29:11.282 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-17T22:29:11.284 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.294 7f9af7ef5700 1 -- 172.21.15.102:0/2352434770 >> v1:172.21.15.102:6816/42360 conn(0x7f9ad80145a0 legacy=0x7f9ad8016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:11.285 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.294 7f9af7ef5700 1 -- 172.21.15.102:0/2352434770 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f9af00d5ff0 msgr2=0x7f9af00d6440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:11.285 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.294 7f9af7ef5700 1 --2- 172.21.15.102:0/2352434770 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f9af00d5ff0 0x7f9af00d6440 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7f9af00d75e0 tx=0x7f9ae400eab0).stop 2024-04-17T22:29:11.285 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.294 7f9af7ef5700 1 -- 172.21.15.102:0/2352434770 shutdown_connections 2024-04-17T22:29:11.285 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.294 7f9af7ef5700 1 --2- 172.21.15.102:0/2352434770 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9af01b2370 0x7f9af01b27c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:11.285 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.294 7f9af7ef5700 1 --2- 172.21.15.102:0/2352434770 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f9af00d5ff0 0x7f9af00d6440 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:11.285 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.294 7f9af7ef5700 1 --2- 172.21.15.102:0/2352434770 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f9af00d6a70 0x7f9af01b1d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:11.285 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.294 7f9af7ef5700 1 -- 172.21.15.102:0/2352434770 >> 172.21.15.102:0/2352434770 conn(0x7f9af00c56b0 msgr2=0x7f9af00cf4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:29:11.285 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.294 7f9af7ef5700 1 -- 172.21.15.102:0/2352434770 shutdown_connections 2024-04-17T22:29:11.285 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.294 7f9af7ef5700 1 -- 172.21.15.102:0/2352434770 wait complete. 2024-04-17T22:29:11.285 INFO:teuthology.orchestra.run.smithi102.stderr:dumped all 2024-04-17T22:29:11.291 INFO:teuthology.orchestra.run.smithi102.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2024-04-17 22:29:11.187563","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2286,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":272,"acting":272,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":152,"num_osds":4,"num_per_pool_osds":4,"kb":377487360,"kb_used":4194944,"kb_used_data":608,"kb_used_omap":0,"kb_used_meta":4194304,"kb_avail":373292416,"statfs":{"total":386547056640,"available":382251433984,"internally_reserved":4294967296,"allocated":622592,"data_stored":161692,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4294967296},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"3.1a","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.820762","last_change":"2024-04-17 22:29:07.814049","last_active":"2024-04-17 22:29:07.820762","last_peered":"2024-04-17 22:29:07.820762","last_clean":"2024-04-17 22:29:07.820762","last_became_active":"2024-04-17 22:29:07.811049","last_became_peered":"2024-04-17 22:29:07.811049","last_unstale":"2024-04-17 22:29:07.820762","last_undegraded":"2024-04-17 22:29:07.820762","last_fullsized":"2024-04-17 22:29:07.820762","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.807568","last_change":"2024-04-17 22:29:07.477053","last_active":"2024-04-17 22:29:07.807568","last_peered":"2024-04-17 22:29:07.807568","last_clean":"2024-04-17 22:29:07.807568","last_became_active":"2024-04-17 22:29:07.476568","last_became_peered":"2024-04-17 22:29:07.476568","last_unstale":"2024-04-17 22:29:07.807568","last_undegraded":"2024-04-17 22:29:07.807568","last_fullsized":"2024-04-17 22:29:07.807568","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.828628","last_change":"2024-04-17 22:29:07.818787","last_active":"2024-04-17 22:29:07.828628","last_peered":"2024-04-17 22:29:07.828628","last_clean":"2024-04-17 22:29:07.828628","last_became_active":"2024-04-17 22:29:07.818347","last_became_peered":"2024-04-17 22:29:07.818347","last_unstale":"2024-04-17 22:29:07.828628","last_undegraded":"2024-04-17 22:29:07.828628","last_fullsized":"2024-04-17 22:29:07.828628","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.805356","last_change":"2024-04-17 22:29:07.477933","last_active":"2024-04-17 22:29:07.805356","last_peered":"2024-04-17 22:29:07.805356","last_clean":"2024-04-17 22:29:07.805356","last_became_active":"2024-04-17 22:29:07.472128","last_became_peered":"2024-04-17 22:29:07.472128","last_unstale":"2024-04-17 22:29:07.805356","last_undegraded":"2024-04-17 22:29:07.805356","last_fullsized":"2024-04-17 22:29:07.805356","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.834368","last_change":"2024-04-17 22:29:07.833107","last_active":"2024-04-17 22:29:07.834368","last_peered":"2024-04-17 22:29:07.834368","last_clean":"2024-04-17 22:29:07.834368","last_became_active":"2024-04-17 22:29:07.832822","last_became_peered":"2024-04-17 22:29:07.832822","last_unstale":"2024-04-17 22:29:07.834368","last_undegraded":"2024-04-17 22:29:07.834368","last_fullsized":"2024-04-17 22:29:07.834368","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.802997","last_change":"2024-04-17 22:29:07.471612","last_active":"2024-04-17 22:29:07.802997","last_peered":"2024-04-17 22:29:07.802997","last_clean":"2024-04-17 22:29:07.802997","last_became_active":"2024-04-17 22:29:07.471322","last_became_peered":"2024-04-17 22:29:07.471322","last_unstale":"2024-04-17 22:29:07.802997","last_undegraded":"2024-04-17 22:29:07.802997","last_fullsized":"2024-04-17 22:29:07.802997","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.812548","last_change":"2024-04-17 22:29:07.810081","last_active":"2024-04-17 22:29:07.812548","last_peered":"2024-04-17 22:29:07.812548","last_clean":"2024-04-17 22:29:07.812548","last_became_active":"2024-04-17 22:29:07.809858","last_became_peered":"2024-04-17 22:29:07.809858","last_unstale":"2024-04-17 22:29:07.812548","last_undegraded":"2024-04-17 22:29:07.812548","last_fullsized":"2024-04-17 22:29:07.812548","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.797655","last_change":"2024-04-17 22:29:07.474346","last_active":"2024-04-17 22:29:07.797655","last_peered":"2024-04-17 22:29:07.797655","last_clean":"2024-04-17 22:29:07.797655","last_became_active":"2024-04-17 22:29:07.473783","last_became_peered":"2024-04-17 22:29:07.473783","last_unstale":"2024-04-17 22:29:07.797655","last_undegraded":"2024-04-17 22:29:07.797655","last_fullsized":"2024-04-17 22:29:07.797655","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.825049","last_change":"2024-04-17 22:29:07.811047","last_active":"2024-04-17 22:29:07.825049","last_peered":"2024-04-17 22:29:07.825049","last_clean":"2024-04-17 22:29:07.825049","last_became_active":"2024-04-17 22:29:07.810747","last_became_peered":"2024-04-17 22:29:07.810747","last_unstale":"2024-04-17 22:29:07.825049","last_undegraded":"2024-04-17 22:29:07.825049","last_fullsized":"2024-04-17 22:29:07.825049","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.798589","last_change":"2024-04-17 22:29:07.460228","last_active":"2024-04-17 22:29:07.798589","last_peered":"2024-04-17 22:29:07.798589","last_clean":"2024-04-17 22:29:07.798589","last_became_active":"2024-04-17 22:29:07.459910","last_became_peered":"2024-04-17 22:29:07.459910","last_unstale":"2024-04-17 22:29:07.798589","last_undegraded":"2024-04-17 22:29:07.798589","last_fullsized":"2024-04-17 22:29:07.798589","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.830592","last_change":"2024-04-17 22:29:07.827143","last_active":"2024-04-17 22:29:07.830592","last_peered":"2024-04-17 22:29:07.830592","last_clean":"2024-04-17 22:29:07.830592","last_became_active":"2024-04-17 22:29:07.826842","last_became_peered":"2024-04-17 22:29:07.826842","last_unstale":"2024-04-17 22:29:07.830592","last_undegraded":"2024-04-17 22:29:07.830592","last_fullsized":"2024-04-17 22:29:07.830592","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.807458","last_change":"2024-04-17 22:29:07.477248","last_active":"2024-04-17 22:29:07.807458","last_peered":"2024-04-17 22:29:07.807458","last_clean":"2024-04-17 22:29:07.807458","last_became_active":"2024-04-17 22:29:07.476944","last_became_peered":"2024-04-17 22:29:07.476944","last_unstale":"2024-04-17 22:29:07.807458","last_undegraded":"2024-04-17 22:29:07.807458","last_fullsized":"2024-04-17 22:29:07.807458","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.826061","last_change":"2024-04-17 22:29:07.822961","last_active":"2024-04-17 22:29:07.826061","last_peered":"2024-04-17 22:29:07.826061","last_clean":"2024-04-17 22:29:07.826061","last_became_active":"2024-04-17 22:29:07.822592","last_became_peered":"2024-04-17 22:29:07.822592","last_unstale":"2024-04-17 22:29:07.826061","last_undegraded":"2024-04-17 22:29:07.826061","last_fullsized":"2024-04-17 22:29:07.826061","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.15","version":"15'3","reported_seq":"13","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.826924","last_change":"2024-04-17 22:29:07.485166","last_active":"2024-04-17 22:29:07.826924","last_peered":"2024-04-17 22:29:07.826924","last_clean":"2024-04-17 22:29:07.826924","last_became_active":"2024-04-17 22:29:07.484870","last_became_peered":"2024-04-17 22:29:07.484870","last_unstale":"2024-04-17 22:29:07.826924","last_undegraded":"2024-04-17 22:29:07.826924","last_fullsized":"2024-04-17 22:29:07.826924","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.825665","last_change":"2024-04-17 22:29:07.814509","last_active":"2024-04-17 22:29:07.825665","last_peered":"2024-04-17 22:29:07.825665","last_clean":"2024-04-17 22:29:07.825665","last_became_active":"2024-04-17 22:29:07.814119","last_became_peered":"2024-04-17 22:29:07.814119","last_unstale":"2024-04-17 22:29:07.825665","last_undegraded":"2024-04-17 22:29:07.825665","last_fullsized":"2024-04-17 22:29:07.825665","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.14","version":"15'2","reported_seq":"12","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.823699","last_change":"2024-04-17 22:29:07.473309","last_active":"2024-04-17 22:29:07.823699","last_peered":"2024-04-17 22:29:07.823699","last_clean":"2024-04-17 22:29:07.823699","last_became_active":"2024-04-17 22:29:07.472976","last_became_peered":"2024-04-17 22:29:07.472976","last_unstale":"2024-04-17 22:29:07.823699","last_undegraded":"2024-04-17 22:29:07.823699","last_fullsized":"2024-04-17 22:29:07.823699","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":634,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.832027","last_change":"2024-04-17 22:29:07.827915","last_active":"2024-04-17 22:29:07.832027","last_peered":"2024-04-17 22:29:07.832027","last_clean":"2024-04-17 22:29:07.832027","last_became_active":"2024-04-17 22:29:07.827670","last_became_peered":"2024-04-17 22:29:07.827670","last_unstale":"2024-04-17 22:29:07.832027","last_undegraded":"2024-04-17 22:29:07.832027","last_fullsized":"2024-04-17 22:29:07.832027","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"15'1","reported_seq":"11","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.827199","last_change":"2024-04-17 22:29:07.471458","last_active":"2024-04-17 22:29:07.827199","last_peered":"2024-04-17 22:29:07.827199","last_clean":"2024-04-17 22:29:07.827199","last_became_active":"2024-04-17 22:29:07.471085","last_became_peered":"2024-04-17 22:29:07.471085","last_unstale":"2024-04-17 22:29:07.827199","last_undegraded":"2024-04-17 22:29:07.827199","last_fullsized":"2024-04-17 22:29:07.827199","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.833535","last_change":"2024-04-17 22:29:07.829288","last_active":"2024-04-17 22:29:07.833535","last_peered":"2024-04-17 22:29:07.833535","last_clean":"2024-04-17 22:29:07.833535","last_became_active":"2024-04-17 22:29:07.828994","last_became_peered":"2024-04-17 22:29:07.828994","last_unstale":"2024-04-17 22:29:07.833535","last_undegraded":"2024-04-17 22:29:07.833535","last_fullsized":"2024-04-17 22:29:07.833535","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.805144","last_change":"2024-04-17 22:29:07.480194","last_active":"2024-04-17 22:29:07.805144","last_peered":"2024-04-17 22:29:07.805144","last_clean":"2024-04-17 22:29:07.805144","last_became_active":"2024-04-17 22:29:07.479728","last_became_peered":"2024-04-17 22:29:07.479728","last_unstale":"2024-04-17 22:29:07.805144","last_undegraded":"2024-04-17 22:29:07.805144","last_fullsized":"2024-04-17 22:29:07.805144","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.820915","last_change":"2024-04-17 22:29:07.815790","last_active":"2024-04-17 22:29:07.820915","last_peered":"2024-04-17 22:29:07.820915","last_clean":"2024-04-17 22:29:07.820915","last_became_active":"2024-04-17 22:29:07.815632","last_became_peered":"2024-04-17 22:29:07.815632","last_unstale":"2024-04-17 22:29:07.820915","last_undegraded":"2024-04-17 22:29:07.820915","last_fullsized":"2024-04-17 22:29:07.820915","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.804035","last_change":"2024-04-17 22:29:07.475830","last_active":"2024-04-17 22:29:07.804035","last_peered":"2024-04-17 22:29:07.804035","last_clean":"2024-04-17 22:29:07.804035","last_became_active":"2024-04-17 22:29:07.474411","last_became_peered":"2024-04-17 22:29:07.474411","last_unstale":"2024-04-17 22:29:07.804035","last_undegraded":"2024-04-17 22:29:07.804035","last_fullsized":"2024-04-17 22:29:07.804035","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.828447","last_change":"2024-04-17 22:29:07.826101","last_active":"2024-04-17 22:29:07.828447","last_peered":"2024-04-17 22:29:07.828447","last_clean":"2024-04-17 22:29:07.828447","last_became_active":"2024-04-17 22:29:07.822917","last_became_peered":"2024-04-17 22:29:07.822917","last_unstale":"2024-04-17 22:29:07.828447","last_undegraded":"2024-04-17 22:29:07.828447","last_fullsized":"2024-04-17 22:29:07.828447","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.10","version":"15'4","reported_seq":"14","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.817471","last_change":"2024-04-17 22:29:07.457017","last_active":"2024-04-17 22:29:07.817471","last_peered":"2024-04-17 22:29:07.817471","last_clean":"2024-04-17 22:29:07.817471","last_became_active":"2024-04-17 22:29:07.456705","last_became_peered":"2024-04-17 22:29:07.456705","last_unstale":"2024-04-17 22:29:07.817471","last_undegraded":"2024-04-17 22:29:07.817471","last_fullsized":"2024-04-17 22:29:07.817471","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.825122","last_change":"2024-04-17 22:29:07.820464","last_active":"2024-04-17 22:29:07.825122","last_peered":"2024-04-17 22:29:07.825122","last_clean":"2024-04-17 22:29:07.825122","last_became_active":"2024-04-17 22:29:07.820151","last_became_peered":"2024-04-17 22:29:07.820151","last_unstale":"2024-04-17 22:29:07.825122","last_undegraded":"2024-04-17 22:29:07.825122","last_fullsized":"2024-04-17 22:29:07.825122","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.f","version":"15'1","reported_seq":"11","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.828338","last_change":"2024-04-17 22:29:07.478464","last_active":"2024-04-17 22:29:07.828338","last_peered":"2024-04-17 22:29:07.828338","last_clean":"2024-04-17 22:29:07.828338","last_became_active":"2024-04-17 22:29:07.478144","last_became_peered":"2024-04-17 22:29:07.478144","last_unstale":"2024-04-17 22:29:07.828338","last_undegraded":"2024-04-17 22:29:07.828338","last_fullsized":"2024-04-17 22:29:07.828338","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.828463","last_change":"2024-04-17 22:29:07.815171","last_active":"2024-04-17 22:29:07.828463","last_peered":"2024-04-17 22:29:07.828463","last_clean":"2024-04-17 22:29:07.828463","last_became_active":"2024-04-17 22:29:07.814830","last_became_peered":"2024-04-17 22:29:07.814830","last_unstale":"2024-04-17 22:29:07.828463","last_undegraded":"2024-04-17 22:29:07.828463","last_fullsized":"2024-04-17 22:29:07.828463","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"15'2","reported_seq":"12","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.811237","last_change":"2024-04-17 22:29:07.481865","last_active":"2024-04-17 22:29:07.811237","last_peered":"2024-04-17 22:29:07.811237","last_clean":"2024-04-17 22:29:07.811237","last_became_active":"2024-04-17 22:29:07.481506","last_became_peered":"2024-04-17 22:29:07.481506","last_unstale":"2024-04-17 22:29:07.811237","last_undegraded":"2024-04-17 22:29:07.811237","last_fullsized":"2024-04-17 22:29:07.811237","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.828178","last_change":"2024-04-17 22:29:07.818238","last_active":"2024-04-17 22:29:07.828178","last_peered":"2024-04-17 22:29:07.828178","last_clean":"2024-04-17 22:29:07.828178","last_became_active":"2024-04-17 22:29:07.818060","last_became_peered":"2024-04-17 22:29:07.818060","last_unstale":"2024-04-17 22:29:07.828178","last_undegraded":"2024-04-17 22:29:07.828178","last_fullsized":"2024-04-17 22:29:07.828178","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.796955","last_change":"2024-04-17 22:29:07.484999","last_active":"2024-04-17 22:29:07.796955","last_peered":"2024-04-17 22:29:07.796955","last_clean":"2024-04-17 22:29:07.796955","last_became_active":"2024-04-17 22:29:07.484668","last_became_peered":"2024-04-17 22:29:07.484668","last_unstale":"2024-04-17 22:29:07.796955","last_undegraded":"2024-04-17 22:29:07.796955","last_fullsized":"2024-04-17 22:29:07.796955","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.828594","last_change":"2024-04-17 22:29:07.826426","last_active":"2024-04-17 22:29:07.828594","last_peered":"2024-04-17 22:29:07.828594","last_clean":"2024-04-17 22:29:07.828594","last_became_active":"2024-04-17 22:29:07.826109","last_became_peered":"2024-04-17 22:29:07.826109","last_unstale":"2024-04-17 22:29:07.828594","last_undegraded":"2024-04-17 22:29:07.828594","last_fullsized":"2024-04-17 22:29:07.828594","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.797207","last_change":"2024-04-17 22:29:07.462218","last_active":"2024-04-17 22:29:07.797207","last_peered":"2024-04-17 22:29:07.797207","last_clean":"2024-04-17 22:29:07.797207","last_became_active":"2024-04-17 22:29:07.461842","last_became_peered":"2024-04-17 22:29:07.461842","last_unstale":"2024-04-17 22:29:07.797207","last_undegraded":"2024-04-17 22:29:07.797207","last_fullsized":"2024-04-17 22:29:07.797207","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.820296","last_change":"2024-04-17 22:29:07.817144","last_active":"2024-04-17 22:29:07.820296","last_peered":"2024-04-17 22:29:07.820296","last_clean":"2024-04-17 22:29:07.820296","last_became_active":"2024-04-17 22:29:07.816859","last_became_peered":"2024-04-17 22:29:07.816859","last_unstale":"2024-04-17 22:29:07.820296","last_undegraded":"2024-04-17 22:29:07.820296","last_fullsized":"2024-04-17 22:29:07.820296","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"15'1","reported_seq":"11","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.821604","last_change":"2024-04-17 22:29:07.479816","last_active":"2024-04-17 22:29:07.821604","last_peered":"2024-04-17 22:29:07.821604","last_clean":"2024-04-17 22:29:07.821604","last_became_active":"2024-04-17 22:29:07.479251","last_became_peered":"2024-04-17 22:29:07.479251","last_unstale":"2024-04-17 22:29:07.821604","last_undegraded":"2024-04-17 22:29:07.821604","last_fullsized":"2024-04-17 22:29:07.821604","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.833516","last_change":"2024-04-17 22:29:07.829215","last_active":"2024-04-17 22:29:07.833516","last_peered":"2024-04-17 22:29:07.833516","last_clean":"2024-04-17 22:29:07.833516","last_became_active":"2024-04-17 22:29:07.828803","last_became_peered":"2024-04-17 22:29:07.828803","last_unstale":"2024-04-17 22:29:07.833516","last_undegraded":"2024-04-17 22:29:07.833516","last_fullsized":"2024-04-17 22:29:07.833516","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.796428","last_change":"2024-04-17 22:29:07.473128","last_active":"2024-04-17 22:29:07.796428","last_peered":"2024-04-17 22:29:07.796428","last_clean":"2024-04-17 22:29:07.796428","last_became_active":"2024-04-17 22:29:07.472806","last_became_peered":"2024-04-17 22:29:07.472806","last_unstale":"2024-04-17 22:29:07.796428","last_undegraded":"2024-04-17 22:29:07.796428","last_fullsized":"2024-04-17 22:29:07.796428","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.820555","last_change":"2024-04-17 22:29:07.817630","last_active":"2024-04-17 22:29:07.820555","last_peered":"2024-04-17 22:29:07.820555","last_clean":"2024-04-17 22:29:07.820555","last_became_active":"2024-04-17 22:29:07.817333","last_became_peered":"2024-04-17 22:29:07.817333","last_unstale":"2024-04-17 22:29:07.820555","last_undegraded":"2024-04-17 22:29:07.820555","last_fullsized":"2024-04-17 22:29:07.820555","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.792279","last_change":"2024-04-17 22:29:07.465053","last_active":"2024-04-17 22:29:07.792279","last_peered":"2024-04-17 22:29:07.792279","last_clean":"2024-04-17 22:29:07.792279","last_became_active":"2024-04-17 22:29:07.464766","last_became_peered":"2024-04-17 22:29:07.464766","last_unstale":"2024-04-17 22:29:07.792279","last_undegraded":"2024-04-17 22:29:07.792279","last_fullsized":"2024-04-17 22:29:07.792279","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.827066","last_change":"2024-04-17 22:29:07.823514","last_active":"2024-04-17 22:29:07.827066","last_peered":"2024-04-17 22:29:07.827066","last_clean":"2024-04-17 22:29:07.827066","last_became_active":"2024-04-17 22:29:07.823007","last_became_peered":"2024-04-17 22:29:07.823007","last_unstale":"2024-04-17 22:29:07.827066","last_undegraded":"2024-04-17 22:29:07.827066","last_fullsized":"2024-04-17 22:29:07.827066","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.801156","last_change":"2024-04-17 22:29:07.461568","last_active":"2024-04-17 22:29:07.801156","last_peered":"2024-04-17 22:29:07.801156","last_clean":"2024-04-17 22:29:07.801156","last_became_active":"2024-04-17 22:29:07.461249","last_became_peered":"2024-04-17 22:29:07.461249","last_unstale":"2024-04-17 22:29:07.801156","last_undegraded":"2024-04-17 22:29:07.801156","last_fullsized":"2024-04-17 22:29:07.801156","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.828426","last_change":"2024-04-17 22:29:07.826342","last_active":"2024-04-17 22:29:07.828426","last_peered":"2024-04-17 22:29:07.828426","last_clean":"2024-04-17 22:29:07.828426","last_became_active":"2024-04-17 22:29:07.826029","last_became_peered":"2024-04-17 22:29:07.826029","last_unstale":"2024-04-17 22:29:07.828426","last_undegraded":"2024-04-17 22:29:07.828426","last_fullsized":"2024-04-17 22:29:07.828426","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":"13","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.795463","last_change":"2024-04-17 22:28:59.253977","last_active":"2024-04-17 22:29:07.795463","last_peered":"2024-04-17 22:29:07.795463","last_clean":"2024-04-17 22:29:07.795463","last_became_active":"2024-04-17 22:28:59.253641","last_became_peered":"2024-04-17 22:28:59.253641","last_unstale":"2024-04-17 22:29:07.795463","last_undegraded":"2024-04-17 22:29:07.795463","last_fullsized":"2024-04-17 22:29:07.795463","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:28:58.239187","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:28:58.239187","last_clean_scrub_stamp":"2024-04-17 22:28:58.239187","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.796951","last_change":"2024-04-17 22:29:07.466907","last_active":"2024-04-17 22:29:07.796951","last_peered":"2024-04-17 22:29:07.796951","last_clean":"2024-04-17 22:29:07.796951","last_became_active":"2024-04-17 22:29:07.466583","last_became_peered":"2024-04-17 22:29:07.466583","last_unstale":"2024-04-17 22:29:07.796951","last_undegraded":"2024-04-17 22:29:07.796951","last_fullsized":"2024-04-17 22:29:07.796951","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.829316","last_change":"2024-04-17 22:29:07.824875","last_active":"2024-04-17 22:29:07.829316","last_peered":"2024-04-17 22:29:07.829316","last_clean":"2024-04-17 22:29:07.829316","last_became_active":"2024-04-17 22:29:07.824377","last_became_peered":"2024-04-17 22:29:07.824377","last_unstale":"2024-04-17 22:29:07.829316","last_undegraded":"2024-04-17 22:29:07.829316","last_fullsized":"2024-04-17 22:29:07.829316","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":"13","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.809532","last_change":"2024-04-17 22:28:59.255733","last_active":"2024-04-17 22:29:07.809532","last_peered":"2024-04-17 22:29:07.809532","last_clean":"2024-04-17 22:29:07.809532","last_became_active":"2024-04-17 22:28:59.255411","last_became_peered":"2024-04-17 22:28:59.255411","last_unstale":"2024-04-17 22:29:07.809532","last_undegraded":"2024-04-17 22:29:07.809532","last_fullsized":"2024-04-17 22:29:07.809532","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:28:58.239187","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:28:58.239187","last_clean_scrub_stamp":"2024-04-17 22:28:58.239187","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"15'2","reported_seq":"12","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.832132","last_change":"2024-04-17 22:29:07.478196","last_active":"2024-04-17 22:29:07.832132","last_peered":"2024-04-17 22:29:07.832132","last_clean":"2024-04-17 22:29:07.832132","last_became_active":"2024-04-17 22:29:07.477887","last_became_peered":"2024-04-17 22:29:07.477887","last_unstale":"2024-04-17 22:29:07.832132","last_undegraded":"2024-04-17 22:29:07.832132","last_fullsized":"2024-04-17 22:29:07.832132","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":"13","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.796430","last_change":"2024-04-17 22:28:59.252728","last_active":"2024-04-17 22:29:07.796430","last_peered":"2024-04-17 22:29:07.796430","last_clean":"2024-04-17 22:29:07.796430","last_became_active":"2024-04-17 22:28:59.252345","last_became_peered":"2024-04-17 22:28:59.252345","last_unstale":"2024-04-17 22:29:07.796430","last_undegraded":"2024-04-17 22:29:07.796430","last_fullsized":"2024-04-17 22:29:07.796430","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:28:58.239187","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:28:58.239187","last_clean_scrub_stamp":"2024-04-17 22:28:58.239187","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.797727","last_change":"2024-04-17 22:29:07.474520","last_active":"2024-04-17 22:29:07.797727","last_peered":"2024-04-17 22:29:07.797727","last_clean":"2024-04-17 22:29:07.797727","last_became_active":"2024-04-17 22:29:07.474029","last_became_peered":"2024-04-17 22:29:07.474029","last_unstale":"2024-04-17 22:29:07.797727","last_undegraded":"2024-04-17 22:29:07.797727","last_fullsized":"2024-04-17 22:29:07.797727","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.829302","last_change":"2024-04-17 22:29:07.825091","last_active":"2024-04-17 22:29:07.829302","last_peered":"2024-04-17 22:29:07.829302","last_clean":"2024-04-17 22:29:07.829302","last_became_active":"2024-04-17 22:29:07.824832","last_became_peered":"2024-04-17 22:29:07.824832","last_unstale":"2024-04-17 22:29:07.829302","last_undegraded":"2024-04-17 22:29:07.829302","last_fullsized":"2024-04-17 22:29:07.829302","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":"13","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.807580","last_change":"2024-04-17 22:28:59.253392","last_active":"2024-04-17 22:29:07.807580","last_peered":"2024-04-17 22:29:07.807580","last_clean":"2024-04-17 22:29:07.807580","last_became_active":"2024-04-17 22:28:59.253049","last_became_peered":"2024-04-17 22:28:59.253049","last_unstale":"2024-04-17 22:29:07.807580","last_undegraded":"2024-04-17 22:29:07.807580","last_fullsized":"2024-04-17 22:29:07.807580","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:28:58.239187","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:28:58.239187","last_clean_scrub_stamp":"2024-04-17 22:28:58.239187","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.796512","last_change":"2024-04-17 22:29:07.471913","last_active":"2024-04-17 22:29:07.796512","last_peered":"2024-04-17 22:29:07.796512","last_clean":"2024-04-17 22:29:07.796512","last_became_active":"2024-04-17 22:29:07.471686","last_became_peered":"2024-04-17 22:29:07.471686","last_unstale":"2024-04-17 22:29:07.796512","last_undegraded":"2024-04-17 22:29:07.796512","last_fullsized":"2024-04-17 22:29:07.796512","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.820450","last_change":"2024-04-17 22:29:07.812999","last_active":"2024-04-17 22:29:07.820450","last_peered":"2024-04-17 22:29:07.820450","last_clean":"2024-04-17 22:29:07.820450","last_became_active":"2024-04-17 22:29:07.812701","last_became_peered":"2024-04-17 22:29:07.812701","last_unstale":"2024-04-17 22:29:07.820450","last_undegraded":"2024-04-17 22:29:07.820450","last_fullsized":"2024-04-17 22:29:07.820450","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":"13","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.791848","last_change":"2024-04-17 22:28:59.254907","last_active":"2024-04-17 22:29:07.791848","last_peered":"2024-04-17 22:29:07.791848","last_clean":"2024-04-17 22:29:07.791848","last_became_active":"2024-04-17 22:28:59.254633","last_became_peered":"2024-04-17 22:28:59.254633","last_unstale":"2024-04-17 22:29:07.791848","last_undegraded":"2024-04-17 22:29:07.791848","last_fullsized":"2024-04-17 22:29:07.791848","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:28:58.239187","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:28:58.239187","last_clean_scrub_stamp":"2024-04-17 22:28:58.239187","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.807141","last_change":"2024-04-17 22:29:07.480020","last_active":"2024-04-17 22:29:07.807141","last_peered":"2024-04-17 22:29:07.807141","last_clean":"2024-04-17 22:29:07.807141","last_became_active":"2024-04-17 22:29:07.479492","last_became_peered":"2024-04-17 22:29:07.479492","last_unstale":"2024-04-17 22:29:07.807141","last_undegraded":"2024-04-17 22:29:07.807141","last_fullsized":"2024-04-17 22:29:07.807141","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.826030","last_change":"2024-04-17 22:29:07.821883","last_active":"2024-04-17 22:29:07.826030","last_peered":"2024-04-17 22:29:07.826030","last_clean":"2024-04-17 22:29:07.826030","last_became_active":"2024-04-17 22:29:07.821569","last_became_peered":"2024-04-17 22:29:07.821569","last_unstale":"2024-04-17 22:29:07.826030","last_undegraded":"2024-04-17 22:29:07.826030","last_fullsized":"2024-04-17 22:29:07.826030","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":"13","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.806466","last_change":"2024-04-17 22:28:59.254217","last_active":"2024-04-17 22:29:07.806466","last_peered":"2024-04-17 22:29:07.806466","last_clean":"2024-04-17 22:29:07.806466","last_became_active":"2024-04-17 22:28:59.253877","last_became_peered":"2024-04-17 22:28:59.253877","last_unstale":"2024-04-17 22:29:07.806466","last_undegraded":"2024-04-17 22:29:07.806466","last_fullsized":"2024-04-17 22:29:07.806466","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:28:58.239187","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:28:58.239187","last_clean_scrub_stamp":"2024-04-17 22:28:58.239187","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.805257","last_change":"2024-04-17 22:29:07.480378","last_active":"2024-04-17 22:29:07.805257","last_peered":"2024-04-17 22:29:07.805257","last_clean":"2024-04-17 22:29:07.805257","last_became_active":"2024-04-17 22:29:07.479866","last_became_peered":"2024-04-17 22:29:07.479866","last_unstale":"2024-04-17 22:29:07.805257","last_undegraded":"2024-04-17 22:29:07.805257","last_fullsized":"2024-04-17 22:29:07.805257","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.829882","last_change":"2024-04-17 22:29:07.821315","last_active":"2024-04-17 22:29:07.829882","last_peered":"2024-04-17 22:29:07.829882","last_clean":"2024-04-17 22:29:07.829882","last_became_active":"2024-04-17 22:29:07.820937","last_became_peered":"2024-04-17 22:29:07.820937","last_unstale":"2024-04-17 22:29:07.829882","last_undegraded":"2024-04-17 22:29:07.829882","last_fullsized":"2024-04-17 22:29:07.829882","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":"13","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.813317","last_change":"2024-04-17 22:28:59.254830","last_active":"2024-04-17 22:29:07.813317","last_peered":"2024-04-17 22:29:07.813317","last_clean":"2024-04-17 22:29:07.813317","last_became_active":"2024-04-17 22:28:59.254505","last_became_peered":"2024-04-17 22:28:59.254505","last_unstale":"2024-04-17 22:29:07.813317","last_undegraded":"2024-04-17 22:29:07.813317","last_fullsized":"2024-04-17 22:29:07.813317","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:28:58.239187","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:28:58.239187","last_clean_scrub_stamp":"2024-04-17 22:28:58.239187","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.809624","last_change":"2024-04-17 22:29:07.475017","last_active":"2024-04-17 22:29:07.809624","last_peered":"2024-04-17 22:29:07.809624","last_clean":"2024-04-17 22:29:07.809624","last_became_active":"2024-04-17 22:29:07.473559","last_became_peered":"2024-04-17 22:29:07.473559","last_unstale":"2024-04-17 22:29:07.809624","last_undegraded":"2024-04-17 22:29:07.809624","last_fullsized":"2024-04-17 22:29:07.809624","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.811282","last_change":"2024-04-17 22:29:07.808988","last_active":"2024-04-17 22:29:07.811282","last_peered":"2024-04-17 22:29:07.811282","last_clean":"2024-04-17 22:29:07.811282","last_became_active":"2024-04-17 22:29:07.808663","last_became_peered":"2024-04-17 22:29:07.808663","last_unstale":"2024-04-17 22:29:07.811282","last_undegraded":"2024-04-17 22:29:07.811282","last_fullsized":"2024-04-17 22:29:07.811282","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":"13","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.796932","last_change":"2024-04-17 22:28:59.254055","last_active":"2024-04-17 22:29:07.796932","last_peered":"2024-04-17 22:29:07.796932","last_clean":"2024-04-17 22:29:07.796932","last_became_active":"2024-04-17 22:28:59.253750","last_became_peered":"2024-04-17 22:28:59.253750","last_unstale":"2024-04-17 22:29:07.796932","last_undegraded":"2024-04-17 22:29:07.796932","last_fullsized":"2024-04-17 22:29:07.796932","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:28:58.239187","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:28:58.239187","last_clean_scrub_stamp":"2024-04-17 22:28:58.239187","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.796660","last_change":"2024-04-17 22:29:07.466238","last_active":"2024-04-17 22:29:07.796660","last_peered":"2024-04-17 22:29:07.796660","last_clean":"2024-04-17 22:29:07.796660","last_became_active":"2024-04-17 22:29:07.465806","last_became_peered":"2024-04-17 22:29:07.465806","last_unstale":"2024-04-17 22:29:07.796660","last_undegraded":"2024-04-17 22:29:07.796660","last_fullsized":"2024-04-17 22:29:07.796660","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.820702","last_change":"2024-04-17 22:29:07.809155","last_active":"2024-04-17 22:29:07.820702","last_peered":"2024-04-17 22:29:07.820702","last_clean":"2024-04-17 22:29:07.820702","last_became_active":"2024-04-17 22:29:07.808850","last_became_peered":"2024-04-17 22:29:07.808850","last_unstale":"2024-04-17 22:29:07.820702","last_undegraded":"2024-04-17 22:29:07.820702","last_fullsized":"2024-04-17 22:29:07.820702","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.833569","last_change":"2024-04-17 22:29:07.832794","last_active":"2024-04-17 22:29:07.833569","last_peered":"2024-04-17 22:29:07.833569","last_clean":"2024-04-17 22:29:07.833569","last_became_active":"2024-04-17 22:29:07.832467","last_became_peered":"2024-04-17 22:29:07.832467","last_unstale":"2024-04-17 22:29:07.833569","last_undegraded":"2024-04-17 22:29:07.833569","last_fullsized":"2024-04-17 22:29:07.833569","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.811305","last_change":"2024-04-17 22:29:07.477743","last_active":"2024-04-17 22:29:07.811305","last_peered":"2024-04-17 22:29:07.811305","last_clean":"2024-04-17 22:29:07.811305","last_became_active":"2024-04-17 22:29:07.477503","last_became_peered":"2024-04-17 22:29:07.477503","last_unstale":"2024-04-17 22:29:07.811305","last_undegraded":"2024-04-17 22:29:07.811305","last_fullsized":"2024-04-17 22:29:07.811305","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.830256","last_change":"2024-04-17 22:29:07.828401","last_active":"2024-04-17 22:29:07.830256","last_peered":"2024-04-17 22:29:07.830256","last_clean":"2024-04-17 22:29:07.830256","last_became_active":"2024-04-17 22:29:07.828093","last_became_peered":"2024-04-17 22:29:07.828093","last_unstale":"2024-04-17 22:29:07.830256","last_undegraded":"2024-04-17 22:29:07.830256","last_fullsized":"2024-04-17 22:29:07.830256","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"15'2","reported_seq":"12","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.824506","last_change":"2024-04-17 22:29:07.478361","last_active":"2024-04-17 22:29:07.824506","last_peered":"2024-04-17 22:29:07.824506","last_clean":"2024-04-17 22:29:07.824506","last_became_active":"2024-04-17 22:29:07.477793","last_became_peered":"2024-04-17 22:29:07.477793","last_unstale":"2024-04-17 22:29:07.824506","last_undegraded":"2024-04-17 22:29:07.824506","last_fullsized":"2024-04-17 22:29:07.824506","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.828482","last_change":"2024-04-17 22:29:07.815548","last_active":"2024-04-17 22:29:07.828482","last_peered":"2024-04-17 22:29:07.828482","last_clean":"2024-04-17 22:29:07.828482","last_became_active":"2024-04-17 22:29:07.815263","last_became_peered":"2024-04-17 22:29:07.815263","last_unstale":"2024-04-17 22:29:07.828482","last_undegraded":"2024-04-17 22:29:07.828482","last_fullsized":"2024-04-17 22:29:07.828482","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"15'1","reported_seq":"11","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.831868","last_change":"2024-04-17 22:29:07.482507","last_active":"2024-04-17 22:29:07.831868","last_peered":"2024-04-17 22:29:07.831868","last_clean":"2024-04-17 22:29:07.831868","last_became_active":"2024-04-17 22:29:07.481981","last_became_peered":"2024-04-17 22:29:07.481981","last_unstale":"2024-04-17 22:29:07.831868","last_undegraded":"2024-04-17 22:29:07.831868","last_fullsized":"2024-04-17 22:29:07.831868","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.833109","last_change":"2024-04-17 22:29:07.828459","last_active":"2024-04-17 22:29:07.833109","last_peered":"2024-04-17 22:29:07.833109","last_clean":"2024-04-17 22:29:07.833109","last_became_active":"2024-04-17 22:29:07.828157","last_became_peered":"2024-04-17 22:29:07.828157","last_unstale":"2024-04-17 22:29:07.833109","last_undegraded":"2024-04-17 22:29:07.833109","last_fullsized":"2024-04-17 22:29:07.833109","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"15'1","reported_seq":"11","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.828398","last_change":"2024-04-17 22:29:07.471751","last_active":"2024-04-17 22:29:07.828398","last_peered":"2024-04-17 22:29:07.828398","last_clean":"2024-04-17 22:29:07.828398","last_became_active":"2024-04-17 22:29:07.471385","last_became_peered":"2024-04-17 22:29:07.471385","last_unstale":"2024-04-17 22:29:07.828398","last_undegraded":"2024-04-17 22:29:07.828398","last_fullsized":"2024-04-17 22:29:07.828398","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.821701","last_change":"2024-04-17 22:29:07.812125","last_active":"2024-04-17 22:29:07.821701","last_peered":"2024-04-17 22:29:07.821701","last_clean":"2024-04-17 22:29:07.821701","last_became_active":"2024-04-17 22:29:07.811846","last_became_peered":"2024-04-17 22:29:07.811846","last_unstale":"2024-04-17 22:29:07.821701","last_undegraded":"2024-04-17 22:29:07.821701","last_fullsized":"2024-04-17 22:29:07.821701","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.20","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.798111","last_change":"2024-04-17 22:29:07.477127","last_active":"2024-04-17 22:29:07.798111","last_peered":"2024-04-17 22:29:07.798111","last_clean":"2024-04-17 22:29:07.798111","last_became_active":"2024-04-17 22:29:07.476773","last_became_peered":"2024-04-17 22:29:07.476773","last_unstale":"2024-04-17 22:29:07.798111","last_undegraded":"2024-04-17 22:29:07.798111","last_fullsized":"2024-04-17 22:29:07.798111","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.824583","last_change":"2024-04-17 22:29:07.821132","last_active":"2024-04-17 22:29:07.824583","last_peered":"2024-04-17 22:29:07.824583","last_clean":"2024-04-17 22:29:07.824583","last_became_active":"2024-04-17 22:29:07.820838","last_became_peered":"2024-04-17 22:29:07.820838","last_unstale":"2024-04-17 22:29:07.824583","last_undegraded":"2024-04-17 22:29:07.824583","last_fullsized":"2024-04-17 22:29:07.824583","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.21","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.796071","last_change":"2024-04-17 22:29:07.460110","last_active":"2024-04-17 22:29:07.796071","last_peered":"2024-04-17 22:29:07.796071","last_clean":"2024-04-17 22:29:07.796071","last_became_active":"2024-04-17 22:29:07.459611","last_became_peered":"2024-04-17 22:29:07.459611","last_unstale":"2024-04-17 22:29:07.796071","last_undegraded":"2024-04-17 22:29:07.796071","last_fullsized":"2024-04-17 22:29:07.796071","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.828609","last_change":"2024-04-17 22:29:07.818496","last_active":"2024-04-17 22:29:07.828609","last_peered":"2024-04-17 22:29:07.828609","last_clean":"2024-04-17 22:29:07.828609","last_became_active":"2024-04-17 22:29:07.818184","last_became_peered":"2024-04-17 22:29:07.818184","last_unstale":"2024-04-17 22:29:07.828609","last_undegraded":"2024-04-17 22:29:07.828609","last_fullsized":"2024-04-17 22:29:07.828609","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.22","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.805451","last_change":"2024-04-17 22:29:07.481695","last_active":"2024-04-17 22:29:07.805451","last_peered":"2024-04-17 22:29:07.805451","last_clean":"2024-04-17 22:29:07.805451","last_became_active":"2024-04-17 22:29:07.479982","last_became_peered":"2024-04-17 22:29:07.479982","last_unstale":"2024-04-17 22:29:07.805451","last_undegraded":"2024-04-17 22:29:07.805451","last_fullsized":"2024-04-17 22:29:07.805451","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.835093","last_change":"2024-04-17 22:29:07.833317","last_active":"2024-04-17 22:29:07.835093","last_peered":"2024-04-17 22:29:07.835093","last_clean":"2024-04-17 22:29:07.835093","last_became_active":"2024-04-17 22:29:07.827050","last_became_peered":"2024-04-17 22:29:07.827050","last_unstale":"2024-04-17 22:29:07.835093","last_undegraded":"2024-04-17 22:29:07.835093","last_fullsized":"2024-04-17 22:29:07.835093","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.23","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.807837","last_change":"2024-04-17 22:29:07.477449","last_active":"2024-04-17 22:29:07.807837","last_peered":"2024-04-17 22:29:07.807837","last_clean":"2024-04-17 22:29:07.807837","last_became_active":"2024-04-17 22:29:07.477155","last_became_peered":"2024-04-17 22:29:07.477155","last_unstale":"2024-04-17 22:29:07.807837","last_undegraded":"2024-04-17 22:29:07.807837","last_fullsized":"2024-04-17 22:29:07.807837","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.810313","last_change":"2024-04-17 22:29:07.808974","last_active":"2024-04-17 22:29:07.810313","last_peered":"2024-04-17 22:29:07.810313","last_clean":"2024-04-17 22:29:07.810313","last_became_active":"2024-04-17 22:29:07.808747","last_became_peered":"2024-04-17 22:29:07.808747","last_unstale":"2024-04-17 22:29:07.810313","last_undegraded":"2024-04-17 22:29:07.810313","last_fullsized":"2024-04-17 22:29:07.810313","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.24","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.796551","last_change":"2024-04-17 22:29:07.466052","last_active":"2024-04-17 22:29:07.796551","last_peered":"2024-04-17 22:29:07.796551","last_clean":"2024-04-17 22:29:07.796551","last_became_active":"2024-04-17 22:29:07.465576","last_became_peered":"2024-04-17 22:29:07.465576","last_unstale":"2024-04-17 22:29:07.796551","last_undegraded":"2024-04-17 22:29:07.796551","last_fullsized":"2024-04-17 22:29:07.796551","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.833471","last_change":"2024-04-17 22:29:07.829066","last_active":"2024-04-17 22:29:07.833471","last_peered":"2024-04-17 22:29:07.833471","last_clean":"2024-04-17 22:29:07.833471","last_became_active":"2024-04-17 22:29:07.828735","last_became_peered":"2024-04-17 22:29:07.828735","last_unstale":"2024-04-17 22:29:07.833471","last_undegraded":"2024-04-17 22:29:07.833471","last_fullsized":"2024-04-17 22:29:07.833471","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.25","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.795566","last_change":"2024-04-17 22:29:07.462011","last_active":"2024-04-17 22:29:07.795566","last_peered":"2024-04-17 22:29:07.795566","last_clean":"2024-04-17 22:29:07.795566","last_became_active":"2024-04-17 22:29:07.461693","last_became_peered":"2024-04-17 22:29:07.461693","last_unstale":"2024-04-17 22:29:07.795566","last_undegraded":"2024-04-17 22:29:07.795566","last_fullsized":"2024-04-17 22:29:07.795566","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.828447","last_change":"2024-04-17 22:29:07.826655","last_active":"2024-04-17 22:29:07.828447","last_peered":"2024-04-17 22:29:07.828447","last_clean":"2024-04-17 22:29:07.828447","last_became_active":"2024-04-17 22:29:07.823665","last_became_peered":"2024-04-17 22:29:07.823665","last_unstale":"2024-04-17 22:29:07.828447","last_undegraded":"2024-04-17 22:29:07.828447","last_fullsized":"2024-04-17 22:29:07.828447","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.26","version":"16'1","reported_seq":"11","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:08.802774","last_change":"2024-04-17 22:29:07.480854","last_active":"2024-04-17 22:29:08.802774","last_peered":"2024-04-17 22:29:08.802774","last_clean":"2024-04-17 22:29:08.802774","last_became_active":"2024-04-17 22:29:07.479975","last_became_peered":"2024-04-17 22:29:07.479975","last_unstale":"2024-04-17 22:29:08.802774","last_undegraded":"2024-04-17 22:29:08.802774","last_fullsized":"2024-04-17 22:29:08.802774","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.825070","last_change":"2024-04-17 22:29:07.813744","last_active":"2024-04-17 22:29:07.825070","last_peered":"2024-04-17 22:29:07.825070","last_clean":"2024-04-17 22:29:07.825070","last_became_active":"2024-04-17 22:29:07.810646","last_became_peered":"2024-04-17 22:29:07.810646","last_unstale":"2024-04-17 22:29:07.825070","last_undegraded":"2024-04-17 22:29:07.825070","last_fullsized":"2024-04-17 22:29:07.825070","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.27","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.815316","last_change":"2024-04-17 22:29:07.476428","last_active":"2024-04-17 22:29:07.815316","last_peered":"2024-04-17 22:29:07.815316","last_clean":"2024-04-17 22:29:07.815316","last_became_active":"2024-04-17 22:29:07.476128","last_became_peered":"2024-04-17 22:29:07.476128","last_unstale":"2024-04-17 22:29:07.815316","last_undegraded":"2024-04-17 22:29:07.815316","last_fullsized":"2024-04-17 22:29:07.815316","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.826418","last_change":"2024-04-17 22:29:07.823340","last_active":"2024-04-17 22:29:07.826418","last_peered":"2024-04-17 22:29:07.826418","last_clean":"2024-04-17 22:29:07.826418","last_became_active":"2024-04-17 22:29:07.822764","last_became_peered":"2024-04-17 22:29:07.822764","last_unstale":"2024-04-17 22:29:07.826418","last_undegraded":"2024-04-17 22:29:07.826418","last_fullsized":"2024-04-17 22:29:07.826418","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.28","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.798818","last_change":"2024-04-17 22:29:07.477137","last_active":"2024-04-17 22:29:07.798818","last_peered":"2024-04-17 22:29:07.798818","last_clean":"2024-04-17 22:29:07.798818","last_became_active":"2024-04-17 22:29:07.476796","last_became_peered":"2024-04-17 22:29:07.476796","last_unstale":"2024-04-17 22:29:07.798818","last_undegraded":"2024-04-17 22:29:07.798818","last_fullsized":"2024-04-17 22:29:07.798818","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.826239","last_change":"2024-04-17 22:29:07.823119","last_active":"2024-04-17 22:29:07.826239","last_peered":"2024-04-17 22:29:07.826239","last_clean":"2024-04-17 22:29:07.826239","last_became_active":"2024-04-17 22:29:07.822640","last_became_peered":"2024-04-17 22:29:07.822640","last_unstale":"2024-04-17 22:29:07.826239","last_undegraded":"2024-04-17 22:29:07.826239","last_fullsized":"2024-04-17 22:29:07.826239","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.29","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.792382","last_change":"2024-04-17 22:29:07.473467","last_active":"2024-04-17 22:29:07.792382","last_peered":"2024-04-17 22:29:07.792382","last_clean":"2024-04-17 22:29:07.792382","last_became_active":"2024-04-17 22:29:07.473028","last_became_peered":"2024-04-17 22:29:07.473028","last_unstale":"2024-04-17 22:29:07.792382","last_undegraded":"2024-04-17 22:29:07.792382","last_fullsized":"2024-04-17 22:29:07.792382","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.827332","last_change":"2024-04-17 22:29:07.817097","last_active":"2024-04-17 22:29:07.827332","last_peered":"2024-04-17 22:29:07.827332","last_clean":"2024-04-17 22:29:07.827332","last_became_active":"2024-04-17 22:29:07.816813","last_became_peered":"2024-04-17 22:29:07.816813","last_unstale":"2024-04-17 22:29:07.827332","last_undegraded":"2024-04-17 22:29:07.827332","last_fullsized":"2024-04-17 22:29:07.827332","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2a","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.796331","last_change":"2024-04-17 22:29:07.464330","last_active":"2024-04-17 22:29:07.796331","last_peered":"2024-04-17 22:29:07.796331","last_clean":"2024-04-17 22:29:07.796331","last_became_active":"2024-04-17 22:29:07.464002","last_became_peered":"2024-04-17 22:29:07.464002","last_unstale":"2024-04-17 22:29:07.796331","last_undegraded":"2024-04-17 22:29:07.796331","last_fullsized":"2024-04-17 22:29:07.796331","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.830305","last_change":"2024-04-17 22:29:07.828690","last_active":"2024-04-17 22:29:07.830305","last_peered":"2024-04-17 22:29:07.830305","last_clean":"2024-04-17 22:29:07.830305","last_became_active":"2024-04-17 22:29:07.828407","last_became_peered":"2024-04-17 22:29:07.828407","last_unstale":"2024-04-17 22:29:07.830305","last_undegraded":"2024-04-17 22:29:07.830305","last_fullsized":"2024-04-17 22:29:07.830305","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2b","version":"15'1","reported_seq":"11","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.810554","last_change":"2024-04-17 22:29:07.471300","last_active":"2024-04-17 22:29:07.810554","last_peered":"2024-04-17 22:29:07.810554","last_clean":"2024-04-17 22:29:07.810554","last_became_active":"2024-04-17 22:29:07.470886","last_became_peered":"2024-04-17 22:29:07.470886","last_unstale":"2024-04-17 22:29:07.810554","last_undegraded":"2024-04-17 22:29:07.810554","last_fullsized":"2024-04-17 22:29:07.810554","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.828614","last_change":"2024-04-17 22:29:07.826555","last_active":"2024-04-17 22:29:07.828614","last_peered":"2024-04-17 22:29:07.828614","last_clean":"2024-04-17 22:29:07.828614","last_became_active":"2024-04-17 22:29:07.826262","last_became_peered":"2024-04-17 22:29:07.826262","last_unstale":"2024-04-17 22:29:07.828614","last_undegraded":"2024-04-17 22:29:07.828614","last_fullsized":"2024-04-17 22:29:07.828614","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2c","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.797039","last_change":"2024-04-17 22:29:07.462176","last_active":"2024-04-17 22:29:07.797039","last_peered":"2024-04-17 22:29:07.797039","last_clean":"2024-04-17 22:29:07.797039","last_became_active":"2024-04-17 22:29:07.461594","last_became_peered":"2024-04-17 22:29:07.461594","last_unstale":"2024-04-17 22:29:07.797039","last_undegraded":"2024-04-17 22:29:07.797039","last_fullsized":"2024-04-17 22:29:07.797039","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.828130","last_change":"2024-04-17 22:29:07.816896","last_active":"2024-04-17 22:29:07.828130","last_peered":"2024-04-17 22:29:07.828130","last_clean":"2024-04-17 22:29:07.828130","last_became_active":"2024-04-17 22:29:07.816538","last_became_peered":"2024-04-17 22:29:07.816538","last_unstale":"2024-04-17 22:29:07.828130","last_undegraded":"2024-04-17 22:29:07.828130","last_fullsized":"2024-04-17 22:29:07.828130","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2d","version":"15'1","reported_seq":"11","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.827538","last_change":"2024-04-17 22:29:07.479214","last_active":"2024-04-17 22:29:07.827538","last_peered":"2024-04-17 22:29:07.827538","last_clean":"2024-04-17 22:29:07.827538","last_became_active":"2024-04-17 22:29:07.478790","last_became_peered":"2024-04-17 22:29:07.478790","last_unstale":"2024-04-17 22:29:07.827538","last_undegraded":"2024-04-17 22:29:07.827538","last_fullsized":"2024-04-17 22:29:07.827538","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.833543","last_change":"2024-04-17 22:29:07.832349","last_active":"2024-04-17 22:29:07.833543","last_peered":"2024-04-17 22:29:07.833543","last_clean":"2024-04-17 22:29:07.833543","last_became_active":"2024-04-17 22:29:07.831999","last_became_peered":"2024-04-17 22:29:07.831999","last_unstale":"2024-04-17 22:29:07.833543","last_undegraded":"2024-04-17 22:29:07.833543","last_fullsized":"2024-04-17 22:29:07.833543","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2e","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.797059","last_change":"2024-04-17 22:29:07.483244","last_active":"2024-04-17 22:29:07.797059","last_peered":"2024-04-17 22:29:07.797059","last_clean":"2024-04-17 22:29:07.797059","last_became_active":"2024-04-17 22:29:07.482893","last_became_peered":"2024-04-17 22:29:07.482893","last_unstale":"2024-04-17 22:29:07.797059","last_undegraded":"2024-04-17 22:29:07.797059","last_fullsized":"2024-04-17 22:29:07.797059","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.825028","last_change":"2024-04-17 22:29:07.806512","last_active":"2024-04-17 22:29:07.825028","last_peered":"2024-04-17 22:29:07.825028","last_clean":"2024-04-17 22:29:07.825028","last_became_active":"2024-04-17 22:29:07.806182","last_became_peered":"2024-04-17 22:29:07.806182","last_unstale":"2024-04-17 22:29:07.825028","last_undegraded":"2024-04-17 22:29:07.825028","last_fullsized":"2024-04-17 22:29:07.825028","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2f","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.815418","last_change":"2024-04-17 22:29:07.478655","last_active":"2024-04-17 22:29:07.815418","last_peered":"2024-04-17 22:29:07.815418","last_clean":"2024-04-17 22:29:07.815418","last_became_active":"2024-04-17 22:29:07.478346","last_became_peered":"2024-04-17 22:29:07.478346","last_unstale":"2024-04-17 22:29:07.815418","last_undegraded":"2024-04-17 22:29:07.815418","last_fullsized":"2024-04-17 22:29:07.815418","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.826848","last_change":"2024-04-17 22:29:07.825658","last_active":"2024-04-17 22:29:07.826848","last_peered":"2024-04-17 22:29:07.826848","last_clean":"2024-04-17 22:29:07.826848","last_became_active":"2024-04-17 22:29:07.825349","last_became_peered":"2024-04-17 22:29:07.825349","last_unstale":"2024-04-17 22:29:07.826848","last_undegraded":"2024-04-17 22:29:07.826848","last_fullsized":"2024-04-17 22:29:07.826848","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.30","version":"15'4","reported_seq":"14","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.824396","last_change":"2024-04-17 22:29:07.475470","last_active":"2024-04-17 22:29:07.824396","last_peered":"2024-04-17 22:29:07.824396","last_clean":"2024-04-17 22:29:07.824396","last_became_active":"2024-04-17 22:29:07.474917","last_became_peered":"2024-04-17 22:29:07.474917","last_unstale":"2024-04-17 22:29:07.824396","last_undegraded":"2024-04-17 22:29:07.824396","last_fullsized":"2024-04-17 22:29:07.824396","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.826262","last_change":"2024-04-17 22:29:07.823624","last_active":"2024-04-17 22:29:07.826262","last_peered":"2024-04-17 22:29:07.826262","last_clean":"2024-04-17 22:29:07.826262","last_became_active":"2024-04-17 22:29:07.822815","last_became_peered":"2024-04-17 22:29:07.822815","last_unstale":"2024-04-17 22:29:07.826262","last_undegraded":"2024-04-17 22:29:07.826262","last_fullsized":"2024-04-17 22:29:07.826262","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.31","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.804183","last_change":"2024-04-17 22:29:07.474318","last_active":"2024-04-17 22:29:07.804183","last_peered":"2024-04-17 22:29:07.804183","last_clean":"2024-04-17 22:29:07.804183","last_became_active":"2024-04-17 22:29:07.473754","last_became_peered":"2024-04-17 22:29:07.473754","last_unstale":"2024-04-17 22:29:07.804183","last_undegraded":"2024-04-17 22:29:07.804183","last_fullsized":"2024-04-17 22:29:07.804183","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.833495","last_change":"2024-04-17 22:29:07.829098","last_active":"2024-04-17 22:29:07.833495","last_peered":"2024-04-17 22:29:07.833495","last_clean":"2024-04-17 22:29:07.833495","last_became_active":"2024-04-17 22:29:07.828773","last_became_peered":"2024-04-17 22:29:07.828773","last_unstale":"2024-04-17 22:29:07.833495","last_undegraded":"2024-04-17 22:29:07.833495","last_fullsized":"2024-04-17 22:29:07.833495","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.32","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.795411","last_change":"2024-04-17 22:29:07.470666","last_active":"2024-04-17 22:29:07.795411","last_peered":"2024-04-17 22:29:07.795411","last_clean":"2024-04-17 22:29:07.795411","last_became_active":"2024-04-17 22:29:07.470363","last_became_peered":"2024-04-17 22:29:07.470363","last_unstale":"2024-04-17 22:29:07.795411","last_undegraded":"2024-04-17 22:29:07.795411","last_fullsized":"2024-04-17 22:29:07.795411","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.830343","last_change":"2024-04-17 22:29:07.827580","last_active":"2024-04-17 22:29:07.830343","last_peered":"2024-04-17 22:29:07.830343","last_clean":"2024-04-17 22:29:07.830343","last_became_active":"2024-04-17 22:29:07.827233","last_became_peered":"2024-04-17 22:29:07.827233","last_unstale":"2024-04-17 22:29:07.830343","last_undegraded":"2024-04-17 22:29:07.830343","last_fullsized":"2024-04-17 22:29:07.830343","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.33","version":"15'1","reported_seq":"11","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.811613","last_change":"2024-04-17 22:29:07.460978","last_active":"2024-04-17 22:29:07.811613","last_peered":"2024-04-17 22:29:07.811613","last_clean":"2024-04-17 22:29:07.811613","last_became_active":"2024-04-17 22:29:07.460659","last_became_peered":"2024-04-17 22:29:07.460659","last_unstale":"2024-04-17 22:29:07.811613","last_undegraded":"2024-04-17 22:29:07.811613","last_fullsized":"2024-04-17 22:29:07.811613","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.832128","last_change":"2024-04-17 22:29:07.816335","last_active":"2024-04-17 22:29:07.832128","last_peered":"2024-04-17 22:29:07.832128","last_clean":"2024-04-17 22:29:07.832128","last_became_active":"2024-04-17 22:29:07.816090","last_became_peered":"2024-04-17 22:29:07.816090","last_unstale":"2024-04-17 22:29:07.832128","last_undegraded":"2024-04-17 22:29:07.832128","last_fullsized":"2024-04-17 22:29:07.832128","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.34","version":"15'4","reported_seq":"14","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.835387","last_change":"2024-04-17 22:29:07.475702","last_active":"2024-04-17 22:29:07.835387","last_peered":"2024-04-17 22:29:07.835387","last_clean":"2024-04-17 22:29:07.835387","last_became_active":"2024-04-17 22:29:07.475395","last_became_peered":"2024-04-17 22:29:07.475395","last_unstale":"2024-04-17 22:29:07.835387","last_undegraded":"2024-04-17 22:29:07.835387","last_fullsized":"2024-04-17 22:29:07.835387","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":892,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.828199","last_change":"2024-04-17 22:29:07.818538","last_active":"2024-04-17 22:29:07.828199","last_peered":"2024-04-17 22:29:07.828199","last_clean":"2024-04-17 22:29:07.828199","last_became_active":"2024-04-17 22:29:07.817164","last_became_peered":"2024-04-17 22:29:07.817164","last_unstale":"2024-04-17 22:29:07.828199","last_undegraded":"2024-04-17 22:29:07.828199","last_fullsized":"2024-04-17 22:29:07.828199","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.35","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.797071","last_change":"2024-04-17 22:29:07.467651","last_active":"2024-04-17 22:29:07.797071","last_peered":"2024-04-17 22:29:07.797071","last_clean":"2024-04-17 22:29:07.797071","last_became_active":"2024-04-17 22:29:07.467327","last_became_peered":"2024-04-17 22:29:07.467327","last_unstale":"2024-04-17 22:29:07.797071","last_undegraded":"2024-04-17 22:29:07.797071","last_fullsized":"2024-04-17 22:29:07.797071","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.810303","last_change":"2024-04-17 22:29:07.809693","last_active":"2024-04-17 22:29:07.810303","last_peered":"2024-04-17 22:29:07.810303","last_clean":"2024-04-17 22:29:07.810303","last_became_active":"2024-04-17 22:29:07.809415","last_became_peered":"2024-04-17 22:29:07.809415","last_unstale":"2024-04-17 22:29:07.810303","last_undegraded":"2024-04-17 22:29:07.810303","last_fullsized":"2024-04-17 22:29:07.810303","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.36","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.796841","last_change":"2024-04-17 22:29:07.483402","last_active":"2024-04-17 22:29:07.796841","last_peered":"2024-04-17 22:29:07.796841","last_clean":"2024-04-17 22:29:07.796841","last_became_active":"2024-04-17 22:29:07.482914","last_became_peered":"2024-04-17 22:29:07.482914","last_unstale":"2024-04-17 22:29:07.796841","last_undegraded":"2024-04-17 22:29:07.796841","last_fullsized":"2024-04-17 22:29:07.796841","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.830317","last_change":"2024-04-17 22:29:07.827623","last_active":"2024-04-17 22:29:07.830317","last_peered":"2024-04-17 22:29:07.830317","last_clean":"2024-04-17 22:29:07.830317","last_became_active":"2024-04-17 22:29:07.827337","last_became_peered":"2024-04-17 22:29:07.827337","last_unstale":"2024-04-17 22:29:07.830317","last_undegraded":"2024-04-17 22:29:07.830317","last_fullsized":"2024-04-17 22:29:07.830317","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.37","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.808508","last_change":"2024-04-17 22:29:07.463449","last_active":"2024-04-17 22:29:07.808508","last_peered":"2024-04-17 22:29:07.808508","last_clean":"2024-04-17 22:29:07.808508","last_became_active":"2024-04-17 22:29:07.463110","last_became_peered":"2024-04-17 22:29:07.463110","last_unstale":"2024-04-17 22:29:07.808508","last_undegraded":"2024-04-17 22:29:07.808508","last_fullsized":"2024-04-17 22:29:07.808508","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.827399","last_change":"2024-04-17 22:29:07.815214","last_active":"2024-04-17 22:29:07.827399","last_peered":"2024-04-17 22:29:07.827399","last_clean":"2024-04-17 22:29:07.827399","last_became_active":"2024-04-17 22:29:07.814938","last_became_peered":"2024-04-17 22:29:07.814938","last_unstale":"2024-04-17 22:29:07.827399","last_undegraded":"2024-04-17 22:29:07.827399","last_fullsized":"2024-04-17 22:29:07.827399","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.38","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.801328","last_change":"2024-04-17 22:29:07.474682","last_active":"2024-04-17 22:29:07.801328","last_peered":"2024-04-17 22:29:07.801328","last_clean":"2024-04-17 22:29:07.801328","last_became_active":"2024-04-17 22:29:07.474305","last_became_peered":"2024-04-17 22:29:07.474305","last_unstale":"2024-04-17 22:29:07.801328","last_undegraded":"2024-04-17 22:29:07.801328","last_fullsized":"2024-04-17 22:29:07.801328","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.833512","last_change":"2024-04-17 22:29:07.829769","last_active":"2024-04-17 22:29:07.833512","last_peered":"2024-04-17 22:29:07.833512","last_clean":"2024-04-17 22:29:07.833512","last_became_active":"2024-04-17 22:29:07.829484","last_became_peered":"2024-04-17 22:29:07.829484","last_unstale":"2024-04-17 22:29:07.833512","last_undegraded":"2024-04-17 22:29:07.833512","last_fullsized":"2024-04-17 22:29:07.833512","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.39","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.794112","last_change":"2024-04-17 22:29:07.458386","last_active":"2024-04-17 22:29:07.794112","last_peered":"2024-04-17 22:29:07.794112","last_clean":"2024-04-17 22:29:07.794112","last_became_active":"2024-04-17 22:29:07.456695","last_became_peered":"2024-04-17 22:29:07.456695","last_unstale":"2024-04-17 22:29:07.794112","last_undegraded":"2024-04-17 22:29:07.794112","last_fullsized":"2024-04-17 22:29:07.794112","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.829903","last_change":"2024-04-17 22:29:07.821490","last_active":"2024-04-17 22:29:07.829903","last_peered":"2024-04-17 22:29:07.829903","last_clean":"2024-04-17 22:29:07.829903","last_became_active":"2024-04-17 22:29:07.821103","last_became_peered":"2024-04-17 22:29:07.821103","last_unstale":"2024-04-17 22:29:07.829903","last_undegraded":"2024-04-17 22:29:07.829903","last_fullsized":"2024-04-17 22:29:07.829903","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3a","version":"15'1","reported_seq":"11","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.819620","last_change":"2024-04-17 22:29:07.472610","last_active":"2024-04-17 22:29:07.819620","last_peered":"2024-04-17 22:29:07.819620","last_clean":"2024-04-17 22:29:07.819620","last_became_active":"2024-04-17 22:29:07.472297","last_became_peered":"2024-04-17 22:29:07.472297","last_unstale":"2024-04-17 22:29:07.819620","last_undegraded":"2024-04-17 22:29:07.819620","last_fullsized":"2024-04-17 22:29:07.819620","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.824029","last_change":"2024-04-17 22:29:07.820738","last_active":"2024-04-17 22:29:07.824029","last_peered":"2024-04-17 22:29:07.824029","last_clean":"2024-04-17 22:29:07.824029","last_became_active":"2024-04-17 22:29:07.820354","last_became_peered":"2024-04-17 22:29:07.820354","last_unstale":"2024-04-17 22:29:07.824029","last_undegraded":"2024-04-17 22:29:07.824029","last_fullsized":"2024-04-17 22:29:07.824029","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3b","version":"15'1","reported_seq":"11","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.824121","last_change":"2024-04-17 22:29:07.473963","last_active":"2024-04-17 22:29:07.824121","last_peered":"2024-04-17 22:29:07.824121","last_clean":"2024-04-17 22:29:07.824121","last_became_active":"2024-04-17 22:29:07.472018","last_became_peered":"2024-04-17 22:29:07.472018","last_unstale":"2024-04-17 22:29:07.824121","last_undegraded":"2024-04-17 22:29:07.824121","last_fullsized":"2024-04-17 22:29:07.824121","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.832188","last_change":"2024-04-17 22:29:07.824919","last_active":"2024-04-17 22:29:07.832188","last_peered":"2024-04-17 22:29:07.832188","last_clean":"2024-04-17 22:29:07.832188","last_became_active":"2024-04-17 22:29:07.824600","last_became_peered":"2024-04-17 22:29:07.824600","last_unstale":"2024-04-17 22:29:07.832188","last_undegraded":"2024-04-17 22:29:07.832188","last_fullsized":"2024-04-17 22:29:07.832188","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3c","version":"15'2","reported_seq":"12","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.825427","last_change":"2024-04-17 22:29:07.475485","last_active":"2024-04-17 22:29:07.825427","last_peered":"2024-04-17 22:29:07.825427","last_clean":"2024-04-17 22:29:07.825427","last_became_active":"2024-04-17 22:29:07.475182","last_became_peered":"2024-04-17 22:29:07.475182","last_unstale":"2024-04-17 22:29:07.825427","last_undegraded":"2024-04-17 22:29:07.825427","last_fullsized":"2024-04-17 22:29:07.825427","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.821750","last_change":"2024-04-17 22:29:07.810455","last_active":"2024-04-17 22:29:07.821750","last_peered":"2024-04-17 22:29:07.821750","last_clean":"2024-04-17 22:29:07.821750","last_became_active":"2024-04-17 22:29:07.810143","last_became_peered":"2024-04-17 22:29:07.810143","last_unstale":"2024-04-17 22:29:07.821750","last_undegraded":"2024-04-17 22:29:07.821750","last_fullsized":"2024-04-17 22:29:07.821750","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3d","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.797015","last_change":"2024-04-17 22:29:07.468187","last_active":"2024-04-17 22:29:07.797015","last_peered":"2024-04-17 22:29:07.797015","last_clean":"2024-04-17 22:29:07.797015","last_became_active":"2024-04-17 22:29:07.467849","last_became_peered":"2024-04-17 22:29:07.467849","last_unstale":"2024-04-17 22:29:07.797015","last_undegraded":"2024-04-17 22:29:07.797015","last_fullsized":"2024-04-17 22:29:07.797015","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.830571","last_change":"2024-04-17 22:29:07.822173","last_active":"2024-04-17 22:29:07.830571","last_peered":"2024-04-17 22:29:07.830571","last_clean":"2024-04-17 22:29:07.830571","last_became_active":"2024-04-17 22:29:07.821875","last_became_peered":"2024-04-17 22:29:07.821875","last_unstale":"2024-04-17 22:29:07.830571","last_undegraded":"2024-04-17 22:29:07.830571","last_fullsized":"2024-04-17 22:29:07.830571","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3e","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.807733","last_change":"2024-04-17 22:29:07.479547","last_active":"2024-04-17 22:29:07.807733","last_peered":"2024-04-17 22:29:07.807733","last_clean":"2024-04-17 22:29:07.807733","last_became_active":"2024-04-17 22:29:07.479004","last_became_peered":"2024-04-17 22:29:07.479004","last_unstale":"2024-04-17 22:29:07.807733","last_undegraded":"2024-04-17 22:29:07.807733","last_fullsized":"2024-04-17 22:29:07.807733","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.823835","last_change":"2024-04-17 22:29:07.819432","last_active":"2024-04-17 22:29:07.823835","last_peered":"2024-04-17 22:29:07.823835","last_clean":"2024-04-17 22:29:07.823835","last_became_active":"2024-04-17 22:29:07.819001","last_became_peered":"2024-04-17 22:29:07.819001","last_unstale":"2024-04-17 22:29:07.823835","last_undegraded":"2024-04-17 22:29:07.823835","last_fullsized":"2024-04-17 22:29:07.823835","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3f","version":"15'3","reported_seq":"13","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.829638","last_change":"2024-04-17 22:29:07.466765","last_active":"2024-04-17 22:29:07.829638","last_peered":"2024-04-17 22:29:07.829638","last_clean":"2024-04-17 22:29:07.829638","last_became_active":"2024-04-17 22:29:07.463177","last_became_peered":"2024-04-17 22:29:07.463177","last_unstale":"2024-04-17 22:29:07.829638","last_undegraded":"2024-04-17 22:29:07.829638","last_fullsized":"2024-04-17 22:29:07.829638","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":544,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":4},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":4},{"poolid":2,"num_pg":64,"stat_sum":{"num_bytes":2286,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":128,"acting":128,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705669,"num_pgs":82,"num_osds":1,"num_per_pool_osds":1,"kb":94371840,"kb_used":1048744,"kb_used_data":160,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323096,"statfs":{"total":96636764160,"available":95562850304,"internally_reserved":1073741824,"allocated":163840,"data_stored":41000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.19900000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27500000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.22900000000000001}]}]},{"osd":2,"up_from":9,"seq":38654705669,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"kb":94371840,"kb_used":1048732,"kb_used_data":148,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323108,"statfs":{"total":96636764160,"available":95562862592,"internally_reserved":1073741824,"allocated":151552,"data_stored":39984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.129}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.217}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.20100000000000001}]}]},{"osd":0,"up_from":9,"seq":38654705669,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"kb":94371840,"kb_used":1048736,"kb_used_data":152,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323104,"statfs":{"total":96636764160,"available":95562858496,"internally_reserved":1073741824,"allocated":155648,"data_stored":40772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29399999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.31}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.307}]}]},{"osd":1,"up_from":9,"seq":38654705668,"num_pgs":63,"num_osds":1,"num_per_pool_osds":1,"kb":94371840,"kb_used":1048732,"kb_used_data":148,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323108,"statfs":{"total":96636764160,"available":95562862592,"internally_reserved":1073741824,"allocated":151552,"data_stored":39936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27300000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.307}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33000000000000002}]}]}]}} 2024-04-17T22:29:11.297 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-17T22:29:11.457 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.469 7f92c66b5700 1 Processor -- start 2024-04-17T22:29:11.457 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.469 7f92c66b5700 1 -- start start 2024-04-17T22:29:11.457 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.469 7f92c66b5700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f92c010e260 0x7f92c010e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:11.457 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.469 7f92c66b5700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f92c010ece0 0x7f92c00f1750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:11.457 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.469 7f92c66b5700 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92c00f21d0 con 0x7f92c00f1d80 2024-04-17T22:29:11.457 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.469 7f92c66b5700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92c00f9f30 con 0x7f92c010ece0 2024-04-17T22:29:11.458 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.469 7f92c66b5700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f92c00f9050 con 0x7f92c010e260 2024-04-17T22:29:11.458 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.469 7f92b7fff700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f92c010ece0 0x7f92c00f1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:11.458 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.469 7f92bfbb1700 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f92c00f1d80 0x7f92c01aa4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:46520/0 (socket says 172.21.15.102:46520) 2024-04-17T22:29:11.458 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.469 7f92bfbb1700 1 -- 172.21.15.102:0/1207754633 learned_addr learned my addr 172.21.15.102:0/1207754633 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-17T22:29:11.458 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.469 7f92bf3b0700 1 --2- 172.21.15.102:0/1207754633 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f92c010e260 0x7f92c010e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:11.458 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.469 7f92bf3b0700 1 -- 172.21.15.102:0/1207754633 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f92c010ece0 msgr2=0x7f92c00f1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:29:11.458 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.469 7f92bebaf700 1 -- 172.21.15.102:0/1207754633 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3843225025 0 0) 0x7f92b8001490 con 0x7f92c00f1d80 2024-04-17T22:29:11.458 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.469 7f92bf3b0700 1 --2- 172.21.15.102:0/1207754633 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f92c010ece0 0x7f92c00f1750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:11.458 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.469 7f92bf3b0700 1 -- 172.21.15.102:0/1207754633 >> v1:172.21.15.102:6789/0 conn(0x7f92c00f1d80 legacy=0x7f92c01aa4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:11.458 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.469 7f92bf3b0700 1 -- 172.21.15.102:0/1207754633 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92c00f23f0 con 0x7f92c010e260 2024-04-17T22:29:11.458 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.470 7f92bf3b0700 1 --2- 172.21.15.102:0/1207754633 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f92c010e260 0x7f92c010e6b0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7f92b0002f70 tx=0x7f92b000b9a0).ready entity=mon.2 client_cookie=f17db1a4a04050e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:29:11.458 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.470 7f92bebaf700 1 -- 172.21.15.102:0/1207754633 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f92c00f23f0 con 0x7f92c010e260 2024-04-17T22:29:11.458 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.470 7f92bebaf700 1 -- 172.21.15.102:0/1207754633 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92b000d570 con 0x7f92c010e260 2024-04-17T22:29:11.459 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.470 7f92bebaf700 1 -- 172.21.15.102:0/1207754633 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f92b000d940 con 0x7f92c010e260 2024-04-17T22:29:11.459 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.470 7f92c66b5700 1 -- 172.21.15.102:0/1207754633 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f92c010e260 msgr2=0x7f92c010e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:11.459 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.470 7f92c66b5700 1 --2- 172.21.15.102:0/1207754633 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f92c010e260 0x7f92c010e6b0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7f92b0002f70 tx=0x7f92b000b9a0).stop 2024-04-17T22:29:11.459 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.470 7f92c66b5700 1 -- 172.21.15.102:0/1207754633 shutdown_connections 2024-04-17T22:29:11.459 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.470 7f92c66b5700 1 --2- 172.21.15.102:0/1207754633 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f92c010e260 0x7f92c010e6b0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:11.459 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.470 7f92c66b5700 1 --2- 172.21.15.102:0/1207754633 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f92c010ece0 0x7f92c00f1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:11.459 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.470 7f92c66b5700 1 -- 172.21.15.102:0/1207754633 >> 172.21.15.102:0/1207754633 conn(0x7f92c01138c0 msgr2=0x7f92c00f4f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:29:11.459 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.470 7f92c66b5700 1 -- 172.21.15.102:0/1207754633 shutdown_connections 2024-04-17T22:29:11.459 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.470 7f92c66b5700 1 -- 172.21.15.102:0/1207754633 wait complete. 2024-04-17T22:29:11.481 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.493 7f92c66b5700 1 Processor -- start 2024-04-17T22:29:11.482 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.493 7f92c66b5700 1 -- start start 2024-04-17T22:29:11.482 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.493 7f92c66b5700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f92c00dd4b0 0x7f92c00d5e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:11.482 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.493 7f92c66b5700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f92c00d64c0 0x7f92c01b3e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:11.482 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.493 7f92c66b5700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f92c00d6910 0x7f92c01b44b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:11.482 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.493 7f92c66b5700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92c00fa1d0 con 0x7f92c00dd4b0 2024-04-17T22:29:11.482 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.493 7f92c66b5700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92c00d8810 con 0x7f92c00d6910 2024-04-17T22:29:11.482 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.493 7f92c66b5700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f92c00ddc00 con 0x7f92c00d64c0 2024-04-17T22:29:11.482 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.493 7f92bf3b0700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f92c00dd4b0 0x7f92c00d5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:11.482 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.493 7f92b7fff700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f92c00d64c0 0x7f92c01b3e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:11.482 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.493 7f92bfbb1700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f92c00d6910 0x7f92c01b44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:11.482 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.493 7f92bf3b0700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f92c00dd4b0 0x7f92c00d5e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:49318/0 (socket says 172.21.15.102:49318) 2024-04-17T22:29:11.482 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.493 7f92bf3b0700 1 -- 172.21.15.102:0/1669969021 learned_addr learned my addr 172.21.15.102:0/1669969021 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:29:11.483 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.494 7f92bf3b0700 1 -- 172.21.15.102:0/1669969021 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f92c00d64c0 msgr2=0x7f92c01b3e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:11.483 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.494 7f92bf3b0700 1 --2- 172.21.15.102:0/1669969021 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f92c00d64c0 0x7f92c01b3e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:11.483 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.494 7f92bf3b0700 1 -- 172.21.15.102:0/1669969021 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f92c00d6910 msgr2=0x7f92c01b44b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:11.483 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.494 7f92bf3b0700 1 --2- 172.21.15.102:0/1669969021 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f92c00d6910 0x7f92c01b44b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:11.483 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.494 7f92bf3b0700 1 -- 172.21.15.102:0/1669969021 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92c01b4ae0 con 0x7f92c00dd4b0 2024-04-17T22:29:11.483 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.494 7f92bfbb1700 1 --2- 172.21.15.102:0/1669969021 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f92c00d6910 0x7f92c01b44b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:29:11.483 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.494 7f92b7fff700 1 --2- 172.21.15.102:0/1669969021 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f92c00d64c0 0x7f92c01b3e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:29:11.483 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.494 7f92bf3b0700 1 --2- 172.21.15.102:0/1669969021 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f92c00dd4b0 0x7f92c00d5e90 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7f92c00d71f0 tx=0x7f92b00068c0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:29:11.483 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.494 7f92bd3ac700 1 -- 172.21.15.102:0/1669969021 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f92c01b4ae0 con 0x7f92c00dd4b0 2024-04-17T22:29:11.483 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.494 7f92bd3ac700 1 -- 172.21.15.102:0/1669969021 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92b0011980 con 0x7f92c00dd4b0 2024-04-17T22:29:11.483 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.494 7f92bd3ac700 1 -- 172.21.15.102:0/1669969021 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f92b0017040 con 0x7f92c00dd4b0 2024-04-17T22:29:11.483 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.494 7f92c66b5700 1 -- 172.21.15.102:0/1669969021 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92c01b4da0 con 0x7f92c00dd4b0 2024-04-17T22:29:11.483 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.494 7f92c66b5700 1 -- 172.21.15.102:0/1669969021 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92c01bb750 con 0x7f92c00dd4b0 2024-04-17T22:29:11.483 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.495 7f92bd3ac700 1 -- 172.21.15.102:0/1669969021 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7f92b00273b0 con 0x7f92c00dd4b0 2024-04-17T22:29:11.483 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.495 7f92bd3ac700 1 -- 172.21.15.102:0/1669969021 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 3194+0+0 (secure 0 0 0) 0x7f92b0018070 con 0x7f92c00dd4b0 2024-04-17T22:29:11.484 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.495 7f92c66b5700 1 -- 172.21.15.102:0/1669969021 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92c00f0820 con 0x7f92c00dd4b0 2024-04-17T22:29:11.485 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.498 7f92bd3ac700 1 -- 172.21.15.102:0/1669969021 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+110455 (secure 0 0 0) 0x7f92b0011770 con 0x7f92c00dd4b0 2024-04-17T22:29:11.621 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.635 7f92c66b5700 1 -- 172.21.15.102:0/1669969021 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f92c01ae5a0 con 0x7f92c00dd4b0 2024-04-17T22:29:11.622 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.636 7f92bd3ac700 1 -- 172.21.15.102:0/1669969021 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+32989 (secure 0 0 0) 0x7f92b000d360 con 0x7f92c00dd4b0 2024-04-17T22:29:11.623 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-17T22:29:11.623 INFO:teuthology.orchestra.run.smithi102.stdout:{"fsid":"cb63e789-091d-49b1-b120-55f9db526871","health":{"checks":{},"status":"HEALTH_OK"},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":23,"monmap":{"epoch":1,"min_mon_release_name":"14","num_mons":3},"osdmap":{"osdmap":{"epoch":16,"num_osds":4,"num_up_osds":4,"num_in_osds":4,"num_remapped_pgs":0}},"pgmap":{"pgs_by_state":[{"state_name":"unknown","count":128},{"state_name":"active+clean","count":8}],"num_pgs":136,"num_pools":3,"num_objects":0,"data_bytes":0,"bytes_used":4295426048,"bytes_avail":382251630592,"bytes_total":386547056640,"unknown_pgs_ratio":0.94117647409439087},"fsmap":{"epoch":5,"id":1,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":1,"rank":0,"name":"a","status":"up:active","gid":4177}],"up:standby":0},"mgrmap":{"epoch":3,"active_gid":4104,"active_name":"x","active_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6816","nonce":42360}]},"active_addr":"172.21.15.102:6816/42360","active_change":"2024-04-17 22:28:51.181757","available":true,"standbys":[],"modules":["iostat","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"ansible","can_run":true,"error_string":"","module_options":{"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_url":{"name":"server_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"verify_server":{"name":"verify_server","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"none","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_iterations":{"name":"upmap_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimization iterations","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_cephfs":{"name":"FEATURE_TOGGLE_cephfs","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_iscsi":{"name":"FEATURE_TOGGLE_iscsi","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_mirroring":{"name":"FEATURE_TOGGLE_mirroring","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_rbd":{"name":"FEATURE_TOGGLE_rbd","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_rgw":{"name":"FEATURE_TOGGLE_rgw","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_HOST":{"name":"RGW_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_PORT":{"name":"RGW_API_PORT","type":"int","level":"advanced","flags":0,"default_value":"80","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SCHEME":{"name":"RGW_API_SCHEME","type":"str","level":"advanced","flags":0,"default_value":"http","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_USER_ID":{"name":"RGW_API_USER_ID","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"deepsea","can_run":true,"error_string":"","module_options":{"salt_api_eauth":{"name":"salt_api_eauth","type":"str","level":"advanced","flags":0,"default_value":"sharedsecret","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"salt_api_password":{"name":"salt_api_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"salt_api_url":{"name":"salt_api_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"salt_api_username":{"name":"salt_api_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"str","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"str","level":"advanced","flags":0,"default_value":"30","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"str","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"str","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{}},{"name":"iostat","can_run":true,"error_string":"","module_options":{}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator_cli","can_run":true,"error_string":"","module_options":{"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"persist_interval":{"name":"persist_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how frequently to persist completed events","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{}},{"name":"restful","can_run":true,"error_string":"","module_options":{"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"nautilus":["balancer","crash","devicehealth","orchestrator_cli","progress","rbd_support","status","volumes"]}},"servicemap":{"epoch":1,"modified":"0.000000","services":{}},"progress_events":{}} 2024-04-17T22:29:11.624 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.637 7f92c66b5700 1 -- 172.21.15.102:0/1669969021 >> v1:172.21.15.102:6816/42360 conn(0x7f92a40145a0 legacy=0x7f92a4016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:11.624 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.637 7f92c66b5700 1 -- 172.21.15.102:0/1669969021 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f92c00dd4b0 msgr2=0x7f92c00d5e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:11.624 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.637 7f92c66b5700 1 --2- 172.21.15.102:0/1669969021 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f92c00dd4b0 0x7f92c00d5e90 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7f92c00d71f0 tx=0x7f92b00068c0).stop 2024-04-17T22:29:11.624 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.637 7f92c66b5700 1 -- 172.21.15.102:0/1669969021 shutdown_connections 2024-04-17T22:29:11.624 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.637 7f92c66b5700 1 --2- 172.21.15.102:0/1669969021 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f92c00d6910 0x7f92c01b44b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:11.625 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.637 7f92c66b5700 1 --2- 172.21.15.102:0/1669969021 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f92c00dd4b0 0x7f92c00d5e90 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:11.625 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.637 7f92c66b5700 1 --2- 172.21.15.102:0/1669969021 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f92c00d64c0 0x7f92c01b3e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:11.625 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.637 7f92c66b5700 1 -- 172.21.15.102:0/1669969021 >> 172.21.15.102:0/1669969021 conn(0x7f92c00c54b0 msgr2=0x7f92c00cf380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:29:11.625 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.637 7f92c66b5700 1 -- 172.21.15.102:0/1669969021 shutdown_connections 2024-04-17T22:29:11.625 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.637 7f92c66b5700 1 -- 172.21.15.102:0/1669969021 wait complete. 2024-04-17T22:29:11.635 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-04-17T22:29:11.636 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-17T22:29:11.762 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.773 7fa92a0ff700 1 Processor -- start 2024-04-17T22:29:11.762 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.773 7fa92a0ff700 1 -- start start 2024-04-17T22:29:11.762 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.774 7fa92a0ff700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fa92410e1d0 0x7fa92410e620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:11.762 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.774 7fa92a0ff700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fa92410ec50 0x7fa9240f16c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:11.762 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.774 7fa92a0ff700 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9240f2140 con 0x7fa9240f1cf0 2024-04-17T22:29:11.762 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.774 7fa92a0ff700 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9240f9ea0 con 0x7fa92410e1d0 2024-04-17T22:29:11.763 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.774 7fa92a0ff700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa9240f8fc0 con 0x7fa92410ec50 2024-04-17T22:29:11.763 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.774 7fa91bfff700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fa92410ec50 0x7fa9240f16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:11.763 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.774 7fa91bfff700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fa92410ec50 0x7fa9240f16c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:41070/0 (socket says 172.21.15.102:41070) 2024-04-17T22:29:11.763 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.774 7fa91bfff700 1 -- 172.21.15.102:0/2378703366 learned_addr learned my addr 172.21.15.102:0/2378703366 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:29:11.763 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.774 7fa922dc6700 1 --2- 172.21.15.102:0/2378703366 >> v2:172.21.15.102:3300/0 conn(0x7fa92410e1d0 0x7fa92410e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:11.763 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.774 7fa922dc6700 1 -- 172.21.15.102:0/2378703366 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fa92410ec50 msgr2=0x7fa9240f16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:29:11.763 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.774 7fa922dc6700 1 --2- 172.21.15.102:0/2378703366 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fa92410ec50 0x7fa9240f16c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:11.763 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.774 7fa922dc6700 1 -- 172.21.15.102:0/2378703366 >> v1:172.21.15.102:6789/0 conn(0x7fa9240f1cf0 legacy=0x7fa9241aa460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:11.763 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.774 7fa9225c5700 1 -- 172.21.15.102:0/2378703366 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1899287908 0 0) 0x7fa91c001490 con 0x7fa9240f1cf0 2024-04-17T22:29:11.763 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.774 7fa922dc6700 1 -- 172.21.15.102:0/2378703366 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9240f2360 con 0x7fa92410e1d0 2024-04-17T22:29:11.763 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.774 7fa91bfff700 1 --2- 172.21.15.102:0/2378703366 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fa92410ec50 0x7fa9240f16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:29:11.763 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.774 7fa922dc6700 1 --2- 172.21.15.102:0/2378703366 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa92410e1d0 0x7fa92410e620 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0x7fa914002f70 tx=0x7fa91400b9f0).ready entity=mon.0 client_cookie=93ec0d8a8d2c2ccf server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:29:11.763 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.774 7fa9225c5700 1 -- 172.21.15.102:0/2378703366 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa9240f2360 con 0x7fa92410e1d0 2024-04-17T22:29:11.763 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.774 7fa9225c5700 1 -- 172.21.15.102:0/2378703366 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa91400d610 con 0x7fa92410e1d0 2024-04-17T22:29:11.764 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.774 7fa92a0ff700 1 -- 172.21.15.102:0/2378703366 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa92410e1d0 msgr2=0x7fa92410e620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:11.764 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.774 7fa9225c5700 1 -- 172.21.15.102:0/2378703366 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa91400d9e0 con 0x7fa92410e1d0 2024-04-17T22:29:11.764 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.774 7fa92a0ff700 1 --2- 172.21.15.102:0/2378703366 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa92410e1d0 0x7fa92410e620 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0x7fa914002f70 tx=0x7fa91400b9f0).stop 2024-04-17T22:29:11.764 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.775 7fa92a0ff700 1 -- 172.21.15.102:0/2378703366 shutdown_connections 2024-04-17T22:29:11.764 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.775 7fa92a0ff700 1 --2- 172.21.15.102:0/2378703366 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa92410e1d0 0x7fa92410e620 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:11.764 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.775 7fa92a0ff700 1 --2- 172.21.15.102:0/2378703366 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fa92410ec50 0x7fa9240f16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:11.764 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.775 7fa92a0ff700 1 -- 172.21.15.102:0/2378703366 >> 172.21.15.102:0/2378703366 conn(0x7fa924113830 msgr2=0x7fa9240f4ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:29:11.764 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.775 7fa92a0ff700 1 -- 172.21.15.102:0/2378703366 shutdown_connections 2024-04-17T22:29:11.764 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.775 7fa92a0ff700 1 -- 172.21.15.102:0/2378703366 wait complete. 2024-04-17T22:29:11.786 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.798 7fa92a0ff700 1 Processor -- start 2024-04-17T22:29:11.786 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.798 7fa92a0ff700 1 -- start start 2024-04-17T22:29:11.786 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.798 7fa92a0ff700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fa9240d5090 0x7fa9240d54e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:11.786 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.798 7fa92a0ff700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa9240d5b10 0x7fa9240cba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:11.786 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.798 7fa92a0ff700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa9240cc0c0 0x7fa9241bb8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:11.786 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.798 7fa92a0ff700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa924113a40 con 0x7fa9240d5b10 2024-04-17T22:29:11.786 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.798 7fa92a0ff700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9240fa140 con 0x7fa9240cc0c0 2024-04-17T22:29:11.787 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.798 7fa92a0ff700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa9240ccb70 con 0x7fa9240d5090 2024-04-17T22:29:11.787 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.798 7fa922dc6700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fa9240d5090 0x7fa9240d54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:11.787 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.798 7fa922dc6700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fa9240d5090 0x7fa9240d54e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:41072/0 (socket says 172.21.15.102:41072) 2024-04-17T22:29:11.787 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.798 7fa922dc6700 1 -- 172.21.15.102:0/2008626272 learned_addr learned my addr 172.21.15.102:0/2008626272 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:29:11.787 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.798 7fa91bfff700 1 --2- 172.21.15.102:0/2008626272 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa9240d5b10 0x7fa9240cba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:11.787 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.798 7fa9235c7700 1 --2- 172.21.15.102:0/2008626272 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa9240cc0c0 0x7fa9241bb8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:11.787 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.798 7fa91bfff700 1 -- 172.21.15.102:0/2008626272 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fa9240d5090 msgr2=0x7fa9240d54e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:11.787 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.798 7fa91bfff700 1 --2- 172.21.15.102:0/2008626272 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fa9240d5090 0x7fa9240d54e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:11.787 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.798 7fa91bfff700 1 -- 172.21.15.102:0/2008626272 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa9240cc0c0 msgr2=0x7fa9241bb8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:11.787 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.798 7fa91bfff700 1 --2- 172.21.15.102:0/2008626272 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa9240cc0c0 0x7fa9241bb8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:11.787 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.798 7fa91bfff700 1 -- 172.21.15.102:0/2008626272 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9240cda80 con 0x7fa9240d5b10 2024-04-17T22:29:11.787 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.798 7fa922dc6700 1 --2- 172.21.15.102:0/2008626272 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fa9240d5090 0x7fa9240d54e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:29:11.787 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.798 7fa91bfff700 1 --2- 172.21.15.102:0/2008626272 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa9240d5b10 0x7fa9240cba90 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0x7fa9241b4230 tx=0x7fa90c00eb00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:29:11.787 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.799 7fa920dc2700 1 -- 172.21.15.102:0/2008626272 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa9240cda80 con 0x7fa9240d5b10 2024-04-17T22:29:11.788 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.799 7fa920dc2700 1 -- 172.21.15.102:0/2008626272 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa90c012600 con 0x7fa9240d5b10 2024-04-17T22:29:11.788 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.799 7fa920dc2700 1 -- 172.21.15.102:0/2008626272 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa90c00f040 con 0x7fa9240d5b10 2024-04-17T22:29:11.788 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.799 7fa92a0ff700 1 -- 172.21.15.102:0/2008626272 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9240cf710 con 0x7fa9240d5b10 2024-04-17T22:29:11.788 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.799 7fa92a0ff700 1 -- 172.21.15.102:0/2008626272 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9240cc510 con 0x7fa9240d5b10 2024-04-17T22:29:11.788 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.800 7fa92a0ff700 1 -- 172.21.15.102:0/2008626272 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9240f0790 con 0x7fa9240d5b10 2024-04-17T22:29:11.788 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.801 7fa920dc2700 1 -- 172.21.15.102:0/2008626272 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7fa90c022060 con 0x7fa9240d5b10 2024-04-17T22:29:11.788 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.801 7fa920dc2700 1 -- 172.21.15.102:0/2008626272 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 3194+0+0 (secure 0 0 0) 0x7fa90c010040 con 0x7fa9240d5b10 2024-04-17T22:29:11.790 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.804 7fa920dc2700 1 -- 172.21.15.102:0/2008626272 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+110455 (secure 0 0 0) 0x7fa90c0123f0 con 0x7fa9240d5b10 2024-04-17T22:29:11.892 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.905 7fa92a0ff700 1 -- 172.21.15.102:0/2008626272 --> v1:172.21.15.102:6816/42360 -- command(tid 0: {"prefix": "pg dump", "target": ["mgr", ""], "format": "json"}) v1 -- 0x7fa92408a300 con 0x7fa9080145a0 2024-04-17T22:29:11.899 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.913 7fa920dc2700 1 -- 172.21.15.102:0/2008626272 <== mgr.4104 v1:172.21.15.102:6816/42360 1 ==== command_reply(tid 0: 0 dumped all) v1 ==== 18+0+306111 (unknown 2965378022 0 992435385) 0x7fa914006b30 con 0x7fa9080145a0 2024-04-17T22:29:11.903 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-17T22:29:11.906 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.915 7fa92a0ff700 1 -- 172.21.15.102:0/2008626272 >> v1:172.21.15.102:6816/42360 conn(0x7fa9080145a0 legacy=0x7fa908016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:11.906 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.915 7fa92a0ff700 1 -- 172.21.15.102:0/2008626272 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa9240d5b10 msgr2=0x7fa9240cba90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:11.906 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.915 7fa92a0ff700 1 --2- 172.21.15.102:0/2008626272 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa9240d5b10 0x7fa9240cba90 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0x7fa9241b4230 tx=0x7fa90c00eb00).stop 2024-04-17T22:29:11.906 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.915 7fa92a0ff700 1 -- 172.21.15.102:0/2008626272 shutdown_connections 2024-04-17T22:29:11.906 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.915 7fa92a0ff700 1 --2- 172.21.15.102:0/2008626272 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa9240cc0c0 0x7fa9241bb8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:11.906 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.915 7fa92a0ff700 1 --2- 172.21.15.102:0/2008626272 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fa9240d5090 0x7fa9240d54e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:11.906 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.915 7fa92a0ff700 1 --2- 172.21.15.102:0/2008626272 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa9240d5b10 0x7fa9240cba90 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:11.906 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.915 7fa92a0ff700 1 -- 172.21.15.102:0/2008626272 >> 172.21.15.102:0/2008626272 conn(0x7fa9240c6a60 msgr2=0x7fa9240c7290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:29:11.906 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.916 7fa92a0ff700 1 -- 172.21.15.102:0/2008626272 shutdown_connections 2024-04-17T22:29:11.906 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:11.916 7fa92a0ff700 1 -- 172.21.15.102:0/2008626272 wait complete. 2024-04-17T22:29:11.906 INFO:teuthology.orchestra.run.smithi102.stderr:dumped all 2024-04-17T22:29:11.913 INFO:teuthology.orchestra.run.smithi102.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2024-04-17 22:29:11.187563","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2286,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":272,"acting":272,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":152,"num_osds":4,"num_per_pool_osds":4,"kb":377487360,"kb_used":4194944,"kb_used_data":608,"kb_used_omap":0,"kb_used_meta":4194304,"kb_avail":373292416,"statfs":{"total":386547056640,"available":382251433984,"internally_reserved":4294967296,"allocated":622592,"data_stored":161692,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4294967296},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"3.1a","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.820762","last_change":"2024-04-17 22:29:07.814049","last_active":"2024-04-17 22:29:07.820762","last_peered":"2024-04-17 22:29:07.820762","last_clean":"2024-04-17 22:29:07.820762","last_became_active":"2024-04-17 22:29:07.811049","last_became_peered":"2024-04-17 22:29:07.811049","last_unstale":"2024-04-17 22:29:07.820762","last_undegraded":"2024-04-17 22:29:07.820762","last_fullsized":"2024-04-17 22:29:07.820762","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.807568","last_change":"2024-04-17 22:29:07.477053","last_active":"2024-04-17 22:29:07.807568","last_peered":"2024-04-17 22:29:07.807568","last_clean":"2024-04-17 22:29:07.807568","last_became_active":"2024-04-17 22:29:07.476568","last_became_peered":"2024-04-17 22:29:07.476568","last_unstale":"2024-04-17 22:29:07.807568","last_undegraded":"2024-04-17 22:29:07.807568","last_fullsized":"2024-04-17 22:29:07.807568","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.828628","last_change":"2024-04-17 22:29:07.818787","last_active":"2024-04-17 22:29:07.828628","last_peered":"2024-04-17 22:29:07.828628","last_clean":"2024-04-17 22:29:07.828628","last_became_active":"2024-04-17 22:29:07.818347","last_became_peered":"2024-04-17 22:29:07.818347","last_unstale":"2024-04-17 22:29:07.828628","last_undegraded":"2024-04-17 22:29:07.828628","last_fullsized":"2024-04-17 22:29:07.828628","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.805356","last_change":"2024-04-17 22:29:07.477933","last_active":"2024-04-17 22:29:07.805356","last_peered":"2024-04-17 22:29:07.805356","last_clean":"2024-04-17 22:29:07.805356","last_became_active":"2024-04-17 22:29:07.472128","last_became_peered":"2024-04-17 22:29:07.472128","last_unstale":"2024-04-17 22:29:07.805356","last_undegraded":"2024-04-17 22:29:07.805356","last_fullsized":"2024-04-17 22:29:07.805356","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.834368","last_change":"2024-04-17 22:29:07.833107","last_active":"2024-04-17 22:29:07.834368","last_peered":"2024-04-17 22:29:07.834368","last_clean":"2024-04-17 22:29:07.834368","last_became_active":"2024-04-17 22:29:07.832822","last_became_peered":"2024-04-17 22:29:07.832822","last_unstale":"2024-04-17 22:29:07.834368","last_undegraded":"2024-04-17 22:29:07.834368","last_fullsized":"2024-04-17 22:29:07.834368","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.802997","last_change":"2024-04-17 22:29:07.471612","last_active":"2024-04-17 22:29:07.802997","last_peered":"2024-04-17 22:29:07.802997","last_clean":"2024-04-17 22:29:07.802997","last_became_active":"2024-04-17 22:29:07.471322","last_became_peered":"2024-04-17 22:29:07.471322","last_unstale":"2024-04-17 22:29:07.802997","last_undegraded":"2024-04-17 22:29:07.802997","last_fullsized":"2024-04-17 22:29:07.802997","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.812548","last_change":"2024-04-17 22:29:07.810081","last_active":"2024-04-17 22:29:07.812548","last_peered":"2024-04-17 22:29:07.812548","last_clean":"2024-04-17 22:29:07.812548","last_became_active":"2024-04-17 22:29:07.809858","last_became_peered":"2024-04-17 22:29:07.809858","last_unstale":"2024-04-17 22:29:07.812548","last_undegraded":"2024-04-17 22:29:07.812548","last_fullsized":"2024-04-17 22:29:07.812548","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.797655","last_change":"2024-04-17 22:29:07.474346","last_active":"2024-04-17 22:29:07.797655","last_peered":"2024-04-17 22:29:07.797655","last_clean":"2024-04-17 22:29:07.797655","last_became_active":"2024-04-17 22:29:07.473783","last_became_peered":"2024-04-17 22:29:07.473783","last_unstale":"2024-04-17 22:29:07.797655","last_undegraded":"2024-04-17 22:29:07.797655","last_fullsized":"2024-04-17 22:29:07.797655","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.825049","last_change":"2024-04-17 22:29:07.811047","last_active":"2024-04-17 22:29:07.825049","last_peered":"2024-04-17 22:29:07.825049","last_clean":"2024-04-17 22:29:07.825049","last_became_active":"2024-04-17 22:29:07.810747","last_became_peered":"2024-04-17 22:29:07.810747","last_unstale":"2024-04-17 22:29:07.825049","last_undegraded":"2024-04-17 22:29:07.825049","last_fullsized":"2024-04-17 22:29:07.825049","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.798589","last_change":"2024-04-17 22:29:07.460228","last_active":"2024-04-17 22:29:07.798589","last_peered":"2024-04-17 22:29:07.798589","last_clean":"2024-04-17 22:29:07.798589","last_became_active":"2024-04-17 22:29:07.459910","last_became_peered":"2024-04-17 22:29:07.459910","last_unstale":"2024-04-17 22:29:07.798589","last_undegraded":"2024-04-17 22:29:07.798589","last_fullsized":"2024-04-17 22:29:07.798589","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.830592","last_change":"2024-04-17 22:29:07.827143","last_active":"2024-04-17 22:29:07.830592","last_peered":"2024-04-17 22:29:07.830592","last_clean":"2024-04-17 22:29:07.830592","last_became_active":"2024-04-17 22:29:07.826842","last_became_peered":"2024-04-17 22:29:07.826842","last_unstale":"2024-04-17 22:29:07.830592","last_undegraded":"2024-04-17 22:29:07.830592","last_fullsized":"2024-04-17 22:29:07.830592","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.807458","last_change":"2024-04-17 22:29:07.477248","last_active":"2024-04-17 22:29:07.807458","last_peered":"2024-04-17 22:29:07.807458","last_clean":"2024-04-17 22:29:07.807458","last_became_active":"2024-04-17 22:29:07.476944","last_became_peered":"2024-04-17 22:29:07.476944","last_unstale":"2024-04-17 22:29:07.807458","last_undegraded":"2024-04-17 22:29:07.807458","last_fullsized":"2024-04-17 22:29:07.807458","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.826061","last_change":"2024-04-17 22:29:07.822961","last_active":"2024-04-17 22:29:07.826061","last_peered":"2024-04-17 22:29:07.826061","last_clean":"2024-04-17 22:29:07.826061","last_became_active":"2024-04-17 22:29:07.822592","last_became_peered":"2024-04-17 22:29:07.822592","last_unstale":"2024-04-17 22:29:07.826061","last_undegraded":"2024-04-17 22:29:07.826061","last_fullsized":"2024-04-17 22:29:07.826061","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.15","version":"15'3","reported_seq":"13","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.826924","last_change":"2024-04-17 22:29:07.485166","last_active":"2024-04-17 22:29:07.826924","last_peered":"2024-04-17 22:29:07.826924","last_clean":"2024-04-17 22:29:07.826924","last_became_active":"2024-04-17 22:29:07.484870","last_became_peered":"2024-04-17 22:29:07.484870","last_unstale":"2024-04-17 22:29:07.826924","last_undegraded":"2024-04-17 22:29:07.826924","last_fullsized":"2024-04-17 22:29:07.826924","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.825665","last_change":"2024-04-17 22:29:07.814509","last_active":"2024-04-17 22:29:07.825665","last_peered":"2024-04-17 22:29:07.825665","last_clean":"2024-04-17 22:29:07.825665","last_became_active":"2024-04-17 22:29:07.814119","last_became_peered":"2024-04-17 22:29:07.814119","last_unstale":"2024-04-17 22:29:07.825665","last_undegraded":"2024-04-17 22:29:07.825665","last_fullsized":"2024-04-17 22:29:07.825665","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.14","version":"15'2","reported_seq":"12","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.823699","last_change":"2024-04-17 22:29:07.473309","last_active":"2024-04-17 22:29:07.823699","last_peered":"2024-04-17 22:29:07.823699","last_clean":"2024-04-17 22:29:07.823699","last_became_active":"2024-04-17 22:29:07.472976","last_became_peered":"2024-04-17 22:29:07.472976","last_unstale":"2024-04-17 22:29:07.823699","last_undegraded":"2024-04-17 22:29:07.823699","last_fullsized":"2024-04-17 22:29:07.823699","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":634,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.832027","last_change":"2024-04-17 22:29:07.827915","last_active":"2024-04-17 22:29:07.832027","last_peered":"2024-04-17 22:29:07.832027","last_clean":"2024-04-17 22:29:07.832027","last_became_active":"2024-04-17 22:29:07.827670","last_became_peered":"2024-04-17 22:29:07.827670","last_unstale":"2024-04-17 22:29:07.832027","last_undegraded":"2024-04-17 22:29:07.832027","last_fullsized":"2024-04-17 22:29:07.832027","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"15'1","reported_seq":"11","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.827199","last_change":"2024-04-17 22:29:07.471458","last_active":"2024-04-17 22:29:07.827199","last_peered":"2024-04-17 22:29:07.827199","last_clean":"2024-04-17 22:29:07.827199","last_became_active":"2024-04-17 22:29:07.471085","last_became_peered":"2024-04-17 22:29:07.471085","last_unstale":"2024-04-17 22:29:07.827199","last_undegraded":"2024-04-17 22:29:07.827199","last_fullsized":"2024-04-17 22:29:07.827199","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.833535","last_change":"2024-04-17 22:29:07.829288","last_active":"2024-04-17 22:29:07.833535","last_peered":"2024-04-17 22:29:07.833535","last_clean":"2024-04-17 22:29:07.833535","last_became_active":"2024-04-17 22:29:07.828994","last_became_peered":"2024-04-17 22:29:07.828994","last_unstale":"2024-04-17 22:29:07.833535","last_undegraded":"2024-04-17 22:29:07.833535","last_fullsized":"2024-04-17 22:29:07.833535","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.805144","last_change":"2024-04-17 22:29:07.480194","last_active":"2024-04-17 22:29:07.805144","last_peered":"2024-04-17 22:29:07.805144","last_clean":"2024-04-17 22:29:07.805144","last_became_active":"2024-04-17 22:29:07.479728","last_became_peered":"2024-04-17 22:29:07.479728","last_unstale":"2024-04-17 22:29:07.805144","last_undegraded":"2024-04-17 22:29:07.805144","last_fullsized":"2024-04-17 22:29:07.805144","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.820915","last_change":"2024-04-17 22:29:07.815790","last_active":"2024-04-17 22:29:07.820915","last_peered":"2024-04-17 22:29:07.820915","last_clean":"2024-04-17 22:29:07.820915","last_became_active":"2024-04-17 22:29:07.815632","last_became_peered":"2024-04-17 22:29:07.815632","last_unstale":"2024-04-17 22:29:07.820915","last_undegraded":"2024-04-17 22:29:07.820915","last_fullsized":"2024-04-17 22:29:07.820915","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.804035","last_change":"2024-04-17 22:29:07.475830","last_active":"2024-04-17 22:29:07.804035","last_peered":"2024-04-17 22:29:07.804035","last_clean":"2024-04-17 22:29:07.804035","last_became_active":"2024-04-17 22:29:07.474411","last_became_peered":"2024-04-17 22:29:07.474411","last_unstale":"2024-04-17 22:29:07.804035","last_undegraded":"2024-04-17 22:29:07.804035","last_fullsized":"2024-04-17 22:29:07.804035","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.828447","last_change":"2024-04-17 22:29:07.826101","last_active":"2024-04-17 22:29:07.828447","last_peered":"2024-04-17 22:29:07.828447","last_clean":"2024-04-17 22:29:07.828447","last_became_active":"2024-04-17 22:29:07.822917","last_became_peered":"2024-04-17 22:29:07.822917","last_unstale":"2024-04-17 22:29:07.828447","last_undegraded":"2024-04-17 22:29:07.828447","last_fullsized":"2024-04-17 22:29:07.828447","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.10","version":"15'4","reported_seq":"14","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.817471","last_change":"2024-04-17 22:29:07.457017","last_active":"2024-04-17 22:29:07.817471","last_peered":"2024-04-17 22:29:07.817471","last_clean":"2024-04-17 22:29:07.817471","last_became_active":"2024-04-17 22:29:07.456705","last_became_peered":"2024-04-17 22:29:07.456705","last_unstale":"2024-04-17 22:29:07.817471","last_undegraded":"2024-04-17 22:29:07.817471","last_fullsized":"2024-04-17 22:29:07.817471","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.825122","last_change":"2024-04-17 22:29:07.820464","last_active":"2024-04-17 22:29:07.825122","last_peered":"2024-04-17 22:29:07.825122","last_clean":"2024-04-17 22:29:07.825122","last_became_active":"2024-04-17 22:29:07.820151","last_became_peered":"2024-04-17 22:29:07.820151","last_unstale":"2024-04-17 22:29:07.825122","last_undegraded":"2024-04-17 22:29:07.825122","last_fullsized":"2024-04-17 22:29:07.825122","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.f","version":"15'1","reported_seq":"11","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.828338","last_change":"2024-04-17 22:29:07.478464","last_active":"2024-04-17 22:29:07.828338","last_peered":"2024-04-17 22:29:07.828338","last_clean":"2024-04-17 22:29:07.828338","last_became_active":"2024-04-17 22:29:07.478144","last_became_peered":"2024-04-17 22:29:07.478144","last_unstale":"2024-04-17 22:29:07.828338","last_undegraded":"2024-04-17 22:29:07.828338","last_fullsized":"2024-04-17 22:29:07.828338","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.828463","last_change":"2024-04-17 22:29:07.815171","last_active":"2024-04-17 22:29:07.828463","last_peered":"2024-04-17 22:29:07.828463","last_clean":"2024-04-17 22:29:07.828463","last_became_active":"2024-04-17 22:29:07.814830","last_became_peered":"2024-04-17 22:29:07.814830","last_unstale":"2024-04-17 22:29:07.828463","last_undegraded":"2024-04-17 22:29:07.828463","last_fullsized":"2024-04-17 22:29:07.828463","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"15'2","reported_seq":"12","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.811237","last_change":"2024-04-17 22:29:07.481865","last_active":"2024-04-17 22:29:07.811237","last_peered":"2024-04-17 22:29:07.811237","last_clean":"2024-04-17 22:29:07.811237","last_became_active":"2024-04-17 22:29:07.481506","last_became_peered":"2024-04-17 22:29:07.481506","last_unstale":"2024-04-17 22:29:07.811237","last_undegraded":"2024-04-17 22:29:07.811237","last_fullsized":"2024-04-17 22:29:07.811237","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.828178","last_change":"2024-04-17 22:29:07.818238","last_active":"2024-04-17 22:29:07.828178","last_peered":"2024-04-17 22:29:07.828178","last_clean":"2024-04-17 22:29:07.828178","last_became_active":"2024-04-17 22:29:07.818060","last_became_peered":"2024-04-17 22:29:07.818060","last_unstale":"2024-04-17 22:29:07.828178","last_undegraded":"2024-04-17 22:29:07.828178","last_fullsized":"2024-04-17 22:29:07.828178","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.796955","last_change":"2024-04-17 22:29:07.484999","last_active":"2024-04-17 22:29:07.796955","last_peered":"2024-04-17 22:29:07.796955","last_clean":"2024-04-17 22:29:07.796955","last_became_active":"2024-04-17 22:29:07.484668","last_became_peered":"2024-04-17 22:29:07.484668","last_unstale":"2024-04-17 22:29:07.796955","last_undegraded":"2024-04-17 22:29:07.796955","last_fullsized":"2024-04-17 22:29:07.796955","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.828594","last_change":"2024-04-17 22:29:07.826426","last_active":"2024-04-17 22:29:07.828594","last_peered":"2024-04-17 22:29:07.828594","last_clean":"2024-04-17 22:29:07.828594","last_became_active":"2024-04-17 22:29:07.826109","last_became_peered":"2024-04-17 22:29:07.826109","last_unstale":"2024-04-17 22:29:07.828594","last_undegraded":"2024-04-17 22:29:07.828594","last_fullsized":"2024-04-17 22:29:07.828594","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.797207","last_change":"2024-04-17 22:29:07.462218","last_active":"2024-04-17 22:29:07.797207","last_peered":"2024-04-17 22:29:07.797207","last_clean":"2024-04-17 22:29:07.797207","last_became_active":"2024-04-17 22:29:07.461842","last_became_peered":"2024-04-17 22:29:07.461842","last_unstale":"2024-04-17 22:29:07.797207","last_undegraded":"2024-04-17 22:29:07.797207","last_fullsized":"2024-04-17 22:29:07.797207","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.820296","last_change":"2024-04-17 22:29:07.817144","last_active":"2024-04-17 22:29:07.820296","last_peered":"2024-04-17 22:29:07.820296","last_clean":"2024-04-17 22:29:07.820296","last_became_active":"2024-04-17 22:29:07.816859","last_became_peered":"2024-04-17 22:29:07.816859","last_unstale":"2024-04-17 22:29:07.820296","last_undegraded":"2024-04-17 22:29:07.820296","last_fullsized":"2024-04-17 22:29:07.820296","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"15'1","reported_seq":"11","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.821604","last_change":"2024-04-17 22:29:07.479816","last_active":"2024-04-17 22:29:07.821604","last_peered":"2024-04-17 22:29:07.821604","last_clean":"2024-04-17 22:29:07.821604","last_became_active":"2024-04-17 22:29:07.479251","last_became_peered":"2024-04-17 22:29:07.479251","last_unstale":"2024-04-17 22:29:07.821604","last_undegraded":"2024-04-17 22:29:07.821604","last_fullsized":"2024-04-17 22:29:07.821604","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.833516","last_change":"2024-04-17 22:29:07.829215","last_active":"2024-04-17 22:29:07.833516","last_peered":"2024-04-17 22:29:07.833516","last_clean":"2024-04-17 22:29:07.833516","last_became_active":"2024-04-17 22:29:07.828803","last_became_peered":"2024-04-17 22:29:07.828803","last_unstale":"2024-04-17 22:29:07.833516","last_undegraded":"2024-04-17 22:29:07.833516","last_fullsized":"2024-04-17 22:29:07.833516","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.796428","last_change":"2024-04-17 22:29:07.473128","last_active":"2024-04-17 22:29:07.796428","last_peered":"2024-04-17 22:29:07.796428","last_clean":"2024-04-17 22:29:07.796428","last_became_active":"2024-04-17 22:29:07.472806","last_became_peered":"2024-04-17 22:29:07.472806","last_unstale":"2024-04-17 22:29:07.796428","last_undegraded":"2024-04-17 22:29:07.796428","last_fullsized":"2024-04-17 22:29:07.796428","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.820555","last_change":"2024-04-17 22:29:07.817630","last_active":"2024-04-17 22:29:07.820555","last_peered":"2024-04-17 22:29:07.820555","last_clean":"2024-04-17 22:29:07.820555","last_became_active":"2024-04-17 22:29:07.817333","last_became_peered":"2024-04-17 22:29:07.817333","last_unstale":"2024-04-17 22:29:07.820555","last_undegraded":"2024-04-17 22:29:07.820555","last_fullsized":"2024-04-17 22:29:07.820555","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.792279","last_change":"2024-04-17 22:29:07.465053","last_active":"2024-04-17 22:29:07.792279","last_peered":"2024-04-17 22:29:07.792279","last_clean":"2024-04-17 22:29:07.792279","last_became_active":"2024-04-17 22:29:07.464766","last_became_peered":"2024-04-17 22:29:07.464766","last_unstale":"2024-04-17 22:29:07.792279","last_undegraded":"2024-04-17 22:29:07.792279","last_fullsized":"2024-04-17 22:29:07.792279","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.827066","last_change":"2024-04-17 22:29:07.823514","last_active":"2024-04-17 22:29:07.827066","last_peered":"2024-04-17 22:29:07.827066","last_clean":"2024-04-17 22:29:07.827066","last_became_active":"2024-04-17 22:29:07.823007","last_became_peered":"2024-04-17 22:29:07.823007","last_unstale":"2024-04-17 22:29:07.827066","last_undegraded":"2024-04-17 22:29:07.827066","last_fullsized":"2024-04-17 22:29:07.827066","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.801156","last_change":"2024-04-17 22:29:07.461568","last_active":"2024-04-17 22:29:07.801156","last_peered":"2024-04-17 22:29:07.801156","last_clean":"2024-04-17 22:29:07.801156","last_became_active":"2024-04-17 22:29:07.461249","last_became_peered":"2024-04-17 22:29:07.461249","last_unstale":"2024-04-17 22:29:07.801156","last_undegraded":"2024-04-17 22:29:07.801156","last_fullsized":"2024-04-17 22:29:07.801156","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.828426","last_change":"2024-04-17 22:29:07.826342","last_active":"2024-04-17 22:29:07.828426","last_peered":"2024-04-17 22:29:07.828426","last_clean":"2024-04-17 22:29:07.828426","last_became_active":"2024-04-17 22:29:07.826029","last_became_peered":"2024-04-17 22:29:07.826029","last_unstale":"2024-04-17 22:29:07.828426","last_undegraded":"2024-04-17 22:29:07.828426","last_fullsized":"2024-04-17 22:29:07.828426","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":"13","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.795463","last_change":"2024-04-17 22:28:59.253977","last_active":"2024-04-17 22:29:07.795463","last_peered":"2024-04-17 22:29:07.795463","last_clean":"2024-04-17 22:29:07.795463","last_became_active":"2024-04-17 22:28:59.253641","last_became_peered":"2024-04-17 22:28:59.253641","last_unstale":"2024-04-17 22:29:07.795463","last_undegraded":"2024-04-17 22:29:07.795463","last_fullsized":"2024-04-17 22:29:07.795463","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:28:58.239187","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:28:58.239187","last_clean_scrub_stamp":"2024-04-17 22:28:58.239187","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.796951","last_change":"2024-04-17 22:29:07.466907","last_active":"2024-04-17 22:29:07.796951","last_peered":"2024-04-17 22:29:07.796951","last_clean":"2024-04-17 22:29:07.796951","last_became_active":"2024-04-17 22:29:07.466583","last_became_peered":"2024-04-17 22:29:07.466583","last_unstale":"2024-04-17 22:29:07.796951","last_undegraded":"2024-04-17 22:29:07.796951","last_fullsized":"2024-04-17 22:29:07.796951","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.829316","last_change":"2024-04-17 22:29:07.824875","last_active":"2024-04-17 22:29:07.829316","last_peered":"2024-04-17 22:29:07.829316","last_clean":"2024-04-17 22:29:07.829316","last_became_active":"2024-04-17 22:29:07.824377","last_became_peered":"2024-04-17 22:29:07.824377","last_unstale":"2024-04-17 22:29:07.829316","last_undegraded":"2024-04-17 22:29:07.829316","last_fullsized":"2024-04-17 22:29:07.829316","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":"13","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.809532","last_change":"2024-04-17 22:28:59.255733","last_active":"2024-04-17 22:29:07.809532","last_peered":"2024-04-17 22:29:07.809532","last_clean":"2024-04-17 22:29:07.809532","last_became_active":"2024-04-17 22:28:59.255411","last_became_peered":"2024-04-17 22:28:59.255411","last_unstale":"2024-04-17 22:29:07.809532","last_undegraded":"2024-04-17 22:29:07.809532","last_fullsized":"2024-04-17 22:29:07.809532","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:28:58.239187","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:28:58.239187","last_clean_scrub_stamp":"2024-04-17 22:28:58.239187","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"15'2","reported_seq":"12","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.832132","last_change":"2024-04-17 22:29:07.478196","last_active":"2024-04-17 22:29:07.832132","last_peered":"2024-04-17 22:29:07.832132","last_clean":"2024-04-17 22:29:07.832132","last_became_active":"2024-04-17 22:29:07.477887","last_became_peered":"2024-04-17 22:29:07.477887","last_unstale":"2024-04-17 22:29:07.832132","last_undegraded":"2024-04-17 22:29:07.832132","last_fullsized":"2024-04-17 22:29:07.832132","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":"13","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.796430","last_change":"2024-04-17 22:28:59.252728","last_active":"2024-04-17 22:29:07.796430","last_peered":"2024-04-17 22:29:07.796430","last_clean":"2024-04-17 22:29:07.796430","last_became_active":"2024-04-17 22:28:59.252345","last_became_peered":"2024-04-17 22:28:59.252345","last_unstale":"2024-04-17 22:29:07.796430","last_undegraded":"2024-04-17 22:29:07.796430","last_fullsized":"2024-04-17 22:29:07.796430","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:28:58.239187","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:28:58.239187","last_clean_scrub_stamp":"2024-04-17 22:28:58.239187","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.797727","last_change":"2024-04-17 22:29:07.474520","last_active":"2024-04-17 22:29:07.797727","last_peered":"2024-04-17 22:29:07.797727","last_clean":"2024-04-17 22:29:07.797727","last_became_active":"2024-04-17 22:29:07.474029","last_became_peered":"2024-04-17 22:29:07.474029","last_unstale":"2024-04-17 22:29:07.797727","last_undegraded":"2024-04-17 22:29:07.797727","last_fullsized":"2024-04-17 22:29:07.797727","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.829302","last_change":"2024-04-17 22:29:07.825091","last_active":"2024-04-17 22:29:07.829302","last_peered":"2024-04-17 22:29:07.829302","last_clean":"2024-04-17 22:29:07.829302","last_became_active":"2024-04-17 22:29:07.824832","last_became_peered":"2024-04-17 22:29:07.824832","last_unstale":"2024-04-17 22:29:07.829302","last_undegraded":"2024-04-17 22:29:07.829302","last_fullsized":"2024-04-17 22:29:07.829302","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":"13","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.807580","last_change":"2024-04-17 22:28:59.253392","last_active":"2024-04-17 22:29:07.807580","last_peered":"2024-04-17 22:29:07.807580","last_clean":"2024-04-17 22:29:07.807580","last_became_active":"2024-04-17 22:28:59.253049","last_became_peered":"2024-04-17 22:28:59.253049","last_unstale":"2024-04-17 22:29:07.807580","last_undegraded":"2024-04-17 22:29:07.807580","last_fullsized":"2024-04-17 22:29:07.807580","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:28:58.239187","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:28:58.239187","last_clean_scrub_stamp":"2024-04-17 22:28:58.239187","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.796512","last_change":"2024-04-17 22:29:07.471913","last_active":"2024-04-17 22:29:07.796512","last_peered":"2024-04-17 22:29:07.796512","last_clean":"2024-04-17 22:29:07.796512","last_became_active":"2024-04-17 22:29:07.471686","last_became_peered":"2024-04-17 22:29:07.471686","last_unstale":"2024-04-17 22:29:07.796512","last_undegraded":"2024-04-17 22:29:07.796512","last_fullsized":"2024-04-17 22:29:07.796512","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.820450","last_change":"2024-04-17 22:29:07.812999","last_active":"2024-04-17 22:29:07.820450","last_peered":"2024-04-17 22:29:07.820450","last_clean":"2024-04-17 22:29:07.820450","last_became_active":"2024-04-17 22:29:07.812701","last_became_peered":"2024-04-17 22:29:07.812701","last_unstale":"2024-04-17 22:29:07.820450","last_undegraded":"2024-04-17 22:29:07.820450","last_fullsized":"2024-04-17 22:29:07.820450","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":"13","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.791848","last_change":"2024-04-17 22:28:59.254907","last_active":"2024-04-17 22:29:07.791848","last_peered":"2024-04-17 22:29:07.791848","last_clean":"2024-04-17 22:29:07.791848","last_became_active":"2024-04-17 22:28:59.254633","last_became_peered":"2024-04-17 22:28:59.254633","last_unstale":"2024-04-17 22:29:07.791848","last_undegraded":"2024-04-17 22:29:07.791848","last_fullsized":"2024-04-17 22:29:07.791848","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:28:58.239187","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:28:58.239187","last_clean_scrub_stamp":"2024-04-17 22:28:58.239187","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.807141","last_change":"2024-04-17 22:29:07.480020","last_active":"2024-04-17 22:29:07.807141","last_peered":"2024-04-17 22:29:07.807141","last_clean":"2024-04-17 22:29:07.807141","last_became_active":"2024-04-17 22:29:07.479492","last_became_peered":"2024-04-17 22:29:07.479492","last_unstale":"2024-04-17 22:29:07.807141","last_undegraded":"2024-04-17 22:29:07.807141","last_fullsized":"2024-04-17 22:29:07.807141","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.826030","last_change":"2024-04-17 22:29:07.821883","last_active":"2024-04-17 22:29:07.826030","last_peered":"2024-04-17 22:29:07.826030","last_clean":"2024-04-17 22:29:07.826030","last_became_active":"2024-04-17 22:29:07.821569","last_became_peered":"2024-04-17 22:29:07.821569","last_unstale":"2024-04-17 22:29:07.826030","last_undegraded":"2024-04-17 22:29:07.826030","last_fullsized":"2024-04-17 22:29:07.826030","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":"13","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.806466","last_change":"2024-04-17 22:28:59.254217","last_active":"2024-04-17 22:29:07.806466","last_peered":"2024-04-17 22:29:07.806466","last_clean":"2024-04-17 22:29:07.806466","last_became_active":"2024-04-17 22:28:59.253877","last_became_peered":"2024-04-17 22:28:59.253877","last_unstale":"2024-04-17 22:29:07.806466","last_undegraded":"2024-04-17 22:29:07.806466","last_fullsized":"2024-04-17 22:29:07.806466","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:28:58.239187","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:28:58.239187","last_clean_scrub_stamp":"2024-04-17 22:28:58.239187","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.805257","last_change":"2024-04-17 22:29:07.480378","last_active":"2024-04-17 22:29:07.805257","last_peered":"2024-04-17 22:29:07.805257","last_clean":"2024-04-17 22:29:07.805257","last_became_active":"2024-04-17 22:29:07.479866","last_became_peered":"2024-04-17 22:29:07.479866","last_unstale":"2024-04-17 22:29:07.805257","last_undegraded":"2024-04-17 22:29:07.805257","last_fullsized":"2024-04-17 22:29:07.805257","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.829882","last_change":"2024-04-17 22:29:07.821315","last_active":"2024-04-17 22:29:07.829882","last_peered":"2024-04-17 22:29:07.829882","last_clean":"2024-04-17 22:29:07.829882","last_became_active":"2024-04-17 22:29:07.820937","last_became_peered":"2024-04-17 22:29:07.820937","last_unstale":"2024-04-17 22:29:07.829882","last_undegraded":"2024-04-17 22:29:07.829882","last_fullsized":"2024-04-17 22:29:07.829882","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":"13","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.813317","last_change":"2024-04-17 22:28:59.254830","last_active":"2024-04-17 22:29:07.813317","last_peered":"2024-04-17 22:29:07.813317","last_clean":"2024-04-17 22:29:07.813317","last_became_active":"2024-04-17 22:28:59.254505","last_became_peered":"2024-04-17 22:28:59.254505","last_unstale":"2024-04-17 22:29:07.813317","last_undegraded":"2024-04-17 22:29:07.813317","last_fullsized":"2024-04-17 22:29:07.813317","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:28:58.239187","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:28:58.239187","last_clean_scrub_stamp":"2024-04-17 22:28:58.239187","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.809624","last_change":"2024-04-17 22:29:07.475017","last_active":"2024-04-17 22:29:07.809624","last_peered":"2024-04-17 22:29:07.809624","last_clean":"2024-04-17 22:29:07.809624","last_became_active":"2024-04-17 22:29:07.473559","last_became_peered":"2024-04-17 22:29:07.473559","last_unstale":"2024-04-17 22:29:07.809624","last_undegraded":"2024-04-17 22:29:07.809624","last_fullsized":"2024-04-17 22:29:07.809624","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.811282","last_change":"2024-04-17 22:29:07.808988","last_active":"2024-04-17 22:29:07.811282","last_peered":"2024-04-17 22:29:07.811282","last_clean":"2024-04-17 22:29:07.811282","last_became_active":"2024-04-17 22:29:07.808663","last_became_peered":"2024-04-17 22:29:07.808663","last_unstale":"2024-04-17 22:29:07.811282","last_undegraded":"2024-04-17 22:29:07.811282","last_fullsized":"2024-04-17 22:29:07.811282","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":"13","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.796932","last_change":"2024-04-17 22:28:59.254055","last_active":"2024-04-17 22:29:07.796932","last_peered":"2024-04-17 22:29:07.796932","last_clean":"2024-04-17 22:29:07.796932","last_became_active":"2024-04-17 22:28:59.253750","last_became_peered":"2024-04-17 22:28:59.253750","last_unstale":"2024-04-17 22:29:07.796932","last_undegraded":"2024-04-17 22:29:07.796932","last_fullsized":"2024-04-17 22:29:07.796932","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:28:58.239187","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:28:58.239187","last_clean_scrub_stamp":"2024-04-17 22:28:58.239187","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.796660","last_change":"2024-04-17 22:29:07.466238","last_active":"2024-04-17 22:29:07.796660","last_peered":"2024-04-17 22:29:07.796660","last_clean":"2024-04-17 22:29:07.796660","last_became_active":"2024-04-17 22:29:07.465806","last_became_peered":"2024-04-17 22:29:07.465806","last_unstale":"2024-04-17 22:29:07.796660","last_undegraded":"2024-04-17 22:29:07.796660","last_fullsized":"2024-04-17 22:29:07.796660","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.820702","last_change":"2024-04-17 22:29:07.809155","last_active":"2024-04-17 22:29:07.820702","last_peered":"2024-04-17 22:29:07.820702","last_clean":"2024-04-17 22:29:07.820702","last_became_active":"2024-04-17 22:29:07.808850","last_became_peered":"2024-04-17 22:29:07.808850","last_unstale":"2024-04-17 22:29:07.820702","last_undegraded":"2024-04-17 22:29:07.820702","last_fullsized":"2024-04-17 22:29:07.820702","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.833569","last_change":"2024-04-17 22:29:07.832794","last_active":"2024-04-17 22:29:07.833569","last_peered":"2024-04-17 22:29:07.833569","last_clean":"2024-04-17 22:29:07.833569","last_became_active":"2024-04-17 22:29:07.832467","last_became_peered":"2024-04-17 22:29:07.832467","last_unstale":"2024-04-17 22:29:07.833569","last_undegraded":"2024-04-17 22:29:07.833569","last_fullsized":"2024-04-17 22:29:07.833569","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.811305","last_change":"2024-04-17 22:29:07.477743","last_active":"2024-04-17 22:29:07.811305","last_peered":"2024-04-17 22:29:07.811305","last_clean":"2024-04-17 22:29:07.811305","last_became_active":"2024-04-17 22:29:07.477503","last_became_peered":"2024-04-17 22:29:07.477503","last_unstale":"2024-04-17 22:29:07.811305","last_undegraded":"2024-04-17 22:29:07.811305","last_fullsized":"2024-04-17 22:29:07.811305","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.830256","last_change":"2024-04-17 22:29:07.828401","last_active":"2024-04-17 22:29:07.830256","last_peered":"2024-04-17 22:29:07.830256","last_clean":"2024-04-17 22:29:07.830256","last_became_active":"2024-04-17 22:29:07.828093","last_became_peered":"2024-04-17 22:29:07.828093","last_unstale":"2024-04-17 22:29:07.830256","last_undegraded":"2024-04-17 22:29:07.830256","last_fullsized":"2024-04-17 22:29:07.830256","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"15'2","reported_seq":"12","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.824506","last_change":"2024-04-17 22:29:07.478361","last_active":"2024-04-17 22:29:07.824506","last_peered":"2024-04-17 22:29:07.824506","last_clean":"2024-04-17 22:29:07.824506","last_became_active":"2024-04-17 22:29:07.477793","last_became_peered":"2024-04-17 22:29:07.477793","last_unstale":"2024-04-17 22:29:07.824506","last_undegraded":"2024-04-17 22:29:07.824506","last_fullsized":"2024-04-17 22:29:07.824506","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.828482","last_change":"2024-04-17 22:29:07.815548","last_active":"2024-04-17 22:29:07.828482","last_peered":"2024-04-17 22:29:07.828482","last_clean":"2024-04-17 22:29:07.828482","last_became_active":"2024-04-17 22:29:07.815263","last_became_peered":"2024-04-17 22:29:07.815263","last_unstale":"2024-04-17 22:29:07.828482","last_undegraded":"2024-04-17 22:29:07.828482","last_fullsized":"2024-04-17 22:29:07.828482","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"15'1","reported_seq":"11","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.831868","last_change":"2024-04-17 22:29:07.482507","last_active":"2024-04-17 22:29:07.831868","last_peered":"2024-04-17 22:29:07.831868","last_clean":"2024-04-17 22:29:07.831868","last_became_active":"2024-04-17 22:29:07.481981","last_became_peered":"2024-04-17 22:29:07.481981","last_unstale":"2024-04-17 22:29:07.831868","last_undegraded":"2024-04-17 22:29:07.831868","last_fullsized":"2024-04-17 22:29:07.831868","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.833109","last_change":"2024-04-17 22:29:07.828459","last_active":"2024-04-17 22:29:07.833109","last_peered":"2024-04-17 22:29:07.833109","last_clean":"2024-04-17 22:29:07.833109","last_became_active":"2024-04-17 22:29:07.828157","last_became_peered":"2024-04-17 22:29:07.828157","last_unstale":"2024-04-17 22:29:07.833109","last_undegraded":"2024-04-17 22:29:07.833109","last_fullsized":"2024-04-17 22:29:07.833109","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"15'1","reported_seq":"11","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.828398","last_change":"2024-04-17 22:29:07.471751","last_active":"2024-04-17 22:29:07.828398","last_peered":"2024-04-17 22:29:07.828398","last_clean":"2024-04-17 22:29:07.828398","last_became_active":"2024-04-17 22:29:07.471385","last_became_peered":"2024-04-17 22:29:07.471385","last_unstale":"2024-04-17 22:29:07.828398","last_undegraded":"2024-04-17 22:29:07.828398","last_fullsized":"2024-04-17 22:29:07.828398","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.821701","last_change":"2024-04-17 22:29:07.812125","last_active":"2024-04-17 22:29:07.821701","last_peered":"2024-04-17 22:29:07.821701","last_clean":"2024-04-17 22:29:07.821701","last_became_active":"2024-04-17 22:29:07.811846","last_became_peered":"2024-04-17 22:29:07.811846","last_unstale":"2024-04-17 22:29:07.821701","last_undegraded":"2024-04-17 22:29:07.821701","last_fullsized":"2024-04-17 22:29:07.821701","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.20","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.798111","last_change":"2024-04-17 22:29:07.477127","last_active":"2024-04-17 22:29:07.798111","last_peered":"2024-04-17 22:29:07.798111","last_clean":"2024-04-17 22:29:07.798111","last_became_active":"2024-04-17 22:29:07.476773","last_became_peered":"2024-04-17 22:29:07.476773","last_unstale":"2024-04-17 22:29:07.798111","last_undegraded":"2024-04-17 22:29:07.798111","last_fullsized":"2024-04-17 22:29:07.798111","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.824583","last_change":"2024-04-17 22:29:07.821132","last_active":"2024-04-17 22:29:07.824583","last_peered":"2024-04-17 22:29:07.824583","last_clean":"2024-04-17 22:29:07.824583","last_became_active":"2024-04-17 22:29:07.820838","last_became_peered":"2024-04-17 22:29:07.820838","last_unstale":"2024-04-17 22:29:07.824583","last_undegraded":"2024-04-17 22:29:07.824583","last_fullsized":"2024-04-17 22:29:07.824583","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.21","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.796071","last_change":"2024-04-17 22:29:07.460110","last_active":"2024-04-17 22:29:07.796071","last_peered":"2024-04-17 22:29:07.796071","last_clean":"2024-04-17 22:29:07.796071","last_became_active":"2024-04-17 22:29:07.459611","last_became_peered":"2024-04-17 22:29:07.459611","last_unstale":"2024-04-17 22:29:07.796071","last_undegraded":"2024-04-17 22:29:07.796071","last_fullsized":"2024-04-17 22:29:07.796071","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.828609","last_change":"2024-04-17 22:29:07.818496","last_active":"2024-04-17 22:29:07.828609","last_peered":"2024-04-17 22:29:07.828609","last_clean":"2024-04-17 22:29:07.828609","last_became_active":"2024-04-17 22:29:07.818184","last_became_peered":"2024-04-17 22:29:07.818184","last_unstale":"2024-04-17 22:29:07.828609","last_undegraded":"2024-04-17 22:29:07.828609","last_fullsized":"2024-04-17 22:29:07.828609","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.22","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.805451","last_change":"2024-04-17 22:29:07.481695","last_active":"2024-04-17 22:29:07.805451","last_peered":"2024-04-17 22:29:07.805451","last_clean":"2024-04-17 22:29:07.805451","last_became_active":"2024-04-17 22:29:07.479982","last_became_peered":"2024-04-17 22:29:07.479982","last_unstale":"2024-04-17 22:29:07.805451","last_undegraded":"2024-04-17 22:29:07.805451","last_fullsized":"2024-04-17 22:29:07.805451","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.835093","last_change":"2024-04-17 22:29:07.833317","last_active":"2024-04-17 22:29:07.835093","last_peered":"2024-04-17 22:29:07.835093","last_clean":"2024-04-17 22:29:07.835093","last_became_active":"2024-04-17 22:29:07.827050","last_became_peered":"2024-04-17 22:29:07.827050","last_unstale":"2024-04-17 22:29:07.835093","last_undegraded":"2024-04-17 22:29:07.835093","last_fullsized":"2024-04-17 22:29:07.835093","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.23","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.807837","last_change":"2024-04-17 22:29:07.477449","last_active":"2024-04-17 22:29:07.807837","last_peered":"2024-04-17 22:29:07.807837","last_clean":"2024-04-17 22:29:07.807837","last_became_active":"2024-04-17 22:29:07.477155","last_became_peered":"2024-04-17 22:29:07.477155","last_unstale":"2024-04-17 22:29:07.807837","last_undegraded":"2024-04-17 22:29:07.807837","last_fullsized":"2024-04-17 22:29:07.807837","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.810313","last_change":"2024-04-17 22:29:07.808974","last_active":"2024-04-17 22:29:07.810313","last_peered":"2024-04-17 22:29:07.810313","last_clean":"2024-04-17 22:29:07.810313","last_became_active":"2024-04-17 22:29:07.808747","last_became_peered":"2024-04-17 22:29:07.808747","last_unstale":"2024-04-17 22:29:07.810313","last_undegraded":"2024-04-17 22:29:07.810313","last_fullsized":"2024-04-17 22:29:07.810313","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.24","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.796551","last_change":"2024-04-17 22:29:07.466052","last_active":"2024-04-17 22:29:07.796551","last_peered":"2024-04-17 22:29:07.796551","last_clean":"2024-04-17 22:29:07.796551","last_became_active":"2024-04-17 22:29:07.465576","last_became_peered":"2024-04-17 22:29:07.465576","last_unstale":"2024-04-17 22:29:07.796551","last_undegraded":"2024-04-17 22:29:07.796551","last_fullsized":"2024-04-17 22:29:07.796551","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.833471","last_change":"2024-04-17 22:29:07.829066","last_active":"2024-04-17 22:29:07.833471","last_peered":"2024-04-17 22:29:07.833471","last_clean":"2024-04-17 22:29:07.833471","last_became_active":"2024-04-17 22:29:07.828735","last_became_peered":"2024-04-17 22:29:07.828735","last_unstale":"2024-04-17 22:29:07.833471","last_undegraded":"2024-04-17 22:29:07.833471","last_fullsized":"2024-04-17 22:29:07.833471","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.25","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.795566","last_change":"2024-04-17 22:29:07.462011","last_active":"2024-04-17 22:29:07.795566","last_peered":"2024-04-17 22:29:07.795566","last_clean":"2024-04-17 22:29:07.795566","last_became_active":"2024-04-17 22:29:07.461693","last_became_peered":"2024-04-17 22:29:07.461693","last_unstale":"2024-04-17 22:29:07.795566","last_undegraded":"2024-04-17 22:29:07.795566","last_fullsized":"2024-04-17 22:29:07.795566","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.828447","last_change":"2024-04-17 22:29:07.826655","last_active":"2024-04-17 22:29:07.828447","last_peered":"2024-04-17 22:29:07.828447","last_clean":"2024-04-17 22:29:07.828447","last_became_active":"2024-04-17 22:29:07.823665","last_became_peered":"2024-04-17 22:29:07.823665","last_unstale":"2024-04-17 22:29:07.828447","last_undegraded":"2024-04-17 22:29:07.828447","last_fullsized":"2024-04-17 22:29:07.828447","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.26","version":"16'1","reported_seq":"11","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:08.802774","last_change":"2024-04-17 22:29:07.480854","last_active":"2024-04-17 22:29:08.802774","last_peered":"2024-04-17 22:29:08.802774","last_clean":"2024-04-17 22:29:08.802774","last_became_active":"2024-04-17 22:29:07.479975","last_became_peered":"2024-04-17 22:29:07.479975","last_unstale":"2024-04-17 22:29:08.802774","last_undegraded":"2024-04-17 22:29:08.802774","last_fullsized":"2024-04-17 22:29:08.802774","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.825070","last_change":"2024-04-17 22:29:07.813744","last_active":"2024-04-17 22:29:07.825070","last_peered":"2024-04-17 22:29:07.825070","last_clean":"2024-04-17 22:29:07.825070","last_became_active":"2024-04-17 22:29:07.810646","last_became_peered":"2024-04-17 22:29:07.810646","last_unstale":"2024-04-17 22:29:07.825070","last_undegraded":"2024-04-17 22:29:07.825070","last_fullsized":"2024-04-17 22:29:07.825070","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.27","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.815316","last_change":"2024-04-17 22:29:07.476428","last_active":"2024-04-17 22:29:07.815316","last_peered":"2024-04-17 22:29:07.815316","last_clean":"2024-04-17 22:29:07.815316","last_became_active":"2024-04-17 22:29:07.476128","last_became_peered":"2024-04-17 22:29:07.476128","last_unstale":"2024-04-17 22:29:07.815316","last_undegraded":"2024-04-17 22:29:07.815316","last_fullsized":"2024-04-17 22:29:07.815316","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.826418","last_change":"2024-04-17 22:29:07.823340","last_active":"2024-04-17 22:29:07.826418","last_peered":"2024-04-17 22:29:07.826418","last_clean":"2024-04-17 22:29:07.826418","last_became_active":"2024-04-17 22:29:07.822764","last_became_peered":"2024-04-17 22:29:07.822764","last_unstale":"2024-04-17 22:29:07.826418","last_undegraded":"2024-04-17 22:29:07.826418","last_fullsized":"2024-04-17 22:29:07.826418","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.28","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.798818","last_change":"2024-04-17 22:29:07.477137","last_active":"2024-04-17 22:29:07.798818","last_peered":"2024-04-17 22:29:07.798818","last_clean":"2024-04-17 22:29:07.798818","last_became_active":"2024-04-17 22:29:07.476796","last_became_peered":"2024-04-17 22:29:07.476796","last_unstale":"2024-04-17 22:29:07.798818","last_undegraded":"2024-04-17 22:29:07.798818","last_fullsized":"2024-04-17 22:29:07.798818","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.826239","last_change":"2024-04-17 22:29:07.823119","last_active":"2024-04-17 22:29:07.826239","last_peered":"2024-04-17 22:29:07.826239","last_clean":"2024-04-17 22:29:07.826239","last_became_active":"2024-04-17 22:29:07.822640","last_became_peered":"2024-04-17 22:29:07.822640","last_unstale":"2024-04-17 22:29:07.826239","last_undegraded":"2024-04-17 22:29:07.826239","last_fullsized":"2024-04-17 22:29:07.826239","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.29","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.792382","last_change":"2024-04-17 22:29:07.473467","last_active":"2024-04-17 22:29:07.792382","last_peered":"2024-04-17 22:29:07.792382","last_clean":"2024-04-17 22:29:07.792382","last_became_active":"2024-04-17 22:29:07.473028","last_became_peered":"2024-04-17 22:29:07.473028","last_unstale":"2024-04-17 22:29:07.792382","last_undegraded":"2024-04-17 22:29:07.792382","last_fullsized":"2024-04-17 22:29:07.792382","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.827332","last_change":"2024-04-17 22:29:07.817097","last_active":"2024-04-17 22:29:07.827332","last_peered":"2024-04-17 22:29:07.827332","last_clean":"2024-04-17 22:29:07.827332","last_became_active":"2024-04-17 22:29:07.816813","last_became_peered":"2024-04-17 22:29:07.816813","last_unstale":"2024-04-17 22:29:07.827332","last_undegraded":"2024-04-17 22:29:07.827332","last_fullsized":"2024-04-17 22:29:07.827332","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2a","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.796331","last_change":"2024-04-17 22:29:07.464330","last_active":"2024-04-17 22:29:07.796331","last_peered":"2024-04-17 22:29:07.796331","last_clean":"2024-04-17 22:29:07.796331","last_became_active":"2024-04-17 22:29:07.464002","last_became_peered":"2024-04-17 22:29:07.464002","last_unstale":"2024-04-17 22:29:07.796331","last_undegraded":"2024-04-17 22:29:07.796331","last_fullsized":"2024-04-17 22:29:07.796331","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.830305","last_change":"2024-04-17 22:29:07.828690","last_active":"2024-04-17 22:29:07.830305","last_peered":"2024-04-17 22:29:07.830305","last_clean":"2024-04-17 22:29:07.830305","last_became_active":"2024-04-17 22:29:07.828407","last_became_peered":"2024-04-17 22:29:07.828407","last_unstale":"2024-04-17 22:29:07.830305","last_undegraded":"2024-04-17 22:29:07.830305","last_fullsized":"2024-04-17 22:29:07.830305","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2b","version":"15'1","reported_seq":"11","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.810554","last_change":"2024-04-17 22:29:07.471300","last_active":"2024-04-17 22:29:07.810554","last_peered":"2024-04-17 22:29:07.810554","last_clean":"2024-04-17 22:29:07.810554","last_became_active":"2024-04-17 22:29:07.470886","last_became_peered":"2024-04-17 22:29:07.470886","last_unstale":"2024-04-17 22:29:07.810554","last_undegraded":"2024-04-17 22:29:07.810554","last_fullsized":"2024-04-17 22:29:07.810554","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.828614","last_change":"2024-04-17 22:29:07.826555","last_active":"2024-04-17 22:29:07.828614","last_peered":"2024-04-17 22:29:07.828614","last_clean":"2024-04-17 22:29:07.828614","last_became_active":"2024-04-17 22:29:07.826262","last_became_peered":"2024-04-17 22:29:07.826262","last_unstale":"2024-04-17 22:29:07.828614","last_undegraded":"2024-04-17 22:29:07.828614","last_fullsized":"2024-04-17 22:29:07.828614","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2c","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.797039","last_change":"2024-04-17 22:29:07.462176","last_active":"2024-04-17 22:29:07.797039","last_peered":"2024-04-17 22:29:07.797039","last_clean":"2024-04-17 22:29:07.797039","last_became_active":"2024-04-17 22:29:07.461594","last_became_peered":"2024-04-17 22:29:07.461594","last_unstale":"2024-04-17 22:29:07.797039","last_undegraded":"2024-04-17 22:29:07.797039","last_fullsized":"2024-04-17 22:29:07.797039","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.828130","last_change":"2024-04-17 22:29:07.816896","last_active":"2024-04-17 22:29:07.828130","last_peered":"2024-04-17 22:29:07.828130","last_clean":"2024-04-17 22:29:07.828130","last_became_active":"2024-04-17 22:29:07.816538","last_became_peered":"2024-04-17 22:29:07.816538","last_unstale":"2024-04-17 22:29:07.828130","last_undegraded":"2024-04-17 22:29:07.828130","last_fullsized":"2024-04-17 22:29:07.828130","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2d","version":"15'1","reported_seq":"11","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.827538","last_change":"2024-04-17 22:29:07.479214","last_active":"2024-04-17 22:29:07.827538","last_peered":"2024-04-17 22:29:07.827538","last_clean":"2024-04-17 22:29:07.827538","last_became_active":"2024-04-17 22:29:07.478790","last_became_peered":"2024-04-17 22:29:07.478790","last_unstale":"2024-04-17 22:29:07.827538","last_undegraded":"2024-04-17 22:29:07.827538","last_fullsized":"2024-04-17 22:29:07.827538","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.833543","last_change":"2024-04-17 22:29:07.832349","last_active":"2024-04-17 22:29:07.833543","last_peered":"2024-04-17 22:29:07.833543","last_clean":"2024-04-17 22:29:07.833543","last_became_active":"2024-04-17 22:29:07.831999","last_became_peered":"2024-04-17 22:29:07.831999","last_unstale":"2024-04-17 22:29:07.833543","last_undegraded":"2024-04-17 22:29:07.833543","last_fullsized":"2024-04-17 22:29:07.833543","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2e","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.797059","last_change":"2024-04-17 22:29:07.483244","last_active":"2024-04-17 22:29:07.797059","last_peered":"2024-04-17 22:29:07.797059","last_clean":"2024-04-17 22:29:07.797059","last_became_active":"2024-04-17 22:29:07.482893","last_became_peered":"2024-04-17 22:29:07.482893","last_unstale":"2024-04-17 22:29:07.797059","last_undegraded":"2024-04-17 22:29:07.797059","last_fullsized":"2024-04-17 22:29:07.797059","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.825028","last_change":"2024-04-17 22:29:07.806512","last_active":"2024-04-17 22:29:07.825028","last_peered":"2024-04-17 22:29:07.825028","last_clean":"2024-04-17 22:29:07.825028","last_became_active":"2024-04-17 22:29:07.806182","last_became_peered":"2024-04-17 22:29:07.806182","last_unstale":"2024-04-17 22:29:07.825028","last_undegraded":"2024-04-17 22:29:07.825028","last_fullsized":"2024-04-17 22:29:07.825028","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2f","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.815418","last_change":"2024-04-17 22:29:07.478655","last_active":"2024-04-17 22:29:07.815418","last_peered":"2024-04-17 22:29:07.815418","last_clean":"2024-04-17 22:29:07.815418","last_became_active":"2024-04-17 22:29:07.478346","last_became_peered":"2024-04-17 22:29:07.478346","last_unstale":"2024-04-17 22:29:07.815418","last_undegraded":"2024-04-17 22:29:07.815418","last_fullsized":"2024-04-17 22:29:07.815418","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.826848","last_change":"2024-04-17 22:29:07.825658","last_active":"2024-04-17 22:29:07.826848","last_peered":"2024-04-17 22:29:07.826848","last_clean":"2024-04-17 22:29:07.826848","last_became_active":"2024-04-17 22:29:07.825349","last_became_peered":"2024-04-17 22:29:07.825349","last_unstale":"2024-04-17 22:29:07.826848","last_undegraded":"2024-04-17 22:29:07.826848","last_fullsized":"2024-04-17 22:29:07.826848","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.30","version":"15'4","reported_seq":"14","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.824396","last_change":"2024-04-17 22:29:07.475470","last_active":"2024-04-17 22:29:07.824396","last_peered":"2024-04-17 22:29:07.824396","last_clean":"2024-04-17 22:29:07.824396","last_became_active":"2024-04-17 22:29:07.474917","last_became_peered":"2024-04-17 22:29:07.474917","last_unstale":"2024-04-17 22:29:07.824396","last_undegraded":"2024-04-17 22:29:07.824396","last_fullsized":"2024-04-17 22:29:07.824396","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.826262","last_change":"2024-04-17 22:29:07.823624","last_active":"2024-04-17 22:29:07.826262","last_peered":"2024-04-17 22:29:07.826262","last_clean":"2024-04-17 22:29:07.826262","last_became_active":"2024-04-17 22:29:07.822815","last_became_peered":"2024-04-17 22:29:07.822815","last_unstale":"2024-04-17 22:29:07.826262","last_undegraded":"2024-04-17 22:29:07.826262","last_fullsized":"2024-04-17 22:29:07.826262","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.31","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.804183","last_change":"2024-04-17 22:29:07.474318","last_active":"2024-04-17 22:29:07.804183","last_peered":"2024-04-17 22:29:07.804183","last_clean":"2024-04-17 22:29:07.804183","last_became_active":"2024-04-17 22:29:07.473754","last_became_peered":"2024-04-17 22:29:07.473754","last_unstale":"2024-04-17 22:29:07.804183","last_undegraded":"2024-04-17 22:29:07.804183","last_fullsized":"2024-04-17 22:29:07.804183","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.833495","last_change":"2024-04-17 22:29:07.829098","last_active":"2024-04-17 22:29:07.833495","last_peered":"2024-04-17 22:29:07.833495","last_clean":"2024-04-17 22:29:07.833495","last_became_active":"2024-04-17 22:29:07.828773","last_became_peered":"2024-04-17 22:29:07.828773","last_unstale":"2024-04-17 22:29:07.833495","last_undegraded":"2024-04-17 22:29:07.833495","last_fullsized":"2024-04-17 22:29:07.833495","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.32","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.795411","last_change":"2024-04-17 22:29:07.470666","last_active":"2024-04-17 22:29:07.795411","last_peered":"2024-04-17 22:29:07.795411","last_clean":"2024-04-17 22:29:07.795411","last_became_active":"2024-04-17 22:29:07.470363","last_became_peered":"2024-04-17 22:29:07.470363","last_unstale":"2024-04-17 22:29:07.795411","last_undegraded":"2024-04-17 22:29:07.795411","last_fullsized":"2024-04-17 22:29:07.795411","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.830343","last_change":"2024-04-17 22:29:07.827580","last_active":"2024-04-17 22:29:07.830343","last_peered":"2024-04-17 22:29:07.830343","last_clean":"2024-04-17 22:29:07.830343","last_became_active":"2024-04-17 22:29:07.827233","last_became_peered":"2024-04-17 22:29:07.827233","last_unstale":"2024-04-17 22:29:07.830343","last_undegraded":"2024-04-17 22:29:07.830343","last_fullsized":"2024-04-17 22:29:07.830343","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.33","version":"15'1","reported_seq":"11","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.811613","last_change":"2024-04-17 22:29:07.460978","last_active":"2024-04-17 22:29:07.811613","last_peered":"2024-04-17 22:29:07.811613","last_clean":"2024-04-17 22:29:07.811613","last_became_active":"2024-04-17 22:29:07.460659","last_became_peered":"2024-04-17 22:29:07.460659","last_unstale":"2024-04-17 22:29:07.811613","last_undegraded":"2024-04-17 22:29:07.811613","last_fullsized":"2024-04-17 22:29:07.811613","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.832128","last_change":"2024-04-17 22:29:07.816335","last_active":"2024-04-17 22:29:07.832128","last_peered":"2024-04-17 22:29:07.832128","last_clean":"2024-04-17 22:29:07.832128","last_became_active":"2024-04-17 22:29:07.816090","last_became_peered":"2024-04-17 22:29:07.816090","last_unstale":"2024-04-17 22:29:07.832128","last_undegraded":"2024-04-17 22:29:07.832128","last_fullsized":"2024-04-17 22:29:07.832128","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.34","version":"15'4","reported_seq":"14","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.835387","last_change":"2024-04-17 22:29:07.475702","last_active":"2024-04-17 22:29:07.835387","last_peered":"2024-04-17 22:29:07.835387","last_clean":"2024-04-17 22:29:07.835387","last_became_active":"2024-04-17 22:29:07.475395","last_became_peered":"2024-04-17 22:29:07.475395","last_unstale":"2024-04-17 22:29:07.835387","last_undegraded":"2024-04-17 22:29:07.835387","last_fullsized":"2024-04-17 22:29:07.835387","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":892,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.828199","last_change":"2024-04-17 22:29:07.818538","last_active":"2024-04-17 22:29:07.828199","last_peered":"2024-04-17 22:29:07.828199","last_clean":"2024-04-17 22:29:07.828199","last_became_active":"2024-04-17 22:29:07.817164","last_became_peered":"2024-04-17 22:29:07.817164","last_unstale":"2024-04-17 22:29:07.828199","last_undegraded":"2024-04-17 22:29:07.828199","last_fullsized":"2024-04-17 22:29:07.828199","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.35","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.797071","last_change":"2024-04-17 22:29:07.467651","last_active":"2024-04-17 22:29:07.797071","last_peered":"2024-04-17 22:29:07.797071","last_clean":"2024-04-17 22:29:07.797071","last_became_active":"2024-04-17 22:29:07.467327","last_became_peered":"2024-04-17 22:29:07.467327","last_unstale":"2024-04-17 22:29:07.797071","last_undegraded":"2024-04-17 22:29:07.797071","last_fullsized":"2024-04-17 22:29:07.797071","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.810303","last_change":"2024-04-17 22:29:07.809693","last_active":"2024-04-17 22:29:07.810303","last_peered":"2024-04-17 22:29:07.810303","last_clean":"2024-04-17 22:29:07.810303","last_became_active":"2024-04-17 22:29:07.809415","last_became_peered":"2024-04-17 22:29:07.809415","last_unstale":"2024-04-17 22:29:07.810303","last_undegraded":"2024-04-17 22:29:07.810303","last_fullsized":"2024-04-17 22:29:07.810303","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.36","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.796841","last_change":"2024-04-17 22:29:07.483402","last_active":"2024-04-17 22:29:07.796841","last_peered":"2024-04-17 22:29:07.796841","last_clean":"2024-04-17 22:29:07.796841","last_became_active":"2024-04-17 22:29:07.482914","last_became_peered":"2024-04-17 22:29:07.482914","last_unstale":"2024-04-17 22:29:07.796841","last_undegraded":"2024-04-17 22:29:07.796841","last_fullsized":"2024-04-17 22:29:07.796841","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.830317","last_change":"2024-04-17 22:29:07.827623","last_active":"2024-04-17 22:29:07.830317","last_peered":"2024-04-17 22:29:07.830317","last_clean":"2024-04-17 22:29:07.830317","last_became_active":"2024-04-17 22:29:07.827337","last_became_peered":"2024-04-17 22:29:07.827337","last_unstale":"2024-04-17 22:29:07.830317","last_undegraded":"2024-04-17 22:29:07.830317","last_fullsized":"2024-04-17 22:29:07.830317","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.37","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.808508","last_change":"2024-04-17 22:29:07.463449","last_active":"2024-04-17 22:29:07.808508","last_peered":"2024-04-17 22:29:07.808508","last_clean":"2024-04-17 22:29:07.808508","last_became_active":"2024-04-17 22:29:07.463110","last_became_peered":"2024-04-17 22:29:07.463110","last_unstale":"2024-04-17 22:29:07.808508","last_undegraded":"2024-04-17 22:29:07.808508","last_fullsized":"2024-04-17 22:29:07.808508","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.827399","last_change":"2024-04-17 22:29:07.815214","last_active":"2024-04-17 22:29:07.827399","last_peered":"2024-04-17 22:29:07.827399","last_clean":"2024-04-17 22:29:07.827399","last_became_active":"2024-04-17 22:29:07.814938","last_became_peered":"2024-04-17 22:29:07.814938","last_unstale":"2024-04-17 22:29:07.827399","last_undegraded":"2024-04-17 22:29:07.827399","last_fullsized":"2024-04-17 22:29:07.827399","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.38","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.801328","last_change":"2024-04-17 22:29:07.474682","last_active":"2024-04-17 22:29:07.801328","last_peered":"2024-04-17 22:29:07.801328","last_clean":"2024-04-17 22:29:07.801328","last_became_active":"2024-04-17 22:29:07.474305","last_became_peered":"2024-04-17 22:29:07.474305","last_unstale":"2024-04-17 22:29:07.801328","last_undegraded":"2024-04-17 22:29:07.801328","last_fullsized":"2024-04-17 22:29:07.801328","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.833512","last_change":"2024-04-17 22:29:07.829769","last_active":"2024-04-17 22:29:07.833512","last_peered":"2024-04-17 22:29:07.833512","last_clean":"2024-04-17 22:29:07.833512","last_became_active":"2024-04-17 22:29:07.829484","last_became_peered":"2024-04-17 22:29:07.829484","last_unstale":"2024-04-17 22:29:07.833512","last_undegraded":"2024-04-17 22:29:07.833512","last_fullsized":"2024-04-17 22:29:07.833512","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.39","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.794112","last_change":"2024-04-17 22:29:07.458386","last_active":"2024-04-17 22:29:07.794112","last_peered":"2024-04-17 22:29:07.794112","last_clean":"2024-04-17 22:29:07.794112","last_became_active":"2024-04-17 22:29:07.456695","last_became_peered":"2024-04-17 22:29:07.456695","last_unstale":"2024-04-17 22:29:07.794112","last_undegraded":"2024-04-17 22:29:07.794112","last_fullsized":"2024-04-17 22:29:07.794112","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.829903","last_change":"2024-04-17 22:29:07.821490","last_active":"2024-04-17 22:29:07.829903","last_peered":"2024-04-17 22:29:07.829903","last_clean":"2024-04-17 22:29:07.829903","last_became_active":"2024-04-17 22:29:07.821103","last_became_peered":"2024-04-17 22:29:07.821103","last_unstale":"2024-04-17 22:29:07.829903","last_undegraded":"2024-04-17 22:29:07.829903","last_fullsized":"2024-04-17 22:29:07.829903","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3a","version":"15'1","reported_seq":"11","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.819620","last_change":"2024-04-17 22:29:07.472610","last_active":"2024-04-17 22:29:07.819620","last_peered":"2024-04-17 22:29:07.819620","last_clean":"2024-04-17 22:29:07.819620","last_became_active":"2024-04-17 22:29:07.472297","last_became_peered":"2024-04-17 22:29:07.472297","last_unstale":"2024-04-17 22:29:07.819620","last_undegraded":"2024-04-17 22:29:07.819620","last_fullsized":"2024-04-17 22:29:07.819620","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.824029","last_change":"2024-04-17 22:29:07.820738","last_active":"2024-04-17 22:29:07.824029","last_peered":"2024-04-17 22:29:07.824029","last_clean":"2024-04-17 22:29:07.824029","last_became_active":"2024-04-17 22:29:07.820354","last_became_peered":"2024-04-17 22:29:07.820354","last_unstale":"2024-04-17 22:29:07.824029","last_undegraded":"2024-04-17 22:29:07.824029","last_fullsized":"2024-04-17 22:29:07.824029","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3b","version":"15'1","reported_seq":"11","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.824121","last_change":"2024-04-17 22:29:07.473963","last_active":"2024-04-17 22:29:07.824121","last_peered":"2024-04-17 22:29:07.824121","last_clean":"2024-04-17 22:29:07.824121","last_became_active":"2024-04-17 22:29:07.472018","last_became_peered":"2024-04-17 22:29:07.472018","last_unstale":"2024-04-17 22:29:07.824121","last_undegraded":"2024-04-17 22:29:07.824121","last_fullsized":"2024-04-17 22:29:07.824121","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.832188","last_change":"2024-04-17 22:29:07.824919","last_active":"2024-04-17 22:29:07.832188","last_peered":"2024-04-17 22:29:07.832188","last_clean":"2024-04-17 22:29:07.832188","last_became_active":"2024-04-17 22:29:07.824600","last_became_peered":"2024-04-17 22:29:07.824600","last_unstale":"2024-04-17 22:29:07.832188","last_undegraded":"2024-04-17 22:29:07.832188","last_fullsized":"2024-04-17 22:29:07.832188","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3c","version":"15'2","reported_seq":"12","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.825427","last_change":"2024-04-17 22:29:07.475485","last_active":"2024-04-17 22:29:07.825427","last_peered":"2024-04-17 22:29:07.825427","last_clean":"2024-04-17 22:29:07.825427","last_became_active":"2024-04-17 22:29:07.475182","last_became_peered":"2024-04-17 22:29:07.475182","last_unstale":"2024-04-17 22:29:07.825427","last_undegraded":"2024-04-17 22:29:07.825427","last_fullsized":"2024-04-17 22:29:07.825427","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.821750","last_change":"2024-04-17 22:29:07.810455","last_active":"2024-04-17 22:29:07.821750","last_peered":"2024-04-17 22:29:07.821750","last_clean":"2024-04-17 22:29:07.821750","last_became_active":"2024-04-17 22:29:07.810143","last_became_peered":"2024-04-17 22:29:07.810143","last_unstale":"2024-04-17 22:29:07.821750","last_undegraded":"2024-04-17 22:29:07.821750","last_fullsized":"2024-04-17 22:29:07.821750","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3d","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.797015","last_change":"2024-04-17 22:29:07.468187","last_active":"2024-04-17 22:29:07.797015","last_peered":"2024-04-17 22:29:07.797015","last_clean":"2024-04-17 22:29:07.797015","last_became_active":"2024-04-17 22:29:07.467849","last_became_peered":"2024-04-17 22:29:07.467849","last_unstale":"2024-04-17 22:29:07.797015","last_undegraded":"2024-04-17 22:29:07.797015","last_fullsized":"2024-04-17 22:29:07.797015","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.830571","last_change":"2024-04-17 22:29:07.822173","last_active":"2024-04-17 22:29:07.830571","last_peered":"2024-04-17 22:29:07.830571","last_clean":"2024-04-17 22:29:07.830571","last_became_active":"2024-04-17 22:29:07.821875","last_became_peered":"2024-04-17 22:29:07.821875","last_unstale":"2024-04-17 22:29:07.830571","last_undegraded":"2024-04-17 22:29:07.830571","last_fullsized":"2024-04-17 22:29:07.830571","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3e","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.807733","last_change":"2024-04-17 22:29:07.479547","last_active":"2024-04-17 22:29:07.807733","last_peered":"2024-04-17 22:29:07.807733","last_clean":"2024-04-17 22:29:07.807733","last_became_active":"2024-04-17 22:29:07.479004","last_became_peered":"2024-04-17 22:29:07.479004","last_unstale":"2024-04-17 22:29:07.807733","last_undegraded":"2024-04-17 22:29:07.807733","last_fullsized":"2024-04-17 22:29:07.807733","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.823835","last_change":"2024-04-17 22:29:07.819432","last_active":"2024-04-17 22:29:07.823835","last_peered":"2024-04-17 22:29:07.823835","last_clean":"2024-04-17 22:29:07.823835","last_became_active":"2024-04-17 22:29:07.819001","last_became_peered":"2024-04-17 22:29:07.819001","last_unstale":"2024-04-17 22:29:07.823835","last_undegraded":"2024-04-17 22:29:07.823835","last_fullsized":"2024-04-17 22:29:07.823835","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3f","version":"15'3","reported_seq":"13","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.829638","last_change":"2024-04-17 22:29:07.466765","last_active":"2024-04-17 22:29:07.829638","last_peered":"2024-04-17 22:29:07.829638","last_clean":"2024-04-17 22:29:07.829638","last_became_active":"2024-04-17 22:29:07.463177","last_became_peered":"2024-04-17 22:29:07.463177","last_unstale":"2024-04-17 22:29:07.829638","last_undegraded":"2024-04-17 22:29:07.829638","last_fullsized":"2024-04-17 22:29:07.829638","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":544,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":4},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":4},{"poolid":2,"num_pg":64,"stat_sum":{"num_bytes":2286,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":128,"acting":128,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705669,"num_pgs":82,"num_osds":1,"num_per_pool_osds":1,"kb":94371840,"kb_used":1048744,"kb_used_data":160,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323096,"statfs":{"total":96636764160,"available":95562850304,"internally_reserved":1073741824,"allocated":163840,"data_stored":41000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.19900000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27500000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.22900000000000001}]}]},{"osd":2,"up_from":9,"seq":38654705669,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"kb":94371840,"kb_used":1048732,"kb_used_data":148,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323108,"statfs":{"total":96636764160,"available":95562862592,"internally_reserved":1073741824,"allocated":151552,"data_stored":39984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.129}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.217}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.20100000000000001}]}]},{"osd":0,"up_from":9,"seq":38654705669,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"kb":94371840,"kb_used":1048736,"kb_used_data":152,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323104,"statfs":{"total":96636764160,"available":95562858496,"internally_reserved":1073741824,"allocated":155648,"data_stored":40772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29399999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.31}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.307}]}]},{"osd":1,"up_from":9,"seq":38654705668,"num_pgs":63,"num_osds":1,"num_per_pool_osds":1,"kb":94371840,"kb_used":1048732,"kb_used_data":148,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323108,"statfs":{"total":96636764160,"available":95562862592,"internally_reserved":1073741824,"allocated":151552,"data_stored":39936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27300000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.307}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33000000000000002}]}]}]}} 2024-04-17T22:29:14.919 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-17T22:29:15.060 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.072 7fbd522a6700 1 Processor -- start 2024-04-17T22:29:15.061 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.072 7fbd522a6700 1 -- start start 2024-04-17T22:29:15.061 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.072 7fbd522a6700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fbd4c10e260 0x7fbd4c10e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:15.061 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.072 7fbd522a6700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fbd4c10ece0 0x7fbd4c0f1750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:15.061 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.072 7fbd522a6700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fbd4c0f1d80 0x7fbd4c1aa4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:15.061 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.072 7fbd522a6700 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd4c0f9f30 con 0x7fbd4c10ece0 2024-04-17T22:29:15.061 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.072 7fbd522a6700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd4c0f9050 con 0x7fbd4c10e260 2024-04-17T22:29:15.061 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.072 7fbd522a6700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbd4c0fa1d0 con 0x7fbd4c0f1d80 2024-04-17T22:29:15.061 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.072 7fbd4b887700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fbd4c0f1d80 0x7fbd4c1aa4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:15.061 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.072 7fbd4a885700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fbd4c10ece0 0x7fbd4c0f1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:15.061 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.072 7fbd4b086700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fbd4c10e260 0x7fbd4c10e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:15.061 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.072 7fbd4b887700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fbd4c0f1d80 0x7fbd4c1aa4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:44080/0 (socket says 172.21.15.102:44080) 2024-04-17T22:29:15.062 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.072 7fbd4a885700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fbd4c10ece0 0x7fbd4c0f1750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:40716/0 (socket says 172.21.15.102:40716) 2024-04-17T22:29:15.062 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.072 7fbd4b887700 1 -- 172.21.15.102:0/3091662655 learned_addr learned my addr 172.21.15.102:0/3091662655 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:29:15.062 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.073 7fbd4b887700 1 -- 172.21.15.102:0/3091662655 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fbd4c10e260 msgr2=0x7fbd4c10e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:29:15.062 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.073 7fbd4b887700 1 --2- 172.21.15.102:0/3091662655 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fbd4c10e260 0x7fbd4c10e6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:15.062 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.073 7fbd4b887700 1 -- 172.21.15.102:0/3091662655 >> v2:172.21.15.102:3300/0 conn(0x7fbd4c10ece0 msgr2=0x7fbd4c0f1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:29:15.062 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.073 7fbd4b887700 1 --2- 172.21.15.102:0/3091662655 >> v2:172.21.15.102:3300/0 conn(0x7fbd4c10ece0 0x7fbd4c0f1750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:15.062 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.073 7fbd4b887700 1 -- 172.21.15.102:0/3091662655 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd4c0f21d0 con 0x7fbd4c0f1d80 2024-04-17T22:29:15.062 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.073 7fbd4a885700 1 --2- 172.21.15.102:0/3091662655 >> v2:172.21.15.102:3300/0 conn(0x7fbd4c10ece0 0x7fbd4c0f1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:29:15.062 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.073 7fbd4b086700 1 --2- 172.21.15.102:0/3091662655 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fbd4c10e260 0x7fbd4c10e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:29:15.062 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.073 7fbd4b887700 1 --2- 172.21.15.102:0/3091662655 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fbd4c0f1d80 0x7fbd4c1aa4f0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7fbd4c0f4490 tx=0x7fbd44009dc0).ready entity=mon.2 client_cookie=bfb3c429e75b03f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:29:15.062 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.073 7fbd4a084700 1 -- 172.21.15.102:0/3091662655 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fbd4c0f21d0 con 0x7fbd4c0f1d80 2024-04-17T22:29:15.062 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.073 7fbd4a084700 1 -- 172.21.15.102:0/3091662655 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd4400b8d0 con 0x7fbd4c0f1d80 2024-04-17T22:29:15.062 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.073 7fbd4a084700 1 -- 172.21.15.102:0/3091662655 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fbd4400bab0 con 0x7fbd4c0f1d80 2024-04-17T22:29:15.062 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.073 7fbd522a6700 1 -- 172.21.15.102:0/3091662655 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fbd4c0f1d80 msgr2=0x7fbd4c1aa4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:15.063 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.073 7fbd522a6700 1 --2- 172.21.15.102:0/3091662655 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fbd4c0f1d80 0x7fbd4c1aa4f0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7fbd4c0f4490 tx=0x7fbd44009dc0).stop 2024-04-17T22:29:15.063 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.074 7fbd522a6700 1 -- 172.21.15.102:0/3091662655 shutdown_connections 2024-04-17T22:29:15.063 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.074 7fbd522a6700 1 --2- 172.21.15.102:0/3091662655 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fbd4c0f1d80 0x7fbd4c1aa4f0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:15.063 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.074 7fbd522a6700 1 --2- 172.21.15.102:0/3091662655 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fbd4c10e260 0x7fbd4c10e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:15.063 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.074 7fbd522a6700 1 --2- 172.21.15.102:0/3091662655 >> v2:172.21.15.102:3300/0 conn(0x7fbd4c10ece0 0x7fbd4c0f1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:15.063 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.074 7fbd522a6700 1 -- 172.21.15.102:0/3091662655 >> 172.21.15.102:0/3091662655 conn(0x7fbd4c1138c0 msgr2=0x7fbd4c0f4f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:29:15.063 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.074 7fbd522a6700 1 -- 172.21.15.102:0/3091662655 shutdown_connections 2024-04-17T22:29:15.063 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.074 7fbd522a6700 1 -- 172.21.15.102:0/3091662655 wait complete. 2024-04-17T22:29:15.083 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.096 7fbd522a6700 1 Processor -- start 2024-04-17T22:29:15.083 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.097 7fbd522a6700 1 -- start start 2024-04-17T22:29:15.085 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.097 7fbd522a6700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fbd4c0cd910 0x7fbd4c1ad2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:15.085 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.097 7fbd522a6700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbd4c1ad8f0 0x7fbd4c1b3c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:15.085 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.097 7fbd522a6700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fbd4c1add40 0x7fbd4c1b42b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:15.086 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.097 7fbd522a6700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd4c0ccff0 con 0x7fbd4c1ad8f0 2024-04-17T22:29:15.086 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.097 7fbd522a6700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd4c0ccbf0 con 0x7fbd4c1add40 2024-04-17T22:29:15.086 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.097 7fbd522a6700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbd4c1b48e0 con 0x7fbd4c0cd910 2024-04-17T22:29:15.086 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.097 7fbd4a885700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbd4c1ad8f0 0x7fbd4c1b3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:15.086 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.097 7fbd4b887700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fbd4c1add40 0x7fbd4c1b42b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:15.086 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.097 7fbd4a885700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbd4c1ad8f0 0x7fbd4c1b3c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:40718/0 (socket says 172.21.15.102:40718) 2024-04-17T22:29:15.086 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.097 7fbd4a885700 1 -- 172.21.15.102:0/2610367358 learned_addr learned my addr 172.21.15.102:0/2610367358 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:29:15.086 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.097 7fbd4b086700 1 --2- 172.21.15.102:0/2610367358 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fbd4c0cd910 0x7fbd4c1ad2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:15.086 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.097 7fbd4a885700 1 -- 172.21.15.102:0/2610367358 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fbd4c0cd910 msgr2=0x7fbd4c1ad2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:15.086 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.097 7fbd4a885700 1 --2- 172.21.15.102:0/2610367358 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fbd4c0cd910 0x7fbd4c1ad2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:15.086 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.097 7fbd4a885700 1 -- 172.21.15.102:0/2610367358 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fbd4c1add40 msgr2=0x7fbd4c1b42b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:15.086 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.097 7fbd4a885700 1 --2- 172.21.15.102:0/2610367358 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fbd4c1add40 0x7fbd4c1b42b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:15.086 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.097 7fbd4a885700 1 -- 172.21.15.102:0/2610367358 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd4c1b4b10 con 0x7fbd4c1ad8f0 2024-04-17T22:29:15.087 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.097 7fbd4b086700 1 --2- 172.21.15.102:0/2610367358 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fbd4c0cd910 0x7fbd4c1ad2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:29:15.087 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.097 7fbd4b887700 1 --2- 172.21.15.102:0/2610367358 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fbd4c1add40 0x7fbd4c1b42b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:29:15.087 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.098 7fbd4a885700 1 --2- 172.21.15.102:0/2610367358 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbd4c1ad8f0 0x7fbd4c1b3c80 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 rx=0x7fbd3c00c000 tx=0x7fbd3c00ecd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:29:15.087 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.098 7fbd48881700 1 -- 172.21.15.102:0/2610367358 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fbd4c1b4b10 con 0x7fbd4c1ad8f0 2024-04-17T22:29:15.087 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.098 7fbd48881700 1 -- 172.21.15.102:0/2610367358 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd3c00c7b0 con 0x7fbd4c1ad8f0 2024-04-17T22:29:15.087 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.098 7fbd522a6700 1 -- 172.21.15.102:0/2610367358 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd4c1bb3f0 con 0x7fbd4c1ad8f0 2024-04-17T22:29:15.087 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.098 7fbd48881700 1 -- 172.21.15.102:0/2610367358 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fbd3c00f040 con 0x7fbd4c1ad8f0 2024-04-17T22:29:15.087 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.098 7fbd522a6700 1 -- 172.21.15.102:0/2610367358 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd4c1bb7d0 con 0x7fbd4c1ad8f0 2024-04-17T22:29:15.087 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.098 7fbd48881700 1 -- 172.21.15.102:0/2610367358 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7fbd3c011070 con 0x7fbd4c1ad8f0 2024-04-17T22:29:15.087 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.098 7fbd48881700 1 -- 172.21.15.102:0/2610367358 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 3194+0+0 (secure 0 0 0) 0x7fbd3c010040 con 0x7fbd4c1ad8f0 2024-04-17T22:29:15.087 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.099 7fbd522a6700 1 -- 172.21.15.102:0/2610367358 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd4c0f0820 con 0x7fbd4c1ad8f0 2024-04-17T22:29:15.088 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.102 7fbd48881700 1 -- 172.21.15.102:0/2610367358 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+110455 (secure 0 0 0) 0x7fbd3c000c30 con 0x7fbd4c1ad8f0 2024-04-17T22:29:15.190 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.203 7fbd522a6700 1 -- 172.21.15.102:0/2610367358 --> v1:172.21.15.102:6816/42360 -- command(tid 0: {"prefix": "pg dump", "target": ["mgr", ""], "format": "json"}) v1 -- 0x7fbd4c0cd340 con 0x7fbd2c0145a0 2024-04-17T22:29:15.197 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.211 7fbd48881700 1 -- 172.21.15.102:0/2610367358 <== mgr.4104 v1:172.21.15.102:6816/42360 1 ==== command_reply(tid 0: 0 dumped all) v1 ==== 18+0+306111 (unknown 2965378022 0 336623254) 0x7fbd4000bae0 con 0x7fbd2c0145a0 2024-04-17T22:29:15.201 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-17T22:29:15.204 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.213 7fbd522a6700 1 -- 172.21.15.102:0/2610367358 >> v1:172.21.15.102:6816/42360 conn(0x7fbd2c0145a0 legacy=0x7fbd2c016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:15.204 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.213 7fbd522a6700 1 -- 172.21.15.102:0/2610367358 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbd4c1ad8f0 msgr2=0x7fbd4c1b3c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:15.204 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.213 7fbd522a6700 1 --2- 172.21.15.102:0/2610367358 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbd4c1ad8f0 0x7fbd4c1b3c80 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 rx=0x7fbd3c00c000 tx=0x7fbd3c00ecd0).stop 2024-04-17T22:29:15.204 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.213 7fbd522a6700 1 -- 172.21.15.102:0/2610367358 shutdown_connections 2024-04-17T22:29:15.204 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.213 7fbd522a6700 1 --2- 172.21.15.102:0/2610367358 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fbd4c1add40 0x7fbd4c1b42b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:15.204 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.213 7fbd522a6700 1 --2- 172.21.15.102:0/2610367358 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fbd4c0cd910 0x7fbd4c1ad2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:15.204 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.213 7fbd522a6700 1 --2- 172.21.15.102:0/2610367358 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbd4c1ad8f0 0x7fbd4c1b3c80 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:15.204 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.213 7fbd522a6700 1 -- 172.21.15.102:0/2610367358 >> 172.21.15.102:0/2610367358 conn(0x7fbd4c0e0dc0 msgr2=0x7fbd4c0daae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:29:15.204 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.213 7fbd522a6700 1 -- 172.21.15.102:0/2610367358 shutdown_connections 2024-04-17T22:29:15.204 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.213 7fbd522a6700 1 -- 172.21.15.102:0/2610367358 wait complete. 2024-04-17T22:29:15.204 INFO:teuthology.orchestra.run.smithi102.stderr:dumped all 2024-04-17T22:29:15.210 INFO:teuthology.orchestra.run.smithi102.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2024-04-17 22:29:15.188500","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2286,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":272,"acting":272,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":152,"num_osds":4,"num_per_pool_osds":4,"kb":377487360,"kb_used":4194944,"kb_used_data":608,"kb_used_omap":0,"kb_used_meta":4194304,"kb_avail":373292416,"statfs":{"total":386547056640,"available":382251433984,"internally_reserved":4294967296,"allocated":622592,"data_stored":161692,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4294967296},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000937"},"pg_stats":[{"pgid":"3.1a","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.820762","last_change":"2024-04-17 22:29:07.814049","last_active":"2024-04-17 22:29:07.820762","last_peered":"2024-04-17 22:29:07.820762","last_clean":"2024-04-17 22:29:07.820762","last_became_active":"2024-04-17 22:29:07.811049","last_became_peered":"2024-04-17 22:29:07.811049","last_unstale":"2024-04-17 22:29:07.820762","last_undegraded":"2024-04-17 22:29:07.820762","last_fullsized":"2024-04-17 22:29:07.820762","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.807568","last_change":"2024-04-17 22:29:07.477053","last_active":"2024-04-17 22:29:07.807568","last_peered":"2024-04-17 22:29:07.807568","last_clean":"2024-04-17 22:29:07.807568","last_became_active":"2024-04-17 22:29:07.476568","last_became_peered":"2024-04-17 22:29:07.476568","last_unstale":"2024-04-17 22:29:07.807568","last_undegraded":"2024-04-17 22:29:07.807568","last_fullsized":"2024-04-17 22:29:07.807568","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.828628","last_change":"2024-04-17 22:29:07.818787","last_active":"2024-04-17 22:29:07.828628","last_peered":"2024-04-17 22:29:07.828628","last_clean":"2024-04-17 22:29:07.828628","last_became_active":"2024-04-17 22:29:07.818347","last_became_peered":"2024-04-17 22:29:07.818347","last_unstale":"2024-04-17 22:29:07.828628","last_undegraded":"2024-04-17 22:29:07.828628","last_fullsized":"2024-04-17 22:29:07.828628","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.805356","last_change":"2024-04-17 22:29:07.477933","last_active":"2024-04-17 22:29:07.805356","last_peered":"2024-04-17 22:29:07.805356","last_clean":"2024-04-17 22:29:07.805356","last_became_active":"2024-04-17 22:29:07.472128","last_became_peered":"2024-04-17 22:29:07.472128","last_unstale":"2024-04-17 22:29:07.805356","last_undegraded":"2024-04-17 22:29:07.805356","last_fullsized":"2024-04-17 22:29:07.805356","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.834368","last_change":"2024-04-17 22:29:07.833107","last_active":"2024-04-17 22:29:07.834368","last_peered":"2024-04-17 22:29:07.834368","last_clean":"2024-04-17 22:29:07.834368","last_became_active":"2024-04-17 22:29:07.832822","last_became_peered":"2024-04-17 22:29:07.832822","last_unstale":"2024-04-17 22:29:07.834368","last_undegraded":"2024-04-17 22:29:07.834368","last_fullsized":"2024-04-17 22:29:07.834368","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.802997","last_change":"2024-04-17 22:29:07.471612","last_active":"2024-04-17 22:29:07.802997","last_peered":"2024-04-17 22:29:07.802997","last_clean":"2024-04-17 22:29:07.802997","last_became_active":"2024-04-17 22:29:07.471322","last_became_peered":"2024-04-17 22:29:07.471322","last_unstale":"2024-04-17 22:29:07.802997","last_undegraded":"2024-04-17 22:29:07.802997","last_fullsized":"2024-04-17 22:29:07.802997","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.812548","last_change":"2024-04-17 22:29:07.810081","last_active":"2024-04-17 22:29:07.812548","last_peered":"2024-04-17 22:29:07.812548","last_clean":"2024-04-17 22:29:07.812548","last_became_active":"2024-04-17 22:29:07.809858","last_became_peered":"2024-04-17 22:29:07.809858","last_unstale":"2024-04-17 22:29:07.812548","last_undegraded":"2024-04-17 22:29:07.812548","last_fullsized":"2024-04-17 22:29:07.812548","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.797655","last_change":"2024-04-17 22:29:07.474346","last_active":"2024-04-17 22:29:07.797655","last_peered":"2024-04-17 22:29:07.797655","last_clean":"2024-04-17 22:29:07.797655","last_became_active":"2024-04-17 22:29:07.473783","last_became_peered":"2024-04-17 22:29:07.473783","last_unstale":"2024-04-17 22:29:07.797655","last_undegraded":"2024-04-17 22:29:07.797655","last_fullsized":"2024-04-17 22:29:07.797655","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.825049","last_change":"2024-04-17 22:29:07.811047","last_active":"2024-04-17 22:29:07.825049","last_peered":"2024-04-17 22:29:07.825049","last_clean":"2024-04-17 22:29:07.825049","last_became_active":"2024-04-17 22:29:07.810747","last_became_peered":"2024-04-17 22:29:07.810747","last_unstale":"2024-04-17 22:29:07.825049","last_undegraded":"2024-04-17 22:29:07.825049","last_fullsized":"2024-04-17 22:29:07.825049","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.798589","last_change":"2024-04-17 22:29:07.460228","last_active":"2024-04-17 22:29:07.798589","last_peered":"2024-04-17 22:29:07.798589","last_clean":"2024-04-17 22:29:07.798589","last_became_active":"2024-04-17 22:29:07.459910","last_became_peered":"2024-04-17 22:29:07.459910","last_unstale":"2024-04-17 22:29:07.798589","last_undegraded":"2024-04-17 22:29:07.798589","last_fullsized":"2024-04-17 22:29:07.798589","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.830592","last_change":"2024-04-17 22:29:07.827143","last_active":"2024-04-17 22:29:07.830592","last_peered":"2024-04-17 22:29:07.830592","last_clean":"2024-04-17 22:29:07.830592","last_became_active":"2024-04-17 22:29:07.826842","last_became_peered":"2024-04-17 22:29:07.826842","last_unstale":"2024-04-17 22:29:07.830592","last_undegraded":"2024-04-17 22:29:07.830592","last_fullsized":"2024-04-17 22:29:07.830592","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.807458","last_change":"2024-04-17 22:29:07.477248","last_active":"2024-04-17 22:29:07.807458","last_peered":"2024-04-17 22:29:07.807458","last_clean":"2024-04-17 22:29:07.807458","last_became_active":"2024-04-17 22:29:07.476944","last_became_peered":"2024-04-17 22:29:07.476944","last_unstale":"2024-04-17 22:29:07.807458","last_undegraded":"2024-04-17 22:29:07.807458","last_fullsized":"2024-04-17 22:29:07.807458","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.826061","last_change":"2024-04-17 22:29:07.822961","last_active":"2024-04-17 22:29:07.826061","last_peered":"2024-04-17 22:29:07.826061","last_clean":"2024-04-17 22:29:07.826061","last_became_active":"2024-04-17 22:29:07.822592","last_became_peered":"2024-04-17 22:29:07.822592","last_unstale":"2024-04-17 22:29:07.826061","last_undegraded":"2024-04-17 22:29:07.826061","last_fullsized":"2024-04-17 22:29:07.826061","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.15","version":"15'3","reported_seq":"13","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.826924","last_change":"2024-04-17 22:29:07.485166","last_active":"2024-04-17 22:29:07.826924","last_peered":"2024-04-17 22:29:07.826924","last_clean":"2024-04-17 22:29:07.826924","last_became_active":"2024-04-17 22:29:07.484870","last_became_peered":"2024-04-17 22:29:07.484870","last_unstale":"2024-04-17 22:29:07.826924","last_undegraded":"2024-04-17 22:29:07.826924","last_fullsized":"2024-04-17 22:29:07.826924","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.825665","last_change":"2024-04-17 22:29:07.814509","last_active":"2024-04-17 22:29:07.825665","last_peered":"2024-04-17 22:29:07.825665","last_clean":"2024-04-17 22:29:07.825665","last_became_active":"2024-04-17 22:29:07.814119","last_became_peered":"2024-04-17 22:29:07.814119","last_unstale":"2024-04-17 22:29:07.825665","last_undegraded":"2024-04-17 22:29:07.825665","last_fullsized":"2024-04-17 22:29:07.825665","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.14","version":"15'2","reported_seq":"12","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.823699","last_change":"2024-04-17 22:29:07.473309","last_active":"2024-04-17 22:29:07.823699","last_peered":"2024-04-17 22:29:07.823699","last_clean":"2024-04-17 22:29:07.823699","last_became_active":"2024-04-17 22:29:07.472976","last_became_peered":"2024-04-17 22:29:07.472976","last_unstale":"2024-04-17 22:29:07.823699","last_undegraded":"2024-04-17 22:29:07.823699","last_fullsized":"2024-04-17 22:29:07.823699","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":634,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.832027","last_change":"2024-04-17 22:29:07.827915","last_active":"2024-04-17 22:29:07.832027","last_peered":"2024-04-17 22:29:07.832027","last_clean":"2024-04-17 22:29:07.832027","last_became_active":"2024-04-17 22:29:07.827670","last_became_peered":"2024-04-17 22:29:07.827670","last_unstale":"2024-04-17 22:29:07.832027","last_undegraded":"2024-04-17 22:29:07.832027","last_fullsized":"2024-04-17 22:29:07.832027","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"15'1","reported_seq":"11","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.827199","last_change":"2024-04-17 22:29:07.471458","last_active":"2024-04-17 22:29:07.827199","last_peered":"2024-04-17 22:29:07.827199","last_clean":"2024-04-17 22:29:07.827199","last_became_active":"2024-04-17 22:29:07.471085","last_became_peered":"2024-04-17 22:29:07.471085","last_unstale":"2024-04-17 22:29:07.827199","last_undegraded":"2024-04-17 22:29:07.827199","last_fullsized":"2024-04-17 22:29:07.827199","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.833535","last_change":"2024-04-17 22:29:07.829288","last_active":"2024-04-17 22:29:07.833535","last_peered":"2024-04-17 22:29:07.833535","last_clean":"2024-04-17 22:29:07.833535","last_became_active":"2024-04-17 22:29:07.828994","last_became_peered":"2024-04-17 22:29:07.828994","last_unstale":"2024-04-17 22:29:07.833535","last_undegraded":"2024-04-17 22:29:07.833535","last_fullsized":"2024-04-17 22:29:07.833535","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.805144","last_change":"2024-04-17 22:29:07.480194","last_active":"2024-04-17 22:29:07.805144","last_peered":"2024-04-17 22:29:07.805144","last_clean":"2024-04-17 22:29:07.805144","last_became_active":"2024-04-17 22:29:07.479728","last_became_peered":"2024-04-17 22:29:07.479728","last_unstale":"2024-04-17 22:29:07.805144","last_undegraded":"2024-04-17 22:29:07.805144","last_fullsized":"2024-04-17 22:29:07.805144","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.820915","last_change":"2024-04-17 22:29:07.815790","last_active":"2024-04-17 22:29:07.820915","last_peered":"2024-04-17 22:29:07.820915","last_clean":"2024-04-17 22:29:07.820915","last_became_active":"2024-04-17 22:29:07.815632","last_became_peered":"2024-04-17 22:29:07.815632","last_unstale":"2024-04-17 22:29:07.820915","last_undegraded":"2024-04-17 22:29:07.820915","last_fullsized":"2024-04-17 22:29:07.820915","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.804035","last_change":"2024-04-17 22:29:07.475830","last_active":"2024-04-17 22:29:07.804035","last_peered":"2024-04-17 22:29:07.804035","last_clean":"2024-04-17 22:29:07.804035","last_became_active":"2024-04-17 22:29:07.474411","last_became_peered":"2024-04-17 22:29:07.474411","last_unstale":"2024-04-17 22:29:07.804035","last_undegraded":"2024-04-17 22:29:07.804035","last_fullsized":"2024-04-17 22:29:07.804035","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.828447","last_change":"2024-04-17 22:29:07.826101","last_active":"2024-04-17 22:29:07.828447","last_peered":"2024-04-17 22:29:07.828447","last_clean":"2024-04-17 22:29:07.828447","last_became_active":"2024-04-17 22:29:07.822917","last_became_peered":"2024-04-17 22:29:07.822917","last_unstale":"2024-04-17 22:29:07.828447","last_undegraded":"2024-04-17 22:29:07.828447","last_fullsized":"2024-04-17 22:29:07.828447","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.10","version":"15'4","reported_seq":"14","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.817471","last_change":"2024-04-17 22:29:07.457017","last_active":"2024-04-17 22:29:07.817471","last_peered":"2024-04-17 22:29:07.817471","last_clean":"2024-04-17 22:29:07.817471","last_became_active":"2024-04-17 22:29:07.456705","last_became_peered":"2024-04-17 22:29:07.456705","last_unstale":"2024-04-17 22:29:07.817471","last_undegraded":"2024-04-17 22:29:07.817471","last_fullsized":"2024-04-17 22:29:07.817471","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.825122","last_change":"2024-04-17 22:29:07.820464","last_active":"2024-04-17 22:29:07.825122","last_peered":"2024-04-17 22:29:07.825122","last_clean":"2024-04-17 22:29:07.825122","last_became_active":"2024-04-17 22:29:07.820151","last_became_peered":"2024-04-17 22:29:07.820151","last_unstale":"2024-04-17 22:29:07.825122","last_undegraded":"2024-04-17 22:29:07.825122","last_fullsized":"2024-04-17 22:29:07.825122","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.f","version":"15'1","reported_seq":"11","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.828338","last_change":"2024-04-17 22:29:07.478464","last_active":"2024-04-17 22:29:07.828338","last_peered":"2024-04-17 22:29:07.828338","last_clean":"2024-04-17 22:29:07.828338","last_became_active":"2024-04-17 22:29:07.478144","last_became_peered":"2024-04-17 22:29:07.478144","last_unstale":"2024-04-17 22:29:07.828338","last_undegraded":"2024-04-17 22:29:07.828338","last_fullsized":"2024-04-17 22:29:07.828338","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.828463","last_change":"2024-04-17 22:29:07.815171","last_active":"2024-04-17 22:29:07.828463","last_peered":"2024-04-17 22:29:07.828463","last_clean":"2024-04-17 22:29:07.828463","last_became_active":"2024-04-17 22:29:07.814830","last_became_peered":"2024-04-17 22:29:07.814830","last_unstale":"2024-04-17 22:29:07.828463","last_undegraded":"2024-04-17 22:29:07.828463","last_fullsized":"2024-04-17 22:29:07.828463","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"15'2","reported_seq":"12","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.811237","last_change":"2024-04-17 22:29:07.481865","last_active":"2024-04-17 22:29:07.811237","last_peered":"2024-04-17 22:29:07.811237","last_clean":"2024-04-17 22:29:07.811237","last_became_active":"2024-04-17 22:29:07.481506","last_became_peered":"2024-04-17 22:29:07.481506","last_unstale":"2024-04-17 22:29:07.811237","last_undegraded":"2024-04-17 22:29:07.811237","last_fullsized":"2024-04-17 22:29:07.811237","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.828178","last_change":"2024-04-17 22:29:07.818238","last_active":"2024-04-17 22:29:07.828178","last_peered":"2024-04-17 22:29:07.828178","last_clean":"2024-04-17 22:29:07.828178","last_became_active":"2024-04-17 22:29:07.818060","last_became_peered":"2024-04-17 22:29:07.818060","last_unstale":"2024-04-17 22:29:07.828178","last_undegraded":"2024-04-17 22:29:07.828178","last_fullsized":"2024-04-17 22:29:07.828178","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.796955","last_change":"2024-04-17 22:29:07.484999","last_active":"2024-04-17 22:29:07.796955","last_peered":"2024-04-17 22:29:07.796955","last_clean":"2024-04-17 22:29:07.796955","last_became_active":"2024-04-17 22:29:07.484668","last_became_peered":"2024-04-17 22:29:07.484668","last_unstale":"2024-04-17 22:29:07.796955","last_undegraded":"2024-04-17 22:29:07.796955","last_fullsized":"2024-04-17 22:29:07.796955","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.828594","last_change":"2024-04-17 22:29:07.826426","last_active":"2024-04-17 22:29:07.828594","last_peered":"2024-04-17 22:29:07.828594","last_clean":"2024-04-17 22:29:07.828594","last_became_active":"2024-04-17 22:29:07.826109","last_became_peered":"2024-04-17 22:29:07.826109","last_unstale":"2024-04-17 22:29:07.828594","last_undegraded":"2024-04-17 22:29:07.828594","last_fullsized":"2024-04-17 22:29:07.828594","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.797207","last_change":"2024-04-17 22:29:07.462218","last_active":"2024-04-17 22:29:07.797207","last_peered":"2024-04-17 22:29:07.797207","last_clean":"2024-04-17 22:29:07.797207","last_became_active":"2024-04-17 22:29:07.461842","last_became_peered":"2024-04-17 22:29:07.461842","last_unstale":"2024-04-17 22:29:07.797207","last_undegraded":"2024-04-17 22:29:07.797207","last_fullsized":"2024-04-17 22:29:07.797207","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.820296","last_change":"2024-04-17 22:29:07.817144","last_active":"2024-04-17 22:29:07.820296","last_peered":"2024-04-17 22:29:07.820296","last_clean":"2024-04-17 22:29:07.820296","last_became_active":"2024-04-17 22:29:07.816859","last_became_peered":"2024-04-17 22:29:07.816859","last_unstale":"2024-04-17 22:29:07.820296","last_undegraded":"2024-04-17 22:29:07.820296","last_fullsized":"2024-04-17 22:29:07.820296","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"15'1","reported_seq":"11","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.821604","last_change":"2024-04-17 22:29:07.479816","last_active":"2024-04-17 22:29:07.821604","last_peered":"2024-04-17 22:29:07.821604","last_clean":"2024-04-17 22:29:07.821604","last_became_active":"2024-04-17 22:29:07.479251","last_became_peered":"2024-04-17 22:29:07.479251","last_unstale":"2024-04-17 22:29:07.821604","last_undegraded":"2024-04-17 22:29:07.821604","last_fullsized":"2024-04-17 22:29:07.821604","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.833516","last_change":"2024-04-17 22:29:07.829215","last_active":"2024-04-17 22:29:07.833516","last_peered":"2024-04-17 22:29:07.833516","last_clean":"2024-04-17 22:29:07.833516","last_became_active":"2024-04-17 22:29:07.828803","last_became_peered":"2024-04-17 22:29:07.828803","last_unstale":"2024-04-17 22:29:07.833516","last_undegraded":"2024-04-17 22:29:07.833516","last_fullsized":"2024-04-17 22:29:07.833516","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.796428","last_change":"2024-04-17 22:29:07.473128","last_active":"2024-04-17 22:29:07.796428","last_peered":"2024-04-17 22:29:07.796428","last_clean":"2024-04-17 22:29:07.796428","last_became_active":"2024-04-17 22:29:07.472806","last_became_peered":"2024-04-17 22:29:07.472806","last_unstale":"2024-04-17 22:29:07.796428","last_undegraded":"2024-04-17 22:29:07.796428","last_fullsized":"2024-04-17 22:29:07.796428","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.820555","last_change":"2024-04-17 22:29:07.817630","last_active":"2024-04-17 22:29:07.820555","last_peered":"2024-04-17 22:29:07.820555","last_clean":"2024-04-17 22:29:07.820555","last_became_active":"2024-04-17 22:29:07.817333","last_became_peered":"2024-04-17 22:29:07.817333","last_unstale":"2024-04-17 22:29:07.820555","last_undegraded":"2024-04-17 22:29:07.820555","last_fullsized":"2024-04-17 22:29:07.820555","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.792279","last_change":"2024-04-17 22:29:07.465053","last_active":"2024-04-17 22:29:07.792279","last_peered":"2024-04-17 22:29:07.792279","last_clean":"2024-04-17 22:29:07.792279","last_became_active":"2024-04-17 22:29:07.464766","last_became_peered":"2024-04-17 22:29:07.464766","last_unstale":"2024-04-17 22:29:07.792279","last_undegraded":"2024-04-17 22:29:07.792279","last_fullsized":"2024-04-17 22:29:07.792279","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.827066","last_change":"2024-04-17 22:29:07.823514","last_active":"2024-04-17 22:29:07.827066","last_peered":"2024-04-17 22:29:07.827066","last_clean":"2024-04-17 22:29:07.827066","last_became_active":"2024-04-17 22:29:07.823007","last_became_peered":"2024-04-17 22:29:07.823007","last_unstale":"2024-04-17 22:29:07.827066","last_undegraded":"2024-04-17 22:29:07.827066","last_fullsized":"2024-04-17 22:29:07.827066","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.801156","last_change":"2024-04-17 22:29:07.461568","last_active":"2024-04-17 22:29:07.801156","last_peered":"2024-04-17 22:29:07.801156","last_clean":"2024-04-17 22:29:07.801156","last_became_active":"2024-04-17 22:29:07.461249","last_became_peered":"2024-04-17 22:29:07.461249","last_unstale":"2024-04-17 22:29:07.801156","last_undegraded":"2024-04-17 22:29:07.801156","last_fullsized":"2024-04-17 22:29:07.801156","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.828426","last_change":"2024-04-17 22:29:07.826342","last_active":"2024-04-17 22:29:07.828426","last_peered":"2024-04-17 22:29:07.828426","last_clean":"2024-04-17 22:29:07.828426","last_became_active":"2024-04-17 22:29:07.826029","last_became_peered":"2024-04-17 22:29:07.826029","last_unstale":"2024-04-17 22:29:07.828426","last_undegraded":"2024-04-17 22:29:07.828426","last_fullsized":"2024-04-17 22:29:07.828426","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":"13","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.795463","last_change":"2024-04-17 22:28:59.253977","last_active":"2024-04-17 22:29:07.795463","last_peered":"2024-04-17 22:29:07.795463","last_clean":"2024-04-17 22:29:07.795463","last_became_active":"2024-04-17 22:28:59.253641","last_became_peered":"2024-04-17 22:28:59.253641","last_unstale":"2024-04-17 22:29:07.795463","last_undegraded":"2024-04-17 22:29:07.795463","last_fullsized":"2024-04-17 22:29:07.795463","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:28:58.239187","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:28:58.239187","last_clean_scrub_stamp":"2024-04-17 22:28:58.239187","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.796951","last_change":"2024-04-17 22:29:07.466907","last_active":"2024-04-17 22:29:07.796951","last_peered":"2024-04-17 22:29:07.796951","last_clean":"2024-04-17 22:29:07.796951","last_became_active":"2024-04-17 22:29:07.466583","last_became_peered":"2024-04-17 22:29:07.466583","last_unstale":"2024-04-17 22:29:07.796951","last_undegraded":"2024-04-17 22:29:07.796951","last_fullsized":"2024-04-17 22:29:07.796951","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.829316","last_change":"2024-04-17 22:29:07.824875","last_active":"2024-04-17 22:29:07.829316","last_peered":"2024-04-17 22:29:07.829316","last_clean":"2024-04-17 22:29:07.829316","last_became_active":"2024-04-17 22:29:07.824377","last_became_peered":"2024-04-17 22:29:07.824377","last_unstale":"2024-04-17 22:29:07.829316","last_undegraded":"2024-04-17 22:29:07.829316","last_fullsized":"2024-04-17 22:29:07.829316","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":"13","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.809532","last_change":"2024-04-17 22:28:59.255733","last_active":"2024-04-17 22:29:07.809532","last_peered":"2024-04-17 22:29:07.809532","last_clean":"2024-04-17 22:29:07.809532","last_became_active":"2024-04-17 22:28:59.255411","last_became_peered":"2024-04-17 22:28:59.255411","last_unstale":"2024-04-17 22:29:07.809532","last_undegraded":"2024-04-17 22:29:07.809532","last_fullsized":"2024-04-17 22:29:07.809532","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:28:58.239187","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:28:58.239187","last_clean_scrub_stamp":"2024-04-17 22:28:58.239187","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"15'2","reported_seq":"12","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.832132","last_change":"2024-04-17 22:29:07.478196","last_active":"2024-04-17 22:29:07.832132","last_peered":"2024-04-17 22:29:07.832132","last_clean":"2024-04-17 22:29:07.832132","last_became_active":"2024-04-17 22:29:07.477887","last_became_peered":"2024-04-17 22:29:07.477887","last_unstale":"2024-04-17 22:29:07.832132","last_undegraded":"2024-04-17 22:29:07.832132","last_fullsized":"2024-04-17 22:29:07.832132","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":"13","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.796430","last_change":"2024-04-17 22:28:59.252728","last_active":"2024-04-17 22:29:07.796430","last_peered":"2024-04-17 22:29:07.796430","last_clean":"2024-04-17 22:29:07.796430","last_became_active":"2024-04-17 22:28:59.252345","last_became_peered":"2024-04-17 22:28:59.252345","last_unstale":"2024-04-17 22:29:07.796430","last_undegraded":"2024-04-17 22:29:07.796430","last_fullsized":"2024-04-17 22:29:07.796430","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:28:58.239187","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:28:58.239187","last_clean_scrub_stamp":"2024-04-17 22:28:58.239187","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.797727","last_change":"2024-04-17 22:29:07.474520","last_active":"2024-04-17 22:29:07.797727","last_peered":"2024-04-17 22:29:07.797727","last_clean":"2024-04-17 22:29:07.797727","last_became_active":"2024-04-17 22:29:07.474029","last_became_peered":"2024-04-17 22:29:07.474029","last_unstale":"2024-04-17 22:29:07.797727","last_undegraded":"2024-04-17 22:29:07.797727","last_fullsized":"2024-04-17 22:29:07.797727","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.829302","last_change":"2024-04-17 22:29:07.825091","last_active":"2024-04-17 22:29:07.829302","last_peered":"2024-04-17 22:29:07.829302","last_clean":"2024-04-17 22:29:07.829302","last_became_active":"2024-04-17 22:29:07.824832","last_became_peered":"2024-04-17 22:29:07.824832","last_unstale":"2024-04-17 22:29:07.829302","last_undegraded":"2024-04-17 22:29:07.829302","last_fullsized":"2024-04-17 22:29:07.829302","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":"13","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.807580","last_change":"2024-04-17 22:28:59.253392","last_active":"2024-04-17 22:29:07.807580","last_peered":"2024-04-17 22:29:07.807580","last_clean":"2024-04-17 22:29:07.807580","last_became_active":"2024-04-17 22:28:59.253049","last_became_peered":"2024-04-17 22:28:59.253049","last_unstale":"2024-04-17 22:29:07.807580","last_undegraded":"2024-04-17 22:29:07.807580","last_fullsized":"2024-04-17 22:29:07.807580","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:28:58.239187","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:28:58.239187","last_clean_scrub_stamp":"2024-04-17 22:28:58.239187","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.796512","last_change":"2024-04-17 22:29:07.471913","last_active":"2024-04-17 22:29:07.796512","last_peered":"2024-04-17 22:29:07.796512","last_clean":"2024-04-17 22:29:07.796512","last_became_active":"2024-04-17 22:29:07.471686","last_became_peered":"2024-04-17 22:29:07.471686","last_unstale":"2024-04-17 22:29:07.796512","last_undegraded":"2024-04-17 22:29:07.796512","last_fullsized":"2024-04-17 22:29:07.796512","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.820450","last_change":"2024-04-17 22:29:07.812999","last_active":"2024-04-17 22:29:07.820450","last_peered":"2024-04-17 22:29:07.820450","last_clean":"2024-04-17 22:29:07.820450","last_became_active":"2024-04-17 22:29:07.812701","last_became_peered":"2024-04-17 22:29:07.812701","last_unstale":"2024-04-17 22:29:07.820450","last_undegraded":"2024-04-17 22:29:07.820450","last_fullsized":"2024-04-17 22:29:07.820450","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":"13","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.791848","last_change":"2024-04-17 22:28:59.254907","last_active":"2024-04-17 22:29:07.791848","last_peered":"2024-04-17 22:29:07.791848","last_clean":"2024-04-17 22:29:07.791848","last_became_active":"2024-04-17 22:28:59.254633","last_became_peered":"2024-04-17 22:28:59.254633","last_unstale":"2024-04-17 22:29:07.791848","last_undegraded":"2024-04-17 22:29:07.791848","last_fullsized":"2024-04-17 22:29:07.791848","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:28:58.239187","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:28:58.239187","last_clean_scrub_stamp":"2024-04-17 22:28:58.239187","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.807141","last_change":"2024-04-17 22:29:07.480020","last_active":"2024-04-17 22:29:07.807141","last_peered":"2024-04-17 22:29:07.807141","last_clean":"2024-04-17 22:29:07.807141","last_became_active":"2024-04-17 22:29:07.479492","last_became_peered":"2024-04-17 22:29:07.479492","last_unstale":"2024-04-17 22:29:07.807141","last_undegraded":"2024-04-17 22:29:07.807141","last_fullsized":"2024-04-17 22:29:07.807141","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.826030","last_change":"2024-04-17 22:29:07.821883","last_active":"2024-04-17 22:29:07.826030","last_peered":"2024-04-17 22:29:07.826030","last_clean":"2024-04-17 22:29:07.826030","last_became_active":"2024-04-17 22:29:07.821569","last_became_peered":"2024-04-17 22:29:07.821569","last_unstale":"2024-04-17 22:29:07.826030","last_undegraded":"2024-04-17 22:29:07.826030","last_fullsized":"2024-04-17 22:29:07.826030","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":"13","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.806466","last_change":"2024-04-17 22:28:59.254217","last_active":"2024-04-17 22:29:07.806466","last_peered":"2024-04-17 22:29:07.806466","last_clean":"2024-04-17 22:29:07.806466","last_became_active":"2024-04-17 22:28:59.253877","last_became_peered":"2024-04-17 22:28:59.253877","last_unstale":"2024-04-17 22:29:07.806466","last_undegraded":"2024-04-17 22:29:07.806466","last_fullsized":"2024-04-17 22:29:07.806466","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:28:58.239187","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:28:58.239187","last_clean_scrub_stamp":"2024-04-17 22:28:58.239187","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.805257","last_change":"2024-04-17 22:29:07.480378","last_active":"2024-04-17 22:29:07.805257","last_peered":"2024-04-17 22:29:07.805257","last_clean":"2024-04-17 22:29:07.805257","last_became_active":"2024-04-17 22:29:07.479866","last_became_peered":"2024-04-17 22:29:07.479866","last_unstale":"2024-04-17 22:29:07.805257","last_undegraded":"2024-04-17 22:29:07.805257","last_fullsized":"2024-04-17 22:29:07.805257","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.829882","last_change":"2024-04-17 22:29:07.821315","last_active":"2024-04-17 22:29:07.829882","last_peered":"2024-04-17 22:29:07.829882","last_clean":"2024-04-17 22:29:07.829882","last_became_active":"2024-04-17 22:29:07.820937","last_became_peered":"2024-04-17 22:29:07.820937","last_unstale":"2024-04-17 22:29:07.829882","last_undegraded":"2024-04-17 22:29:07.829882","last_fullsized":"2024-04-17 22:29:07.829882","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":"13","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.813317","last_change":"2024-04-17 22:28:59.254830","last_active":"2024-04-17 22:29:07.813317","last_peered":"2024-04-17 22:29:07.813317","last_clean":"2024-04-17 22:29:07.813317","last_became_active":"2024-04-17 22:28:59.254505","last_became_peered":"2024-04-17 22:28:59.254505","last_unstale":"2024-04-17 22:29:07.813317","last_undegraded":"2024-04-17 22:29:07.813317","last_fullsized":"2024-04-17 22:29:07.813317","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:28:58.239187","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:28:58.239187","last_clean_scrub_stamp":"2024-04-17 22:28:58.239187","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.809624","last_change":"2024-04-17 22:29:07.475017","last_active":"2024-04-17 22:29:07.809624","last_peered":"2024-04-17 22:29:07.809624","last_clean":"2024-04-17 22:29:07.809624","last_became_active":"2024-04-17 22:29:07.473559","last_became_peered":"2024-04-17 22:29:07.473559","last_unstale":"2024-04-17 22:29:07.809624","last_undegraded":"2024-04-17 22:29:07.809624","last_fullsized":"2024-04-17 22:29:07.809624","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.811282","last_change":"2024-04-17 22:29:07.808988","last_active":"2024-04-17 22:29:07.811282","last_peered":"2024-04-17 22:29:07.811282","last_clean":"2024-04-17 22:29:07.811282","last_became_active":"2024-04-17 22:29:07.808663","last_became_peered":"2024-04-17 22:29:07.808663","last_unstale":"2024-04-17 22:29:07.811282","last_undegraded":"2024-04-17 22:29:07.811282","last_fullsized":"2024-04-17 22:29:07.811282","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":"13","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.796932","last_change":"2024-04-17 22:28:59.254055","last_active":"2024-04-17 22:29:07.796932","last_peered":"2024-04-17 22:29:07.796932","last_clean":"2024-04-17 22:29:07.796932","last_became_active":"2024-04-17 22:28:59.253750","last_became_peered":"2024-04-17 22:28:59.253750","last_unstale":"2024-04-17 22:29:07.796932","last_undegraded":"2024-04-17 22:29:07.796932","last_fullsized":"2024-04-17 22:29:07.796932","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:28:58.239187","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:28:58.239187","last_clean_scrub_stamp":"2024-04-17 22:28:58.239187","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.796660","last_change":"2024-04-17 22:29:07.466238","last_active":"2024-04-17 22:29:07.796660","last_peered":"2024-04-17 22:29:07.796660","last_clean":"2024-04-17 22:29:07.796660","last_became_active":"2024-04-17 22:29:07.465806","last_became_peered":"2024-04-17 22:29:07.465806","last_unstale":"2024-04-17 22:29:07.796660","last_undegraded":"2024-04-17 22:29:07.796660","last_fullsized":"2024-04-17 22:29:07.796660","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.820702","last_change":"2024-04-17 22:29:07.809155","last_active":"2024-04-17 22:29:07.820702","last_peered":"2024-04-17 22:29:07.820702","last_clean":"2024-04-17 22:29:07.820702","last_became_active":"2024-04-17 22:29:07.808850","last_became_peered":"2024-04-17 22:29:07.808850","last_unstale":"2024-04-17 22:29:07.820702","last_undegraded":"2024-04-17 22:29:07.820702","last_fullsized":"2024-04-17 22:29:07.820702","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.833569","last_change":"2024-04-17 22:29:07.832794","last_active":"2024-04-17 22:29:07.833569","last_peered":"2024-04-17 22:29:07.833569","last_clean":"2024-04-17 22:29:07.833569","last_became_active":"2024-04-17 22:29:07.832467","last_became_peered":"2024-04-17 22:29:07.832467","last_unstale":"2024-04-17 22:29:07.833569","last_undegraded":"2024-04-17 22:29:07.833569","last_fullsized":"2024-04-17 22:29:07.833569","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.811305","last_change":"2024-04-17 22:29:07.477743","last_active":"2024-04-17 22:29:07.811305","last_peered":"2024-04-17 22:29:07.811305","last_clean":"2024-04-17 22:29:07.811305","last_became_active":"2024-04-17 22:29:07.477503","last_became_peered":"2024-04-17 22:29:07.477503","last_unstale":"2024-04-17 22:29:07.811305","last_undegraded":"2024-04-17 22:29:07.811305","last_fullsized":"2024-04-17 22:29:07.811305","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.830256","last_change":"2024-04-17 22:29:07.828401","last_active":"2024-04-17 22:29:07.830256","last_peered":"2024-04-17 22:29:07.830256","last_clean":"2024-04-17 22:29:07.830256","last_became_active":"2024-04-17 22:29:07.828093","last_became_peered":"2024-04-17 22:29:07.828093","last_unstale":"2024-04-17 22:29:07.830256","last_undegraded":"2024-04-17 22:29:07.830256","last_fullsized":"2024-04-17 22:29:07.830256","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"15'2","reported_seq":"12","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.824506","last_change":"2024-04-17 22:29:07.478361","last_active":"2024-04-17 22:29:07.824506","last_peered":"2024-04-17 22:29:07.824506","last_clean":"2024-04-17 22:29:07.824506","last_became_active":"2024-04-17 22:29:07.477793","last_became_peered":"2024-04-17 22:29:07.477793","last_unstale":"2024-04-17 22:29:07.824506","last_undegraded":"2024-04-17 22:29:07.824506","last_fullsized":"2024-04-17 22:29:07.824506","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.828482","last_change":"2024-04-17 22:29:07.815548","last_active":"2024-04-17 22:29:07.828482","last_peered":"2024-04-17 22:29:07.828482","last_clean":"2024-04-17 22:29:07.828482","last_became_active":"2024-04-17 22:29:07.815263","last_became_peered":"2024-04-17 22:29:07.815263","last_unstale":"2024-04-17 22:29:07.828482","last_undegraded":"2024-04-17 22:29:07.828482","last_fullsized":"2024-04-17 22:29:07.828482","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"15'1","reported_seq":"11","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.831868","last_change":"2024-04-17 22:29:07.482507","last_active":"2024-04-17 22:29:07.831868","last_peered":"2024-04-17 22:29:07.831868","last_clean":"2024-04-17 22:29:07.831868","last_became_active":"2024-04-17 22:29:07.481981","last_became_peered":"2024-04-17 22:29:07.481981","last_unstale":"2024-04-17 22:29:07.831868","last_undegraded":"2024-04-17 22:29:07.831868","last_fullsized":"2024-04-17 22:29:07.831868","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.833109","last_change":"2024-04-17 22:29:07.828459","last_active":"2024-04-17 22:29:07.833109","last_peered":"2024-04-17 22:29:07.833109","last_clean":"2024-04-17 22:29:07.833109","last_became_active":"2024-04-17 22:29:07.828157","last_became_peered":"2024-04-17 22:29:07.828157","last_unstale":"2024-04-17 22:29:07.833109","last_undegraded":"2024-04-17 22:29:07.833109","last_fullsized":"2024-04-17 22:29:07.833109","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"15'1","reported_seq":"11","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.828398","last_change":"2024-04-17 22:29:07.471751","last_active":"2024-04-17 22:29:07.828398","last_peered":"2024-04-17 22:29:07.828398","last_clean":"2024-04-17 22:29:07.828398","last_became_active":"2024-04-17 22:29:07.471385","last_became_peered":"2024-04-17 22:29:07.471385","last_unstale":"2024-04-17 22:29:07.828398","last_undegraded":"2024-04-17 22:29:07.828398","last_fullsized":"2024-04-17 22:29:07.828398","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.821701","last_change":"2024-04-17 22:29:07.812125","last_active":"2024-04-17 22:29:07.821701","last_peered":"2024-04-17 22:29:07.821701","last_clean":"2024-04-17 22:29:07.821701","last_became_active":"2024-04-17 22:29:07.811846","last_became_peered":"2024-04-17 22:29:07.811846","last_unstale":"2024-04-17 22:29:07.821701","last_undegraded":"2024-04-17 22:29:07.821701","last_fullsized":"2024-04-17 22:29:07.821701","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.20","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.798111","last_change":"2024-04-17 22:29:07.477127","last_active":"2024-04-17 22:29:07.798111","last_peered":"2024-04-17 22:29:07.798111","last_clean":"2024-04-17 22:29:07.798111","last_became_active":"2024-04-17 22:29:07.476773","last_became_peered":"2024-04-17 22:29:07.476773","last_unstale":"2024-04-17 22:29:07.798111","last_undegraded":"2024-04-17 22:29:07.798111","last_fullsized":"2024-04-17 22:29:07.798111","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.824583","last_change":"2024-04-17 22:29:07.821132","last_active":"2024-04-17 22:29:07.824583","last_peered":"2024-04-17 22:29:07.824583","last_clean":"2024-04-17 22:29:07.824583","last_became_active":"2024-04-17 22:29:07.820838","last_became_peered":"2024-04-17 22:29:07.820838","last_unstale":"2024-04-17 22:29:07.824583","last_undegraded":"2024-04-17 22:29:07.824583","last_fullsized":"2024-04-17 22:29:07.824583","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.21","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.796071","last_change":"2024-04-17 22:29:07.460110","last_active":"2024-04-17 22:29:07.796071","last_peered":"2024-04-17 22:29:07.796071","last_clean":"2024-04-17 22:29:07.796071","last_became_active":"2024-04-17 22:29:07.459611","last_became_peered":"2024-04-17 22:29:07.459611","last_unstale":"2024-04-17 22:29:07.796071","last_undegraded":"2024-04-17 22:29:07.796071","last_fullsized":"2024-04-17 22:29:07.796071","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.828609","last_change":"2024-04-17 22:29:07.818496","last_active":"2024-04-17 22:29:07.828609","last_peered":"2024-04-17 22:29:07.828609","last_clean":"2024-04-17 22:29:07.828609","last_became_active":"2024-04-17 22:29:07.818184","last_became_peered":"2024-04-17 22:29:07.818184","last_unstale":"2024-04-17 22:29:07.828609","last_undegraded":"2024-04-17 22:29:07.828609","last_fullsized":"2024-04-17 22:29:07.828609","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.22","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.805451","last_change":"2024-04-17 22:29:07.481695","last_active":"2024-04-17 22:29:07.805451","last_peered":"2024-04-17 22:29:07.805451","last_clean":"2024-04-17 22:29:07.805451","last_became_active":"2024-04-17 22:29:07.479982","last_became_peered":"2024-04-17 22:29:07.479982","last_unstale":"2024-04-17 22:29:07.805451","last_undegraded":"2024-04-17 22:29:07.805451","last_fullsized":"2024-04-17 22:29:07.805451","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.835093","last_change":"2024-04-17 22:29:07.833317","last_active":"2024-04-17 22:29:07.835093","last_peered":"2024-04-17 22:29:07.835093","last_clean":"2024-04-17 22:29:07.835093","last_became_active":"2024-04-17 22:29:07.827050","last_became_peered":"2024-04-17 22:29:07.827050","last_unstale":"2024-04-17 22:29:07.835093","last_undegraded":"2024-04-17 22:29:07.835093","last_fullsized":"2024-04-17 22:29:07.835093","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.23","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.807837","last_change":"2024-04-17 22:29:07.477449","last_active":"2024-04-17 22:29:07.807837","last_peered":"2024-04-17 22:29:07.807837","last_clean":"2024-04-17 22:29:07.807837","last_became_active":"2024-04-17 22:29:07.477155","last_became_peered":"2024-04-17 22:29:07.477155","last_unstale":"2024-04-17 22:29:07.807837","last_undegraded":"2024-04-17 22:29:07.807837","last_fullsized":"2024-04-17 22:29:07.807837","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.810313","last_change":"2024-04-17 22:29:07.808974","last_active":"2024-04-17 22:29:07.810313","last_peered":"2024-04-17 22:29:07.810313","last_clean":"2024-04-17 22:29:07.810313","last_became_active":"2024-04-17 22:29:07.808747","last_became_peered":"2024-04-17 22:29:07.808747","last_unstale":"2024-04-17 22:29:07.810313","last_undegraded":"2024-04-17 22:29:07.810313","last_fullsized":"2024-04-17 22:29:07.810313","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.24","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.796551","last_change":"2024-04-17 22:29:07.466052","last_active":"2024-04-17 22:29:07.796551","last_peered":"2024-04-17 22:29:07.796551","last_clean":"2024-04-17 22:29:07.796551","last_became_active":"2024-04-17 22:29:07.465576","last_became_peered":"2024-04-17 22:29:07.465576","last_unstale":"2024-04-17 22:29:07.796551","last_undegraded":"2024-04-17 22:29:07.796551","last_fullsized":"2024-04-17 22:29:07.796551","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.833471","last_change":"2024-04-17 22:29:07.829066","last_active":"2024-04-17 22:29:07.833471","last_peered":"2024-04-17 22:29:07.833471","last_clean":"2024-04-17 22:29:07.833471","last_became_active":"2024-04-17 22:29:07.828735","last_became_peered":"2024-04-17 22:29:07.828735","last_unstale":"2024-04-17 22:29:07.833471","last_undegraded":"2024-04-17 22:29:07.833471","last_fullsized":"2024-04-17 22:29:07.833471","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.25","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.795566","last_change":"2024-04-17 22:29:07.462011","last_active":"2024-04-17 22:29:07.795566","last_peered":"2024-04-17 22:29:07.795566","last_clean":"2024-04-17 22:29:07.795566","last_became_active":"2024-04-17 22:29:07.461693","last_became_peered":"2024-04-17 22:29:07.461693","last_unstale":"2024-04-17 22:29:07.795566","last_undegraded":"2024-04-17 22:29:07.795566","last_fullsized":"2024-04-17 22:29:07.795566","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.828447","last_change":"2024-04-17 22:29:07.826655","last_active":"2024-04-17 22:29:07.828447","last_peered":"2024-04-17 22:29:07.828447","last_clean":"2024-04-17 22:29:07.828447","last_became_active":"2024-04-17 22:29:07.823665","last_became_peered":"2024-04-17 22:29:07.823665","last_unstale":"2024-04-17 22:29:07.828447","last_undegraded":"2024-04-17 22:29:07.828447","last_fullsized":"2024-04-17 22:29:07.828447","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.26","version":"16'1","reported_seq":"11","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:08.802774","last_change":"2024-04-17 22:29:07.480854","last_active":"2024-04-17 22:29:08.802774","last_peered":"2024-04-17 22:29:08.802774","last_clean":"2024-04-17 22:29:08.802774","last_became_active":"2024-04-17 22:29:07.479975","last_became_peered":"2024-04-17 22:29:07.479975","last_unstale":"2024-04-17 22:29:08.802774","last_undegraded":"2024-04-17 22:29:08.802774","last_fullsized":"2024-04-17 22:29:08.802774","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.825070","last_change":"2024-04-17 22:29:07.813744","last_active":"2024-04-17 22:29:07.825070","last_peered":"2024-04-17 22:29:07.825070","last_clean":"2024-04-17 22:29:07.825070","last_became_active":"2024-04-17 22:29:07.810646","last_became_peered":"2024-04-17 22:29:07.810646","last_unstale":"2024-04-17 22:29:07.825070","last_undegraded":"2024-04-17 22:29:07.825070","last_fullsized":"2024-04-17 22:29:07.825070","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.27","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.815316","last_change":"2024-04-17 22:29:07.476428","last_active":"2024-04-17 22:29:07.815316","last_peered":"2024-04-17 22:29:07.815316","last_clean":"2024-04-17 22:29:07.815316","last_became_active":"2024-04-17 22:29:07.476128","last_became_peered":"2024-04-17 22:29:07.476128","last_unstale":"2024-04-17 22:29:07.815316","last_undegraded":"2024-04-17 22:29:07.815316","last_fullsized":"2024-04-17 22:29:07.815316","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.826418","last_change":"2024-04-17 22:29:07.823340","last_active":"2024-04-17 22:29:07.826418","last_peered":"2024-04-17 22:29:07.826418","last_clean":"2024-04-17 22:29:07.826418","last_became_active":"2024-04-17 22:29:07.822764","last_became_peered":"2024-04-17 22:29:07.822764","last_unstale":"2024-04-17 22:29:07.826418","last_undegraded":"2024-04-17 22:29:07.826418","last_fullsized":"2024-04-17 22:29:07.826418","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.28","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.798818","last_change":"2024-04-17 22:29:07.477137","last_active":"2024-04-17 22:29:07.798818","last_peered":"2024-04-17 22:29:07.798818","last_clean":"2024-04-17 22:29:07.798818","last_became_active":"2024-04-17 22:29:07.476796","last_became_peered":"2024-04-17 22:29:07.476796","last_unstale":"2024-04-17 22:29:07.798818","last_undegraded":"2024-04-17 22:29:07.798818","last_fullsized":"2024-04-17 22:29:07.798818","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.826239","last_change":"2024-04-17 22:29:07.823119","last_active":"2024-04-17 22:29:07.826239","last_peered":"2024-04-17 22:29:07.826239","last_clean":"2024-04-17 22:29:07.826239","last_became_active":"2024-04-17 22:29:07.822640","last_became_peered":"2024-04-17 22:29:07.822640","last_unstale":"2024-04-17 22:29:07.826239","last_undegraded":"2024-04-17 22:29:07.826239","last_fullsized":"2024-04-17 22:29:07.826239","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.29","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.792382","last_change":"2024-04-17 22:29:07.473467","last_active":"2024-04-17 22:29:07.792382","last_peered":"2024-04-17 22:29:07.792382","last_clean":"2024-04-17 22:29:07.792382","last_became_active":"2024-04-17 22:29:07.473028","last_became_peered":"2024-04-17 22:29:07.473028","last_unstale":"2024-04-17 22:29:07.792382","last_undegraded":"2024-04-17 22:29:07.792382","last_fullsized":"2024-04-17 22:29:07.792382","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.827332","last_change":"2024-04-17 22:29:07.817097","last_active":"2024-04-17 22:29:07.827332","last_peered":"2024-04-17 22:29:07.827332","last_clean":"2024-04-17 22:29:07.827332","last_became_active":"2024-04-17 22:29:07.816813","last_became_peered":"2024-04-17 22:29:07.816813","last_unstale":"2024-04-17 22:29:07.827332","last_undegraded":"2024-04-17 22:29:07.827332","last_fullsized":"2024-04-17 22:29:07.827332","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2a","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.796331","last_change":"2024-04-17 22:29:07.464330","last_active":"2024-04-17 22:29:07.796331","last_peered":"2024-04-17 22:29:07.796331","last_clean":"2024-04-17 22:29:07.796331","last_became_active":"2024-04-17 22:29:07.464002","last_became_peered":"2024-04-17 22:29:07.464002","last_unstale":"2024-04-17 22:29:07.796331","last_undegraded":"2024-04-17 22:29:07.796331","last_fullsized":"2024-04-17 22:29:07.796331","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.830305","last_change":"2024-04-17 22:29:07.828690","last_active":"2024-04-17 22:29:07.830305","last_peered":"2024-04-17 22:29:07.830305","last_clean":"2024-04-17 22:29:07.830305","last_became_active":"2024-04-17 22:29:07.828407","last_became_peered":"2024-04-17 22:29:07.828407","last_unstale":"2024-04-17 22:29:07.830305","last_undegraded":"2024-04-17 22:29:07.830305","last_fullsized":"2024-04-17 22:29:07.830305","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2b","version":"15'1","reported_seq":"11","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.810554","last_change":"2024-04-17 22:29:07.471300","last_active":"2024-04-17 22:29:07.810554","last_peered":"2024-04-17 22:29:07.810554","last_clean":"2024-04-17 22:29:07.810554","last_became_active":"2024-04-17 22:29:07.470886","last_became_peered":"2024-04-17 22:29:07.470886","last_unstale":"2024-04-17 22:29:07.810554","last_undegraded":"2024-04-17 22:29:07.810554","last_fullsized":"2024-04-17 22:29:07.810554","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.828614","last_change":"2024-04-17 22:29:07.826555","last_active":"2024-04-17 22:29:07.828614","last_peered":"2024-04-17 22:29:07.828614","last_clean":"2024-04-17 22:29:07.828614","last_became_active":"2024-04-17 22:29:07.826262","last_became_peered":"2024-04-17 22:29:07.826262","last_unstale":"2024-04-17 22:29:07.828614","last_undegraded":"2024-04-17 22:29:07.828614","last_fullsized":"2024-04-17 22:29:07.828614","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2c","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.797039","last_change":"2024-04-17 22:29:07.462176","last_active":"2024-04-17 22:29:07.797039","last_peered":"2024-04-17 22:29:07.797039","last_clean":"2024-04-17 22:29:07.797039","last_became_active":"2024-04-17 22:29:07.461594","last_became_peered":"2024-04-17 22:29:07.461594","last_unstale":"2024-04-17 22:29:07.797039","last_undegraded":"2024-04-17 22:29:07.797039","last_fullsized":"2024-04-17 22:29:07.797039","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.828130","last_change":"2024-04-17 22:29:07.816896","last_active":"2024-04-17 22:29:07.828130","last_peered":"2024-04-17 22:29:07.828130","last_clean":"2024-04-17 22:29:07.828130","last_became_active":"2024-04-17 22:29:07.816538","last_became_peered":"2024-04-17 22:29:07.816538","last_unstale":"2024-04-17 22:29:07.828130","last_undegraded":"2024-04-17 22:29:07.828130","last_fullsized":"2024-04-17 22:29:07.828130","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2d","version":"15'1","reported_seq":"11","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.827538","last_change":"2024-04-17 22:29:07.479214","last_active":"2024-04-17 22:29:07.827538","last_peered":"2024-04-17 22:29:07.827538","last_clean":"2024-04-17 22:29:07.827538","last_became_active":"2024-04-17 22:29:07.478790","last_became_peered":"2024-04-17 22:29:07.478790","last_unstale":"2024-04-17 22:29:07.827538","last_undegraded":"2024-04-17 22:29:07.827538","last_fullsized":"2024-04-17 22:29:07.827538","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.833543","last_change":"2024-04-17 22:29:07.832349","last_active":"2024-04-17 22:29:07.833543","last_peered":"2024-04-17 22:29:07.833543","last_clean":"2024-04-17 22:29:07.833543","last_became_active":"2024-04-17 22:29:07.831999","last_became_peered":"2024-04-17 22:29:07.831999","last_unstale":"2024-04-17 22:29:07.833543","last_undegraded":"2024-04-17 22:29:07.833543","last_fullsized":"2024-04-17 22:29:07.833543","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2e","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.797059","last_change":"2024-04-17 22:29:07.483244","last_active":"2024-04-17 22:29:07.797059","last_peered":"2024-04-17 22:29:07.797059","last_clean":"2024-04-17 22:29:07.797059","last_became_active":"2024-04-17 22:29:07.482893","last_became_peered":"2024-04-17 22:29:07.482893","last_unstale":"2024-04-17 22:29:07.797059","last_undegraded":"2024-04-17 22:29:07.797059","last_fullsized":"2024-04-17 22:29:07.797059","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.825028","last_change":"2024-04-17 22:29:07.806512","last_active":"2024-04-17 22:29:07.825028","last_peered":"2024-04-17 22:29:07.825028","last_clean":"2024-04-17 22:29:07.825028","last_became_active":"2024-04-17 22:29:07.806182","last_became_peered":"2024-04-17 22:29:07.806182","last_unstale":"2024-04-17 22:29:07.825028","last_undegraded":"2024-04-17 22:29:07.825028","last_fullsized":"2024-04-17 22:29:07.825028","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2f","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.815418","last_change":"2024-04-17 22:29:07.478655","last_active":"2024-04-17 22:29:07.815418","last_peered":"2024-04-17 22:29:07.815418","last_clean":"2024-04-17 22:29:07.815418","last_became_active":"2024-04-17 22:29:07.478346","last_became_peered":"2024-04-17 22:29:07.478346","last_unstale":"2024-04-17 22:29:07.815418","last_undegraded":"2024-04-17 22:29:07.815418","last_fullsized":"2024-04-17 22:29:07.815418","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.826848","last_change":"2024-04-17 22:29:07.825658","last_active":"2024-04-17 22:29:07.826848","last_peered":"2024-04-17 22:29:07.826848","last_clean":"2024-04-17 22:29:07.826848","last_became_active":"2024-04-17 22:29:07.825349","last_became_peered":"2024-04-17 22:29:07.825349","last_unstale":"2024-04-17 22:29:07.826848","last_undegraded":"2024-04-17 22:29:07.826848","last_fullsized":"2024-04-17 22:29:07.826848","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.30","version":"15'4","reported_seq":"14","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.824396","last_change":"2024-04-17 22:29:07.475470","last_active":"2024-04-17 22:29:07.824396","last_peered":"2024-04-17 22:29:07.824396","last_clean":"2024-04-17 22:29:07.824396","last_became_active":"2024-04-17 22:29:07.474917","last_became_peered":"2024-04-17 22:29:07.474917","last_unstale":"2024-04-17 22:29:07.824396","last_undegraded":"2024-04-17 22:29:07.824396","last_fullsized":"2024-04-17 22:29:07.824396","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.826262","last_change":"2024-04-17 22:29:07.823624","last_active":"2024-04-17 22:29:07.826262","last_peered":"2024-04-17 22:29:07.826262","last_clean":"2024-04-17 22:29:07.826262","last_became_active":"2024-04-17 22:29:07.822815","last_became_peered":"2024-04-17 22:29:07.822815","last_unstale":"2024-04-17 22:29:07.826262","last_undegraded":"2024-04-17 22:29:07.826262","last_fullsized":"2024-04-17 22:29:07.826262","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.31","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.804183","last_change":"2024-04-17 22:29:07.474318","last_active":"2024-04-17 22:29:07.804183","last_peered":"2024-04-17 22:29:07.804183","last_clean":"2024-04-17 22:29:07.804183","last_became_active":"2024-04-17 22:29:07.473754","last_became_peered":"2024-04-17 22:29:07.473754","last_unstale":"2024-04-17 22:29:07.804183","last_undegraded":"2024-04-17 22:29:07.804183","last_fullsized":"2024-04-17 22:29:07.804183","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.833495","last_change":"2024-04-17 22:29:07.829098","last_active":"2024-04-17 22:29:07.833495","last_peered":"2024-04-17 22:29:07.833495","last_clean":"2024-04-17 22:29:07.833495","last_became_active":"2024-04-17 22:29:07.828773","last_became_peered":"2024-04-17 22:29:07.828773","last_unstale":"2024-04-17 22:29:07.833495","last_undegraded":"2024-04-17 22:29:07.833495","last_fullsized":"2024-04-17 22:29:07.833495","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.32","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.795411","last_change":"2024-04-17 22:29:07.470666","last_active":"2024-04-17 22:29:07.795411","last_peered":"2024-04-17 22:29:07.795411","last_clean":"2024-04-17 22:29:07.795411","last_became_active":"2024-04-17 22:29:07.470363","last_became_peered":"2024-04-17 22:29:07.470363","last_unstale":"2024-04-17 22:29:07.795411","last_undegraded":"2024-04-17 22:29:07.795411","last_fullsized":"2024-04-17 22:29:07.795411","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.830343","last_change":"2024-04-17 22:29:07.827580","last_active":"2024-04-17 22:29:07.830343","last_peered":"2024-04-17 22:29:07.830343","last_clean":"2024-04-17 22:29:07.830343","last_became_active":"2024-04-17 22:29:07.827233","last_became_peered":"2024-04-17 22:29:07.827233","last_unstale":"2024-04-17 22:29:07.830343","last_undegraded":"2024-04-17 22:29:07.830343","last_fullsized":"2024-04-17 22:29:07.830343","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.33","version":"15'1","reported_seq":"11","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.811613","last_change":"2024-04-17 22:29:07.460978","last_active":"2024-04-17 22:29:07.811613","last_peered":"2024-04-17 22:29:07.811613","last_clean":"2024-04-17 22:29:07.811613","last_became_active":"2024-04-17 22:29:07.460659","last_became_peered":"2024-04-17 22:29:07.460659","last_unstale":"2024-04-17 22:29:07.811613","last_undegraded":"2024-04-17 22:29:07.811613","last_fullsized":"2024-04-17 22:29:07.811613","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.832128","last_change":"2024-04-17 22:29:07.816335","last_active":"2024-04-17 22:29:07.832128","last_peered":"2024-04-17 22:29:07.832128","last_clean":"2024-04-17 22:29:07.832128","last_became_active":"2024-04-17 22:29:07.816090","last_became_peered":"2024-04-17 22:29:07.816090","last_unstale":"2024-04-17 22:29:07.832128","last_undegraded":"2024-04-17 22:29:07.832128","last_fullsized":"2024-04-17 22:29:07.832128","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.34","version":"15'4","reported_seq":"14","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.835387","last_change":"2024-04-17 22:29:07.475702","last_active":"2024-04-17 22:29:07.835387","last_peered":"2024-04-17 22:29:07.835387","last_clean":"2024-04-17 22:29:07.835387","last_became_active":"2024-04-17 22:29:07.475395","last_became_peered":"2024-04-17 22:29:07.475395","last_unstale":"2024-04-17 22:29:07.835387","last_undegraded":"2024-04-17 22:29:07.835387","last_fullsized":"2024-04-17 22:29:07.835387","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":892,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.828199","last_change":"2024-04-17 22:29:07.818538","last_active":"2024-04-17 22:29:07.828199","last_peered":"2024-04-17 22:29:07.828199","last_clean":"2024-04-17 22:29:07.828199","last_became_active":"2024-04-17 22:29:07.817164","last_became_peered":"2024-04-17 22:29:07.817164","last_unstale":"2024-04-17 22:29:07.828199","last_undegraded":"2024-04-17 22:29:07.828199","last_fullsized":"2024-04-17 22:29:07.828199","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.35","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.797071","last_change":"2024-04-17 22:29:07.467651","last_active":"2024-04-17 22:29:07.797071","last_peered":"2024-04-17 22:29:07.797071","last_clean":"2024-04-17 22:29:07.797071","last_became_active":"2024-04-17 22:29:07.467327","last_became_peered":"2024-04-17 22:29:07.467327","last_unstale":"2024-04-17 22:29:07.797071","last_undegraded":"2024-04-17 22:29:07.797071","last_fullsized":"2024-04-17 22:29:07.797071","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.810303","last_change":"2024-04-17 22:29:07.809693","last_active":"2024-04-17 22:29:07.810303","last_peered":"2024-04-17 22:29:07.810303","last_clean":"2024-04-17 22:29:07.810303","last_became_active":"2024-04-17 22:29:07.809415","last_became_peered":"2024-04-17 22:29:07.809415","last_unstale":"2024-04-17 22:29:07.810303","last_undegraded":"2024-04-17 22:29:07.810303","last_fullsized":"2024-04-17 22:29:07.810303","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.36","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.796841","last_change":"2024-04-17 22:29:07.483402","last_active":"2024-04-17 22:29:07.796841","last_peered":"2024-04-17 22:29:07.796841","last_clean":"2024-04-17 22:29:07.796841","last_became_active":"2024-04-17 22:29:07.482914","last_became_peered":"2024-04-17 22:29:07.482914","last_unstale":"2024-04-17 22:29:07.796841","last_undegraded":"2024-04-17 22:29:07.796841","last_fullsized":"2024-04-17 22:29:07.796841","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.830317","last_change":"2024-04-17 22:29:07.827623","last_active":"2024-04-17 22:29:07.830317","last_peered":"2024-04-17 22:29:07.830317","last_clean":"2024-04-17 22:29:07.830317","last_became_active":"2024-04-17 22:29:07.827337","last_became_peered":"2024-04-17 22:29:07.827337","last_unstale":"2024-04-17 22:29:07.830317","last_undegraded":"2024-04-17 22:29:07.830317","last_fullsized":"2024-04-17 22:29:07.830317","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.37","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.808508","last_change":"2024-04-17 22:29:07.463449","last_active":"2024-04-17 22:29:07.808508","last_peered":"2024-04-17 22:29:07.808508","last_clean":"2024-04-17 22:29:07.808508","last_became_active":"2024-04-17 22:29:07.463110","last_became_peered":"2024-04-17 22:29:07.463110","last_unstale":"2024-04-17 22:29:07.808508","last_undegraded":"2024-04-17 22:29:07.808508","last_fullsized":"2024-04-17 22:29:07.808508","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.827399","last_change":"2024-04-17 22:29:07.815214","last_active":"2024-04-17 22:29:07.827399","last_peered":"2024-04-17 22:29:07.827399","last_clean":"2024-04-17 22:29:07.827399","last_became_active":"2024-04-17 22:29:07.814938","last_became_peered":"2024-04-17 22:29:07.814938","last_unstale":"2024-04-17 22:29:07.827399","last_undegraded":"2024-04-17 22:29:07.827399","last_fullsized":"2024-04-17 22:29:07.827399","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.38","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.801328","last_change":"2024-04-17 22:29:07.474682","last_active":"2024-04-17 22:29:07.801328","last_peered":"2024-04-17 22:29:07.801328","last_clean":"2024-04-17 22:29:07.801328","last_became_active":"2024-04-17 22:29:07.474305","last_became_peered":"2024-04-17 22:29:07.474305","last_unstale":"2024-04-17 22:29:07.801328","last_undegraded":"2024-04-17 22:29:07.801328","last_fullsized":"2024-04-17 22:29:07.801328","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.833512","last_change":"2024-04-17 22:29:07.829769","last_active":"2024-04-17 22:29:07.833512","last_peered":"2024-04-17 22:29:07.833512","last_clean":"2024-04-17 22:29:07.833512","last_became_active":"2024-04-17 22:29:07.829484","last_became_peered":"2024-04-17 22:29:07.829484","last_unstale":"2024-04-17 22:29:07.833512","last_undegraded":"2024-04-17 22:29:07.833512","last_fullsized":"2024-04-17 22:29:07.833512","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.39","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.794112","last_change":"2024-04-17 22:29:07.458386","last_active":"2024-04-17 22:29:07.794112","last_peered":"2024-04-17 22:29:07.794112","last_clean":"2024-04-17 22:29:07.794112","last_became_active":"2024-04-17 22:29:07.456695","last_became_peered":"2024-04-17 22:29:07.456695","last_unstale":"2024-04-17 22:29:07.794112","last_undegraded":"2024-04-17 22:29:07.794112","last_fullsized":"2024-04-17 22:29:07.794112","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.829903","last_change":"2024-04-17 22:29:07.821490","last_active":"2024-04-17 22:29:07.829903","last_peered":"2024-04-17 22:29:07.829903","last_clean":"2024-04-17 22:29:07.829903","last_became_active":"2024-04-17 22:29:07.821103","last_became_peered":"2024-04-17 22:29:07.821103","last_unstale":"2024-04-17 22:29:07.829903","last_undegraded":"2024-04-17 22:29:07.829903","last_fullsized":"2024-04-17 22:29:07.829903","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3a","version":"15'1","reported_seq":"11","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.819620","last_change":"2024-04-17 22:29:07.472610","last_active":"2024-04-17 22:29:07.819620","last_peered":"2024-04-17 22:29:07.819620","last_clean":"2024-04-17 22:29:07.819620","last_became_active":"2024-04-17 22:29:07.472297","last_became_peered":"2024-04-17 22:29:07.472297","last_unstale":"2024-04-17 22:29:07.819620","last_undegraded":"2024-04-17 22:29:07.819620","last_fullsized":"2024-04-17 22:29:07.819620","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.824029","last_change":"2024-04-17 22:29:07.820738","last_active":"2024-04-17 22:29:07.824029","last_peered":"2024-04-17 22:29:07.824029","last_clean":"2024-04-17 22:29:07.824029","last_became_active":"2024-04-17 22:29:07.820354","last_became_peered":"2024-04-17 22:29:07.820354","last_unstale":"2024-04-17 22:29:07.824029","last_undegraded":"2024-04-17 22:29:07.824029","last_fullsized":"2024-04-17 22:29:07.824029","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3b","version":"15'1","reported_seq":"11","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.824121","last_change":"2024-04-17 22:29:07.473963","last_active":"2024-04-17 22:29:07.824121","last_peered":"2024-04-17 22:29:07.824121","last_clean":"2024-04-17 22:29:07.824121","last_became_active":"2024-04-17 22:29:07.472018","last_became_peered":"2024-04-17 22:29:07.472018","last_unstale":"2024-04-17 22:29:07.824121","last_undegraded":"2024-04-17 22:29:07.824121","last_fullsized":"2024-04-17 22:29:07.824121","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.832188","last_change":"2024-04-17 22:29:07.824919","last_active":"2024-04-17 22:29:07.832188","last_peered":"2024-04-17 22:29:07.832188","last_clean":"2024-04-17 22:29:07.832188","last_became_active":"2024-04-17 22:29:07.824600","last_became_peered":"2024-04-17 22:29:07.824600","last_unstale":"2024-04-17 22:29:07.832188","last_undegraded":"2024-04-17 22:29:07.832188","last_fullsized":"2024-04-17 22:29:07.832188","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3c","version":"15'2","reported_seq":"12","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.825427","last_change":"2024-04-17 22:29:07.475485","last_active":"2024-04-17 22:29:07.825427","last_peered":"2024-04-17 22:29:07.825427","last_clean":"2024-04-17 22:29:07.825427","last_became_active":"2024-04-17 22:29:07.475182","last_became_peered":"2024-04-17 22:29:07.475182","last_unstale":"2024-04-17 22:29:07.825427","last_undegraded":"2024-04-17 22:29:07.825427","last_fullsized":"2024-04-17 22:29:07.825427","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.821750","last_change":"2024-04-17 22:29:07.810455","last_active":"2024-04-17 22:29:07.821750","last_peered":"2024-04-17 22:29:07.821750","last_clean":"2024-04-17 22:29:07.821750","last_became_active":"2024-04-17 22:29:07.810143","last_became_peered":"2024-04-17 22:29:07.810143","last_unstale":"2024-04-17 22:29:07.821750","last_undegraded":"2024-04-17 22:29:07.821750","last_fullsized":"2024-04-17 22:29:07.821750","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3d","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.797015","last_change":"2024-04-17 22:29:07.468187","last_active":"2024-04-17 22:29:07.797015","last_peered":"2024-04-17 22:29:07.797015","last_clean":"2024-04-17 22:29:07.797015","last_became_active":"2024-04-17 22:29:07.467849","last_became_peered":"2024-04-17 22:29:07.467849","last_unstale":"2024-04-17 22:29:07.797015","last_undegraded":"2024-04-17 22:29:07.797015","last_fullsized":"2024-04-17 22:29:07.797015","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.830571","last_change":"2024-04-17 22:29:07.822173","last_active":"2024-04-17 22:29:07.830571","last_peered":"2024-04-17 22:29:07.830571","last_clean":"2024-04-17 22:29:07.830571","last_became_active":"2024-04-17 22:29:07.821875","last_became_peered":"2024-04-17 22:29:07.821875","last_unstale":"2024-04-17 22:29:07.830571","last_undegraded":"2024-04-17 22:29:07.830571","last_fullsized":"2024-04-17 22:29:07.830571","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3e","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.807733","last_change":"2024-04-17 22:29:07.479547","last_active":"2024-04-17 22:29:07.807733","last_peered":"2024-04-17 22:29:07.807733","last_clean":"2024-04-17 22:29:07.807733","last_became_active":"2024-04-17 22:29:07.479004","last_became_peered":"2024-04-17 22:29:07.479004","last_unstale":"2024-04-17 22:29:07.807733","last_undegraded":"2024-04-17 22:29:07.807733","last_fullsized":"2024-04-17 22:29:07.807733","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.823835","last_change":"2024-04-17 22:29:07.819432","last_active":"2024-04-17 22:29:07.823835","last_peered":"2024-04-17 22:29:07.823835","last_clean":"2024-04-17 22:29:07.823835","last_became_active":"2024-04-17 22:29:07.819001","last_became_peered":"2024-04-17 22:29:07.819001","last_unstale":"2024-04-17 22:29:07.823835","last_undegraded":"2024-04-17 22:29:07.823835","last_fullsized":"2024-04-17 22:29:07.823835","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3f","version":"15'3","reported_seq":"13","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.829638","last_change":"2024-04-17 22:29:07.466765","last_active":"2024-04-17 22:29:07.829638","last_peered":"2024-04-17 22:29:07.829638","last_clean":"2024-04-17 22:29:07.829638","last_became_active":"2024-04-17 22:29:07.463177","last_became_peered":"2024-04-17 22:29:07.463177","last_unstale":"2024-04-17 22:29:07.829638","last_undegraded":"2024-04-17 22:29:07.829638","last_fullsized":"2024-04-17 22:29:07.829638","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":544,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":4},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":4},{"poolid":2,"num_pg":64,"stat_sum":{"num_bytes":2286,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":128,"acting":128,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705669,"num_pgs":82,"num_osds":1,"num_per_pool_osds":1,"kb":94371840,"kb_used":1048744,"kb_used_data":160,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323096,"statfs":{"total":96636764160,"available":95562850304,"internally_reserved":1073741824,"allocated":163840,"data_stored":41000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.19900000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27500000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.22900000000000001}]}]},{"osd":2,"up_from":9,"seq":38654705669,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"kb":94371840,"kb_used":1048732,"kb_used_data":148,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323108,"statfs":{"total":96636764160,"available":95562862592,"internally_reserved":1073741824,"allocated":151552,"data_stored":39984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.129}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.217}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.20100000000000001}]}]},{"osd":0,"up_from":9,"seq":38654705669,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"kb":94371840,"kb_used":1048736,"kb_used_data":152,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323104,"statfs":{"total":96636764160,"available":95562858496,"internally_reserved":1073741824,"allocated":155648,"data_stored":40772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29399999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.31}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.307}]}]},{"osd":1,"up_from":9,"seq":38654705668,"num_pgs":63,"num_osds":1,"num_per_pool_osds":1,"kb":94371840,"kb_used":1048732,"kb_used_data":148,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323108,"statfs":{"total":96636764160,"available":95562862592,"internally_reserved":1073741824,"allocated":151552,"data_stored":39936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27300000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.307}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33000000000000002}]}]}]}} 2024-04-17T22:29:15.215 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-17T22:29:15.216 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-17T22:29:15.216 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-04-17T22:29:15.216 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-04-17T22:29:15.216 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-04-17T22:29:15.377 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.388 7f453ba14700 1 Processor -- start 2024-04-17T22:29:15.377 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.388 7f453ba14700 1 -- start start 2024-04-17T22:29:15.377 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.389 7f453ba14700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f453410e1d0 0x7f453410e620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:15.377 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.389 7f453ba14700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f453410ec50 0x7f45340f16c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:15.377 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.389 7f453ba14700 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45340f2140 con 0x7f45340f1cf0 2024-04-17T22:29:15.378 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.389 7f453ba14700 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45340f9ea0 con 0x7f453410e1d0 2024-04-17T22:29:15.378 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.389 7f453ba14700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f45340f8fc0 con 0x7f453410ec50 2024-04-17T22:29:15.378 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.389 7f453880f700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f453410e1d0 0x7f453410e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:15.378 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.389 7f4533fff700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f453410ec50 0x7f45340f16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:15.378 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.389 7f453880f700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f453410e1d0 0x7f453410e620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:40720/0 (socket says 172.21.15.102:40720) 2024-04-17T22:29:15.378 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.389 7f4539010700 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f45340f1cf0 0x7f45341aa460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:34618/0 (socket says 172.21.15.102:34618) 2024-04-17T22:29:15.378 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.389 7f453880f700 1 -- 172.21.15.102:0/3434285 learned_addr learned my addr 172.21.15.102:0/3434285 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:29:15.378 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.389 7f453880f700 1 -- 172.21.15.102:0/3434285 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f453410ec50 msgr2=0x7f45340f16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:29:15.378 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.389 7f453880f700 1 --2- 172.21.15.102:0/3434285 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f453410ec50 0x7f45340f16c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:15.378 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.389 7f453880f700 1 -- 172.21.15.102:0/3434285 >> v1:172.21.15.102:6789/0 conn(0x7f45340f1cf0 legacy=0x7f45341aa460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:15.378 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.389 7f453880f700 1 -- 172.21.15.102:0/3434285 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45340f2360 con 0x7f453410e1d0 2024-04-17T22:29:15.378 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.389 7f45337fe700 1 -- 172.21.15.102:0/3434285 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3904102130 0 0) 0x7f452c001490 con 0x7f45340f1cf0 2024-04-17T22:29:15.378 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.389 7f4533fff700 1 --2- 172.21.15.102:0/3434285 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f453410ec50 0x7f45340f16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:29:15.379 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.389 7f453880f700 1 --2- 172.21.15.102:0/3434285 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f453410e1d0 0x7f453410e620 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 rx=0x7f4528002f70 tx=0x7f4528009a50).ready entity=mon.0 client_cookie=a2eb7057928df419 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:29:15.379 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.389 7f45337fe700 1 -- 172.21.15.102:0/3434285 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f45340f2360 con 0x7f453410e1d0 2024-04-17T22:29:15.379 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.389 7f45337fe700 1 -- 172.21.15.102:0/3434285 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f452800f510 con 0x7f453410e1d0 2024-04-17T22:29:15.379 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.389 7f45337fe700 1 -- 172.21.15.102:0/3434285 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f452800f740 con 0x7f453410e1d0 2024-04-17T22:29:15.379 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.389 7f453ba14700 1 -- 172.21.15.102:0/3434285 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f453410e1d0 msgr2=0x7f453410e620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:15.379 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.389 7f453ba14700 1 --2- 172.21.15.102:0/3434285 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f453410e1d0 0x7f453410e620 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 rx=0x7f4528002f70 tx=0x7f4528009a50).stop 2024-04-17T22:29:15.379 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.390 7f453ba14700 1 -- 172.21.15.102:0/3434285 shutdown_connections 2024-04-17T22:29:15.379 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.390 7f453ba14700 1 --2- 172.21.15.102:0/3434285 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f453410e1d0 0x7f453410e620 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:15.379 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.390 7f453ba14700 1 --2- 172.21.15.102:0/3434285 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f453410ec50 0x7f45340f16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:15.379 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.390 7f453ba14700 1 -- 172.21.15.102:0/3434285 >> 172.21.15.102:0/3434285 conn(0x7f4534113830 msgr2=0x7f45340f4ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:29:15.379 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.390 7f453ba14700 1 -- 172.21.15.102:0/3434285 shutdown_connections 2024-04-17T22:29:15.379 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.390 7f453ba14700 1 -- 172.21.15.102:0/3434285 wait complete. 2024-04-17T22:29:15.399 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.413 7f453ba14700 1 Processor -- start 2024-04-17T22:29:15.399 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.413 7f453ba14700 1 -- start start 2024-04-17T22:29:15.401 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.413 7f453ba14700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f45341a6290 0x7f45341a66e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:15.401 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.413 7f453ba14700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f45341a6d10 0x7f45340d5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:15.401 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.413 7f453ba14700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f45340d5770 0x7f45341bba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:15.402 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.413 7f453ba14700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45340fa140 con 0x7f45341a6d10 2024-04-17T22:29:15.402 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.413 7f453ba14700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45341a7780 con 0x7f45341a6290 2024-04-17T22:29:15.402 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.413 7f453ba14700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f45341a73f0 con 0x7f45340d5770 2024-04-17T22:29:15.402 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.413 7f4539010700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f45340d5770 0x7f45341bba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:15.402 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.413 7f4533fff700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f45341a6d10 0x7f45340d5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:15.402 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.413 7f453880f700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f45341a6290 0x7f45341a66e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:15.402 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.413 7f4539010700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f45340d5770 0x7f45341bba80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:44124/0 (socket says 172.21.15.102:44124) 2024-04-17T22:29:15.402 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.413 7f4539010700 1 -- 172.21.15.102:0/3200400552 learned_addr learned my addr 172.21.15.102:0/3200400552 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:29:15.402 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.414 7f4533fff700 1 -- 172.21.15.102:0/3200400552 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f45340d5770 msgr2=0x7f45341bba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:15.402 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.414 7f4533fff700 1 --2- 172.21.15.102:0/3200400552 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f45340d5770 0x7f45341bba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:15.402 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.414 7f4533fff700 1 -- 172.21.15.102:0/3200400552 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f45341a6290 msgr2=0x7f45341a66e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:15.402 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.414 7f4533fff700 1 --2- 172.21.15.102:0/3200400552 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f45341a6290 0x7f45341a66e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:15.402 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.414 7f4533fff700 1 -- 172.21.15.102:0/3200400552 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4534087b30 con 0x7f45341a6d10 2024-04-17T22:29:15.403 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.414 7f453880f700 1 --2- 172.21.15.102:0/3200400552 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f45341a6290 0x7f45341a66e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:29:15.403 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.414 7f4533fff700 1 --2- 172.21.15.102:0/3200400552 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f45341a6d10 0x7f45340d5140 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 rx=0x7f452000c000 tx=0x7f452000eca0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:29:15.403 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.414 7f4531ffb700 1 -- 172.21.15.102:0/3200400552 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f4534087b30 con 0x7f45341a6d10 2024-04-17T22:29:15.403 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.414 7f4531ffb700 1 -- 172.21.15.102:0/3200400552 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f452000c790 con 0x7f45341a6d10 2024-04-17T22:29:15.403 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.414 7f453ba14700 1 -- 172.21.15.102:0/3200400552 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45341a7a40 con 0x7f45341a6d10 2024-04-17T22:29:15.403 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.414 7f4531ffb700 1 -- 172.21.15.102:0/3200400552 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f452000f040 con 0x7f45341a6d10 2024-04-17T22:29:15.403 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.414 7f453ba14700 1 -- 172.21.15.102:0/3200400552 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45340d5bc0 con 0x7f45341a6d10 2024-04-17T22:29:15.403 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.414 7f4531ffb700 1 -- 172.21.15.102:0/3200400552 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7f4520011070 con 0x7f45341a6d10 2024-04-17T22:29:15.403 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.414 7f4531ffb700 1 -- 172.21.15.102:0/3200400552 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 3194+0+0 (secure 0 0 0) 0x7f4520010040 con 0x7f45341a6d10 2024-04-17T22:29:15.403 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.415 7f453ba14700 1 -- 172.21.15.102:0/3200400552 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45340f0790 con 0x7f45341a6d10 2024-04-17T22:29:15.404 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.418 7f4531ffb700 1 -- 172.21.15.102:0/3200400552 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+110455 (secure 0 0 0) 0x7f452000cb60 con 0x7f45341a6d10 2024-04-17T22:29:15.529 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.543 7f453ba14700 1 -- 172.21.15.102:0/3200400552 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f45341aea30 con 0x7f45341a6d10 2024-04-17T22:29:15.530 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.543 7f4531ffb700 1 -- 172.21.15.102:0/3200400552 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v3) v1 ==== 74+0+32134 (secure 0 0 0) 0x7f4520000c30 con 0x7f45341a6d10 2024-04-17T22:29:15.530 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-17T22:29:15.532 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.545 7f453ba14700 1 -- 172.21.15.102:0/3200400552 >> v1:172.21.15.102:6816/42360 conn(0x7f451c0145a0 legacy=0x7f451c016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:15.532 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.545 7f453ba14700 1 -- 172.21.15.102:0/3200400552 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f45341a6d10 msgr2=0x7f45340d5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:15.532 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.545 7f453ba14700 1 --2- 172.21.15.102:0/3200400552 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f45341a6d10 0x7f45340d5140 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 rx=0x7f452000c000 tx=0x7f452000eca0).stop 2024-04-17T22:29:15.532 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.545 7f453ba14700 1 -- 172.21.15.102:0/3200400552 shutdown_connections 2024-04-17T22:29:15.532 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.545 7f453ba14700 1 --2- 172.21.15.102:0/3200400552 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f45340d5770 0x7f45341bba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:15.532 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.545 7f453ba14700 1 --2- 172.21.15.102:0/3200400552 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f45341a6290 0x7f45341a66e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:15.532 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.545 7f453ba14700 1 --2- 172.21.15.102:0/3200400552 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f45341a6d10 0x7f45340d5140 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:15.532 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.545 7f453ba14700 1 -- 172.21.15.102:0/3200400552 >> 172.21.15.102:0/3200400552 conn(0x7f45340c5670 msgr2=0x7f45340c9620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:29:15.532 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.545 7f453ba14700 1 -- 172.21.15.102:0/3200400552 shutdown_connections 2024-04-17T22:29:15.532 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.545 7f453ba14700 1 -- 172.21.15.102:0/3200400552 wait complete. 2024-04-17T22:29:15.542 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":3,"active_gid":4104,"active_name":"x","active_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6816","nonce":42360}]},"active_addr":"172.21.15.102:6816/42360","active_change":"2024-04-17 22:28:51.181757","available":true,"standbys":[],"modules":["iostat","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"ansible","can_run":true,"error_string":"","module_options":{"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_url":{"name":"server_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"verify_server":{"name":"verify_server","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"none","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_iterations":{"name":"upmap_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimization iterations","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_cephfs":{"name":"FEATURE_TOGGLE_cephfs","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_iscsi":{"name":"FEATURE_TOGGLE_iscsi","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_mirroring":{"name":"FEATURE_TOGGLE_mirroring","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_rbd":{"name":"FEATURE_TOGGLE_rbd","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_rgw":{"name":"FEATURE_TOGGLE_rgw","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_HOST":{"name":"RGW_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_PORT":{"name":"RGW_API_PORT","type":"int","level":"advanced","flags":0,"default_value":"80","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SCHEME":{"name":"RGW_API_SCHEME","type":"str","level":"advanced","flags":0,"default_value":"http","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_USER_ID":{"name":"RGW_API_USER_ID","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"deepsea","can_run":true,"error_string":"","module_options":{"salt_api_eauth":{"name":"salt_api_eauth","type":"str","level":"advanced","flags":0,"default_value":"sharedsecret","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"salt_api_password":{"name":"salt_api_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"salt_api_url":{"name":"salt_api_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"salt_api_username":{"name":"salt_api_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"str","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"str","level":"advanced","flags":0,"default_value":"30","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"str","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"str","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{}},{"name":"iostat","can_run":true,"error_string":"","module_options":{}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator_cli","can_run":true,"error_string":"","module_options":{"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"persist_interval":{"name":"persist_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how frequently to persist completed events","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{}},{"name":"restful","can_run":true,"error_string":"","module_options":{"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"nautilus":["balancer","crash","devicehealth","orchestrator_cli","progress","rbd_support","status","volumes"]}} 2024-04-17T22:29:15.543 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-04-17T22:29:15.543 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-04-17T22:29:15.543 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T22:29:15.669 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.680 7f1e43f2f700 1 Processor -- start 2024-04-17T22:29:15.669 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.680 7f1e43f2f700 1 -- start start 2024-04-17T22:29:15.669 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.681 7f1e43f2f700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f1e3c10e260 0x7f1e3c10e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:15.670 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.681 7f1e43f2f700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f1e3c10ece0 0x7f1e3c0f1750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:15.670 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.681 7f1e43f2f700 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e3c0f21d0 con 0x7f1e3c0f1d80 2024-04-17T22:29:15.670 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.681 7f1e43f2f700 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e3c0f9f30 con 0x7f1e3c10ece0 2024-04-17T22:29:15.670 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.681 7f1e43f2f700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1e3c0f9050 con 0x7f1e3c10e260 2024-04-17T22:29:15.670 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.681 7f1e40d2a700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f1e3c10e260 0x7f1e3c10e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:15.670 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.681 7f1e33fff700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f1e3c10ece0 0x7f1e3c0f1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:15.670 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.681 7f1e4152b700 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f1e3c0f1d80 0x7f1e3c1aa4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:34620/0 (socket says 172.21.15.102:34620) 2024-04-17T22:29:15.670 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.681 7f1e40d2a700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f1e3c10e260 0x7f1e3c10e6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:44138/0 (socket says 172.21.15.102:44138) 2024-04-17T22:29:15.671 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.681 7f1e4152b700 1 -- 172.21.15.102:0/1276425691 learned_addr learned my addr 172.21.15.102:0/1276425691 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-17T22:29:15.671 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.681 7f1e33fff700 1 --2- 172.21.15.102:0/1276425691 >> v2:172.21.15.102:3300/0 conn(0x7f1e3c10ece0 0x7f1e3c0f1750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:40746/0 (socket says 172.21.15.102:40746) 2024-04-17T22:29:15.671 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.681 7f1e40d2a700 1 -- 172.21.15.102:0/1276425691 >> v2:172.21.15.102:3300/0 conn(0x7f1e3c10ece0 msgr2=0x7f1e3c0f1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:29:15.671 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.681 7f1e40d2a700 1 --2- 172.21.15.102:0/1276425691 >> v2:172.21.15.102:3300/0 conn(0x7f1e3c10ece0 0x7f1e3c0f1750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:15.671 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.681 7f1e40d2a700 1 -- 172.21.15.102:0/1276425691 >> v1:172.21.15.102:6789/0 conn(0x7f1e3c0f1d80 legacy=0x7f1e3c1aa4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:15.671 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.681 7f1e40d2a700 1 -- 172.21.15.102:0/1276425691 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e3c0f23f0 con 0x7f1e3c10e260 2024-04-17T22:29:15.671 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.681 7f1e337fe700 1 -- 172.21.15.102:0/1276425691 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3080851484 0 0) 0x7f1e34001490 con 0x7f1e3c0f1d80 2024-04-17T22:29:15.671 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.681 7f1e33fff700 1 --2- 172.21.15.102:0/1276425691 >> v2:172.21.15.102:3300/0 conn(0x7f1e3c10ece0 0x7f1e3c0f1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:29:15.671 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.681 7f1e40d2a700 1 --2- 172.21.15.102:0/1276425691 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f1e3c10e260 0x7f1e3c10e6b0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7f1e38002f70 tx=0x7f1e3800b970).ready entity=mon.2 client_cookie=50b8dff3f00bf1ee server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:29:15.671 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.681 7f1e337fe700 1 -- 172.21.15.102:0/1276425691 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f1e3c0f23f0 con 0x7f1e3c10e260 2024-04-17T22:29:15.671 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.681 7f1e337fe700 1 -- 172.21.15.102:0/1276425691 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e3800d610 con 0x7f1e3c10e260 2024-04-17T22:29:15.671 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.681 7f1e337fe700 1 -- 172.21.15.102:0/1276425691 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f1e3800d9e0 con 0x7f1e3c10e260 2024-04-17T22:29:15.671 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.681 7f1e43f2f700 1 -- 172.21.15.102:0/1276425691 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f1e3c10e260 msgr2=0x7f1e3c10e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:15.672 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.681 7f1e43f2f700 1 --2- 172.21.15.102:0/1276425691 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f1e3c10e260 0x7f1e3c10e6b0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7f1e38002f70 tx=0x7f1e3800b970).stop 2024-04-17T22:29:15.672 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.682 7f1e43f2f700 1 -- 172.21.15.102:0/1276425691 shutdown_connections 2024-04-17T22:29:15.672 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.682 7f1e43f2f700 1 --2- 172.21.15.102:0/1276425691 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f1e3c10e260 0x7f1e3c10e6b0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:15.672 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.682 7f1e43f2f700 1 --2- 172.21.15.102:0/1276425691 >> v2:172.21.15.102:3300/0 conn(0x7f1e3c10ece0 0x7f1e3c0f1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:15.672 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.682 7f1e43f2f700 1 -- 172.21.15.102:0/1276425691 >> 172.21.15.102:0/1276425691 conn(0x7f1e3c1138c0 msgr2=0x7f1e3c0f4f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:29:15.672 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.682 7f1e43f2f700 1 -- 172.21.15.102:0/1276425691 shutdown_connections 2024-04-17T22:29:15.672 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.682 7f1e43f2f700 1 -- 172.21.15.102:0/1276425691 wait complete. 2024-04-17T22:29:15.693 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.705 7f1e43f2f700 1 Processor -- start 2024-04-17T22:29:15.693 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.705 7f1e43f2f700 1 -- start start 2024-04-17T22:29:15.693 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.705 7f1e43f2f700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1e3c0d5f40 0x7f1e3c0d6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:15.693 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.705 7f1e43f2f700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f1e3c0d69c0 0x7f1e3c1b3fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:15.694 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.705 7f1e43f2f700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f1e3c1b45e0 0x7f1e3c1b4a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:15.694 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.705 7f1e43f2f700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e3c0fa1d0 con 0x7f1e3c0d5f40 2024-04-17T22:29:15.694 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.705 7f1e43f2f700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e3c1b5060 con 0x7f1e3c0d69c0 2024-04-17T22:29:15.694 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.705 7f1e43f2f700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1e3c0c44a0 con 0x7f1e3c1b45e0 2024-04-17T22:29:15.694 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.705 7f1e40d2a700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1e3c0d5f40 0x7f1e3c0d6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:15.694 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.705 7f1e40d2a700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1e3c0d5f40 0x7f1e3c0d6390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:40762/0 (socket says 172.21.15.102:40762) 2024-04-17T22:29:15.694 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.705 7f1e40d2a700 1 -- 172.21.15.102:0/2136450490 learned_addr learned my addr 172.21.15.102:0/2136450490 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:29:15.694 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.705 7f1e4152b700 1 --2- 172.21.15.102:0/2136450490 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f1e3c1b45e0 0x7f1e3c1b4a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:15.694 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.705 7f1e33fff700 1 --2- 172.21.15.102:0/2136450490 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f1e3c0d69c0 0x7f1e3c1b3fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:15.694 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.706 7f1e40d2a700 1 -- 172.21.15.102:0/2136450490 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f1e3c1b45e0 msgr2=0x7f1e3c1b4a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:15.694 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.706 7f1e40d2a700 1 --2- 172.21.15.102:0/2136450490 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f1e3c1b45e0 0x7f1e3c1b4a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:15.694 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.706 7f1e40d2a700 1 -- 172.21.15.102:0/2136450490 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f1e3c0d69c0 msgr2=0x7f1e3c1b3fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:15.694 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.706 7f1e40d2a700 1 --2- 172.21.15.102:0/2136450490 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f1e3c0d69c0 0x7f1e3c1b3fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:15.695 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.706 7f1e40d2a700 1 -- 172.21.15.102:0/2136450490 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e3c0dd990 con 0x7f1e3c0d5f40 2024-04-17T22:29:15.695 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.706 7f1e40d2a700 1 --2- 172.21.15.102:0/2136450490 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1e3c0d5f40 0x7f1e3c0d6390 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0x7f1e3c0cfbb0 tx=0x7f1e3800a040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:29:15.695 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.706 7f1e31ffb700 1 -- 172.21.15.102:0/2136450490 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f1e3c0dd990 con 0x7f1e3c0d5f40 2024-04-17T22:29:15.695 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.706 7f1e31ffb700 1 -- 172.21.15.102:0/2136450490 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e38013910 con 0x7f1e3c0d5f40 2024-04-17T22:29:15.695 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.706 7f1e43f2f700 1 -- 172.21.15.102:0/2136450490 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e3c0d7ca0 con 0x7f1e3c0d5f40 2024-04-17T22:29:15.695 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.706 7f1e31ffb700 1 -- 172.21.15.102:0/2136450490 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f1e38013ce0 con 0x7f1e3c0d5f40 2024-04-17T22:29:15.695 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.706 7f1e43f2f700 1 -- 172.21.15.102:0/2136450490 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e3c1bb720 con 0x7f1e3c0d5f40 2024-04-17T22:29:15.695 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.706 7f1e31ffb700 1 -- 172.21.15.102:0/2136450490 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7f1e38027070 con 0x7f1e3c0d5f40 2024-04-17T22:29:15.695 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.706 7f1e31ffb700 1 -- 172.21.15.102:0/2136450490 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 3194+0+0 (secure 0 0 0) 0x7f1e38008cc0 con 0x7f1e3c0d5f40 2024-04-17T22:29:15.695 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.707 7f1e43f2f700 1 -- 172.21.15.102:0/2136450490 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e3c0f0820 con 0x7f1e3c0d5f40 2024-04-17T22:29:15.696 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.710 7f1e31ffb700 1 -- 172.21.15.102:0/2136450490 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+110455 (secure 0 0 0) 0x7f1e38013700 con 0x7f1e3c0d5f40 2024-04-17T22:29:15.801 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.815 7f1e43f2f700 1 -- 172.21.15.102:0/2136450490 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1e3c1ae6d0 con 0x7f1e3c0d5f40 2024-04-17T22:29:15.802 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.816 7f1e31ffb700 1 -- 172.21.15.102:0/2136450490 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+8761 (secure 0 0 0) 0x7f1e3800d400 con 0x7f1e3c0d5f40 2024-04-17T22:29:15.802 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-17T22:29:15.802 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":16,"fsid":"cb63e789-091d-49b1-b120-55f9db526871","created":"2024-04-17 22:28:48.209635","modified":"2024-04-17 22:29:08.789044","last_up_change":"2024-04-17 22:28:54.236215","last_in_change":"2024-04-17 22:28:54.236215","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":4,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-04-17 22:28:57.478545","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-17 22:29:06.144708","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-17 22:29:06.748924","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3351ef85-ade4-4b90-a8f4-213f4e00ba01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6812","nonce":42955}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6813","nonce":42955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6815","nonce":42955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6814","nonce":42955}]},"public_addr":"172.21.15.102:6812/42955","cluster_addr":"172.21.15.102:6813/42955","heartbeat_back_addr":"172.21.15.102:6815/42955","heartbeat_front_addr":"172.21.15.102:6814/42955","state":["exists","up"]},{"osd":1,"uuid":"04f67eac-f36b-48f5-982d-4cfc776596d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6808","nonce":42958}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6809","nonce":42958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6811","nonce":42958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6810","nonce":42958}]},"public_addr":"172.21.15.102:6808/42958","cluster_addr":"172.21.15.102:6809/42958","heartbeat_back_addr":"172.21.15.102:6811/42958","heartbeat_front_addr":"172.21.15.102:6810/42958","state":["exists","up"]},{"osd":2,"uuid":"d524204b-70b2-40a8-8f73-f9e7d44318af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6804","nonce":42954}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6805","nonce":42954}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6807","nonce":42954}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6806","nonce":42954}]},"public_addr":"172.21.15.102:6804/42954","cluster_addr":"172.21.15.102:6805/42954","heartbeat_back_addr":"172.21.15.102:6807/42954","heartbeat_front_addr":"172.21.15.102:6806/42954","state":["exists","up"]},{"osd":3,"uuid":"4fc9d20c-1ad0-484f-aafa-2c9a7574245b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6800","nonce":42953}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6801","nonce":42953}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6803","nonce":42953}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6802","nonce":42953}]},"public_addr":"172.21.15.102:6800/42953","cluster_addr":"172.21.15.102:6801/42953","heartbeat_back_addr":"172.21.15.102:6803/42953","heartbeat_front_addr":"172.21.15.102:6802/42953","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2024-04-17T22:29:15.804 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.817 7f1e43f2f700 1 -- 172.21.15.102:0/2136450490 >> v1:172.21.15.102:6816/42360 conn(0x7f1e1c0145a0 legacy=0x7f1e1c016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:15.804 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.817 7f1e43f2f700 1 -- 172.21.15.102:0/2136450490 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1e3c0d5f40 msgr2=0x7f1e3c0d6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:15.804 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.817 7f1e43f2f700 1 --2- 172.21.15.102:0/2136450490 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1e3c0d5f40 0x7f1e3c0d6390 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0x7f1e3c0cfbb0 tx=0x7f1e3800a040).stop 2024-04-17T22:29:15.804 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.817 7f1e43f2f700 1 -- 172.21.15.102:0/2136450490 shutdown_connections 2024-04-17T22:29:15.804 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.817 7f1e43f2f700 1 --2- 172.21.15.102:0/2136450490 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f1e3c1b45e0 0x7f1e3c1b4a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:15.804 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.817 7f1e43f2f700 1 --2- 172.21.15.102:0/2136450490 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1e3c0d5f40 0x7f1e3c0d6390 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:15.804 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.817 7f1e43f2f700 1 --2- 172.21.15.102:0/2136450490 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f1e3c0d69c0 0x7f1e3c1b3fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:15.804 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.817 7f1e43f2f700 1 -- 172.21.15.102:0/2136450490 >> 172.21.15.102:0/2136450490 conn(0x7f1e3c0c8bc0 msgr2=0x7f1e3c0c9410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:29:15.804 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.817 7f1e43f2f700 1 -- 172.21.15.102:0/2136450490 shutdown_connections 2024-04-17T22:29:15.804 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.817 7f1e43f2f700 1 -- 172.21.15.102:0/2136450490 wait complete. 2024-04-17T22:29:15.814 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-04-17T22:29:15.814 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T22:29:15.941 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.952 7ff3e311c700 1 Processor -- start 2024-04-17T22:29:15.941 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.952 7ff3e311c700 1 -- start start 2024-04-17T22:29:15.941 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.952 7ff3e311c700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7ff3dc10e260 0x7ff3dc10e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:15.941 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.952 7ff3e311c700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7ff3dc0f1f50 0x7ff3dc1ab500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:15.941 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.952 7ff3e311c700 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3dc0f23a0 con 0x7ff3dc10ece0 2024-04-17T22:29:15.941 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.952 7ff3e311c700 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3dc0f9f30 con 0x7ff3dc10e260 2024-04-17T22:29:15.942 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.952 7ff3e311c700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff3dc0f9050 con 0x7ff3dc0f1f50 2024-04-17T22:29:15.942 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.952 7ff3db7fe700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7ff3dc10e260 0x7ff3dc10e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:15.942 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.952 7ff3dbfff700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7ff3dc0f1f50 0x7ff3dc1ab500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:15.942 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.952 7ff3db7fe700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7ff3dc10e260 0x7ff3dc10e6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:40766/0 (socket says 172.21.15.102:40766) 2024-04-17T22:29:15.942 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.952 7ff3db7fe700 1 -- 172.21.15.102:0/545251546 learned_addr learned my addr 172.21.15.102:0/545251546 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:29:15.942 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.953 7ff3dbfff700 1 -- 172.21.15.102:0/545251546 >> v2:172.21.15.102:3300/0 conn(0x7ff3dc10e260 msgr2=0x7ff3dc10e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:29:15.942 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.953 7ff3dbfff700 1 --2- 172.21.15.102:0/545251546 >> v2:172.21.15.102:3300/0 conn(0x7ff3dc10e260 0x7ff3dc10e6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:15.942 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.953 7ff3dbfff700 1 -- 172.21.15.102:0/545251546 >> v1:172.21.15.102:6789/0 conn(0x7ff3dc10ece0 legacy=0x7ff3dc0f1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:15.942 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.953 7ff3dbfff700 1 -- 172.21.15.102:0/545251546 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3dc0f25c0 con 0x7ff3dc0f1f50 2024-04-17T22:29:15.942 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.953 7ff3db7fe700 1 --2- 172.21.15.102:0/545251546 >> v2:172.21.15.102:3300/0 conn(0x7ff3dc10e260 0x7ff3dc10e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:29:15.942 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.953 7ff3dbfff700 1 --2- 172.21.15.102:0/545251546 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7ff3dc0f1f50 0x7ff3dc1ab500 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7ff3dc0f4490 tx=0x7ff3d4009cc0).ready entity=mon.2 client_cookie=ad4cddededcbafc0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:29:15.942 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.953 7ff3da7fc700 1 -- 172.21.15.102:0/545251546 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff3dc0f25c0 con 0x7ff3dc0f1f50 2024-04-17T22:29:15.942 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.953 7ff3da7fc700 1 -- 172.21.15.102:0/545251546 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3d400b840 con 0x7ff3dc0f1f50 2024-04-17T22:29:15.943 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.953 7ff3da7fc700 1 -- 172.21.15.102:0/545251546 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff3d400ba20 con 0x7ff3dc0f1f50 2024-04-17T22:29:15.943 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.953 7ff3e311c700 1 -- 172.21.15.102:0/545251546 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7ff3dc0f1f50 msgr2=0x7ff3dc1ab500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:15.943 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.953 7ff3e311c700 1 --2- 172.21.15.102:0/545251546 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7ff3dc0f1f50 0x7ff3dc1ab500 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7ff3dc0f4490 tx=0x7ff3d4009cc0).stop 2024-04-17T22:29:15.943 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.953 7ff3e311c700 1 -- 172.21.15.102:0/545251546 shutdown_connections 2024-04-17T22:29:15.943 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.953 7ff3e311c700 1 --2- 172.21.15.102:0/545251546 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7ff3dc0f1f50 0x7ff3dc1ab500 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:15.943 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.953 7ff3e311c700 1 --2- 172.21.15.102:0/545251546 >> v2:172.21.15.102:3300/0 conn(0x7ff3dc10e260 0x7ff3dc10e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:15.943 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.953 7ff3e311c700 1 -- 172.21.15.102:0/545251546 >> 172.21.15.102:0/545251546 conn(0x7ff3dc1138c0 msgr2=0x7ff3dc0f4f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:29:15.943 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.953 7ff3e311c700 1 -- 172.21.15.102:0/545251546 shutdown_connections 2024-04-17T22:29:15.943 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.953 7ff3e311c700 1 -- 172.21.15.102:0/545251546 wait complete. 2024-04-17T22:29:15.963 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.976 7ff3e311c700 1 Processor -- start 2024-04-17T22:29:15.963 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.977 7ff3e311c700 1 -- start start 2024-04-17T22:29:15.965 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.977 7ff3e311c700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ff3dc0da760 0x7ff3dc0d5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:15.965 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.977 7ff3e311c700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff3dc0d5890 0x7ff3dc1b41d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:15.965 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.977 7ff3e311c700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7ff3dc0d5ce0 0x7ff3dc1b4800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:15.965 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.977 7ff3e311c700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3dc0fa1d0 con 0x7ff3dc0d5890 2024-04-17T22:29:15.965 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.977 7ff3e311c700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3dc0d9de0 con 0x7ff3dc0da760 2024-04-17T22:29:15.966 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.977 7ff3e311c700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff3dc0da130 con 0x7ff3dc0d5ce0 2024-04-17T22:29:15.966 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.977 7ff3dbfff700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7ff3dc0d5ce0 0x7ff3dc1b4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:15.966 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.977 7ff3db7fe700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ff3dc0da760 0x7ff3dc0d5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:15.966 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.977 7ff3dbfff700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7ff3dc0d5ce0 0x7ff3dc1b4800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:44172/0 (socket says 172.21.15.102:44172) 2024-04-17T22:29:15.966 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.977 7ff3daffd700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff3dc0d5890 0x7ff3dc1b41d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:15.966 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.977 7ff3dbfff700 1 -- 172.21.15.102:0/3396158078 learned_addr learned my addr 172.21.15.102:0/3396158078 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:29:15.966 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.977 7ff3db7fe700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ff3dc0da760 0x7ff3dc0d5260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:40188/0 (socket says 172.21.15.102:40188) 2024-04-17T22:29:15.966 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.977 7ff3db7fe700 1 -- 172.21.15.102:0/3396158078 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7ff3dc0d5ce0 msgr2=0x7ff3dc1b4800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:15.966 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.977 7ff3db7fe700 1 --2- 172.21.15.102:0/3396158078 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7ff3dc0d5ce0 0x7ff3dc1b4800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:15.966 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.977 7ff3db7fe700 1 -- 172.21.15.102:0/3396158078 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff3dc0d5890 msgr2=0x7ff3dc1b41d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:15.966 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.977 7ff3db7fe700 1 --2- 172.21.15.102:0/3396158078 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff3dc0d5890 0x7ff3dc1b41d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:15.966 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.977 7ff3db7fe700 1 -- 172.21.15.102:0/3396158078 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3dc1b4e30 con 0x7ff3dc0da760 2024-04-17T22:29:15.967 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.977 7ff3dbfff700 1 --2- 172.21.15.102:0/3396158078 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7ff3dc0d5ce0 0x7ff3dc1b4800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:29:15.967 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.977 7ff3daffd700 1 --2- 172.21.15.102:0/3396158078 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff3dc0d5890 0x7ff3dc1b41d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:29:15.967 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.977 7ff3db7fe700 1 --2- 172.21.15.102:0/3396158078 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ff3dc0da760 0x7ff3dc0d5260 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7ff3dc0c4e00 tx=0x7ff3d000eb00).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:29:15.967 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.978 7ff3d8ff9700 1 -- 172.21.15.102:0/3396158078 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff3dc1b4e30 con 0x7ff3dc0da760 2024-04-17T22:29:15.967 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.978 7ff3d8ff9700 1 -- 172.21.15.102:0/3396158078 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3d0012600 con 0x7ff3dc0da760 2024-04-17T22:29:15.967 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.978 7ff3e311c700 1 -- 172.21.15.102:0/3396158078 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3dc1bb940 con 0x7ff3dc0da760 2024-04-17T22:29:15.967 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.978 7ff3d8ff9700 1 -- 172.21.15.102:0/3396158078 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff3d000f040 con 0x7ff3dc0da760 2024-04-17T22:29:15.967 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.978 7ff3e311c700 1 -- 172.21.15.102:0/3396158078 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3dc1bbd20 con 0x7ff3dc0da760 2024-04-17T22:29:15.967 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.978 7ff3d8ff9700 1 -- 172.21.15.102:0/3396158078 <== mon.1 v2:172.21.15.102:3301/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7ff3d0022060 con 0x7ff3dc0da760 2024-04-17T22:29:15.967 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.978 7ff3d8ff9700 1 -- 172.21.15.102:0/3396158078 <== mon.1 v2:172.21.15.102:3301/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 3194+0+0 (secure 0 0 0) 0x7ff3d0010040 con 0x7ff3dc0da760 2024-04-17T22:29:15.967 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.979 7ff3e311c700 1 -- 172.21.15.102:0/3396158078 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3dc0f0820 con 0x7ff3dc0da760 2024-04-17T22:29:15.968 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:15.982 7ff3d8ff9700 1 -- 172.21.15.102:0/3396158078 <== mon.1 v2:172.21.15.102:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+110455 (secure 0 0 0) 0x7ff3d00123f0 con 0x7ff3dc0da760 2024-04-17T22:29:16.073 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.086 7ff3e311c700 1 -- 172.21.15.102:0/3396158078 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff3dc1b50f0 con 0x7ff3dc0da760 2024-04-17T22:29:16.073 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.087 7ff3d8ff9700 1 -- 172.21.15.102:0/3396158078 <== mon.1 v2:172.21.15.102:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+8761 (secure 0 0 0) 0x7ff3d0000c30 con 0x7ff3dc0da760 2024-04-17T22:29:16.074 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-17T22:29:16.074 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":16,"fsid":"cb63e789-091d-49b1-b120-55f9db526871","created":"2024-04-17 22:28:48.209635","modified":"2024-04-17 22:29:08.789044","last_up_change":"2024-04-17 22:28:54.236215","last_in_change":"2024-04-17 22:28:54.236215","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":4,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-04-17 22:28:57.478545","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-17 22:29:06.144708","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-17 22:29:06.748924","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3351ef85-ade4-4b90-a8f4-213f4e00ba01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6812","nonce":42955}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6813","nonce":42955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6815","nonce":42955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6814","nonce":42955}]},"public_addr":"172.21.15.102:6812/42955","cluster_addr":"172.21.15.102:6813/42955","heartbeat_back_addr":"172.21.15.102:6815/42955","heartbeat_front_addr":"172.21.15.102:6814/42955","state":["exists","up"]},{"osd":1,"uuid":"04f67eac-f36b-48f5-982d-4cfc776596d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6808","nonce":42958}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6809","nonce":42958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6811","nonce":42958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6810","nonce":42958}]},"public_addr":"172.21.15.102:6808/42958","cluster_addr":"172.21.15.102:6809/42958","heartbeat_back_addr":"172.21.15.102:6811/42958","heartbeat_front_addr":"172.21.15.102:6810/42958","state":["exists","up"]},{"osd":2,"uuid":"d524204b-70b2-40a8-8f73-f9e7d44318af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6804","nonce":42954}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6805","nonce":42954}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6807","nonce":42954}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6806","nonce":42954}]},"public_addr":"172.21.15.102:6804/42954","cluster_addr":"172.21.15.102:6805/42954","heartbeat_back_addr":"172.21.15.102:6807/42954","heartbeat_front_addr":"172.21.15.102:6806/42954","state":["exists","up"]},{"osd":3,"uuid":"4fc9d20c-1ad0-484f-aafa-2c9a7574245b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6800","nonce":42953}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6801","nonce":42953}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6803","nonce":42953}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6802","nonce":42953}]},"public_addr":"172.21.15.102:6800/42953","cluster_addr":"172.21.15.102:6801/42953","heartbeat_back_addr":"172.21.15.102:6803/42953","heartbeat_front_addr":"172.21.15.102:6802/42953","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2024-04-17T22:29:16.075 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.088 7ff3e311c700 1 -- 172.21.15.102:0/3396158078 >> v1:172.21.15.102:6816/42360 conn(0x7ff3bc0145a0 legacy=0x7ff3bc016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:16.075 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.088 7ff3e311c700 1 -- 172.21.15.102:0/3396158078 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ff3dc0da760 msgr2=0x7ff3dc0d5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:16.075 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.088 7ff3e311c700 1 --2- 172.21.15.102:0/3396158078 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ff3dc0da760 0x7ff3dc0d5260 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7ff3dc0c4e00 tx=0x7ff3d000eb00).stop 2024-04-17T22:29:16.075 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.088 7ff3e311c700 1 -- 172.21.15.102:0/3396158078 shutdown_connections 2024-04-17T22:29:16.076 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.088 7ff3e311c700 1 --2- 172.21.15.102:0/3396158078 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7ff3dc0d5ce0 0x7ff3dc1b4800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:16.076 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.088 7ff3e311c700 1 --2- 172.21.15.102:0/3396158078 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ff3dc0da760 0x7ff3dc0d5260 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:16.076 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.088 7ff3e311c700 1 --2- 172.21.15.102:0/3396158078 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff3dc0d5890 0x7ff3dc1b41d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:16.076 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.088 7ff3e311c700 1 -- 172.21.15.102:0/3396158078 >> 172.21.15.102:0/3396158078 conn(0x7ff3dc0c6a60 msgr2=0x7ff3dc1a8300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:29:16.076 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.088 7ff3e311c700 1 -- 172.21.15.102:0/3396158078 shutdown_connections 2024-04-17T22:29:16.076 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.088 7ff3e311c700 1 -- 172.21.15.102:0/3396158078 wait complete. 2024-04-17T22:29:16.086 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-04-17T22:29:16.087 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-04-17T22:29:16.087 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-04-17T22:29:16.087 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-04-17T22:29:16.220 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.233 7f51e7406700 1 Processor -- start 2024-04-17T22:29:16.220 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.233 7f51e7406700 1 -- start start 2024-04-17T22:29:16.221 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.234 7fcc23ee3700 1 Processor -- start 2024-04-17T22:29:16.221 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.234 7f51e7406700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f51e00f5750 0x7f51e010c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:16.221 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.234 7f51e7406700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f51e010c720 0x7f51e01a0be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:16.222 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.234 7f51e7406700 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51e01a1a10 con 0x7f51e010cb70 2024-04-17T22:29:16.222 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.234 7f51e7406700 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51e00f93c0 con 0x7f51e010c720 2024-04-17T22:29:16.222 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.234 7f51e7406700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f51e01a1c30 con 0x7f51e00f5750 2024-04-17T22:29:16.223 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.234 7fcc23ee3700 1 -- start start 2024-04-17T22:29:16.223 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.234 7fcc23ee3700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fcc1c0f5840 0x7fcc1c10b010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:16.223 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.234 7fcc23ee3700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fcc1c10b640 0x7fcc1c19faf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:16.223 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.234 7fcc23ee3700 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc1c103b40 con 0x7fcc1c10ba90 2024-04-17T22:29:16.223 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.234 7fcc23ee3700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc1c10bee0 con 0x7fcc1c0f5840 2024-04-17T22:29:16.223 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.234 7fcc23ee3700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcc1c0fa370 con 0x7fcc1c10b640 2024-04-17T22:29:16.223 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.234 7fcc20cde700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fcc1c0f5840 0x7fcc1c10b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:16.223 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.234 7fcc20cde700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fcc1c0f5840 0x7fcc1c10b010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:40198/0 (socket says 172.21.15.102:40198) 2024-04-17T22:29:16.223 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.234 7fcc20cde700 1 -- 172.21.15.102:0/1040911362 learned_addr learned my addr 172.21.15.102:0/1040911362 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:29:16.223 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.234 7fcc1bfff700 1 --2- 172.21.15.102:0/1040911362 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fcc1c10b640 0x7fcc1c19faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:16.223 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.235 7fcc1bfff700 1 -- 172.21.15.102:0/1040911362 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fcc1c0f5840 msgr2=0x7fcc1c10b010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:29:16.224 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.235 7f51dffff700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f51e00f5750 0x7f51e010c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:16.224 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.235 7f51dffff700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f51e00f5750 0x7f51e010c0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:44174/0 (socket says 172.21.15.102:44174) 2024-04-17T22:29:16.224 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.235 7f51dffff700 1 -- 172.21.15.102:0/2018475656 learned_addr learned my addr 172.21.15.102:0/2018475656 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:29:16.225 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.235 7fcc1bfff700 1 --2- 172.21.15.102:0/1040911362 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fcc1c0f5840 0x7fcc1c10b010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:16.225 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.235 7fcc1bfff700 1 -- 172.21.15.102:0/1040911362 >> v1:172.21.15.102:6789/0 conn(0x7fcc1c10ba90 legacy=0x7fcc1c1a0120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:29:16.225 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.235 7fcc1bfff700 1 -- 172.21.15.102:0/1040911362 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc1c1a0920 con 0x7fcc1c10b640 2024-04-17T22:29:16.225 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.235 7fcc20cde700 1 --2- 172.21.15.102:0/1040911362 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fcc1c0f5840 0x7fcc1c10b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:29:16.225 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.235 7fcc1bfff700 1 --2- 172.21.15.102:0/1040911362 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fcc1c10b640 0x7fcc1c19faf0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7fcc08002f70 tx=0x7fcc0800b9a0).ready entity=mon.2 client_cookie=7e11e14117716aff server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:29:16.225 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.235 7fcc1b7fe700 1 -- 172.21.15.102:0/1040911362 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fcc1c1a0920 con 0x7fcc1c10b640 2024-04-17T22:29:16.225 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.235 7fcc1b7fe700 1 -- 172.21.15.102:0/1040911362 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc0800d570 con 0x7fcc1c10b640 2024-04-17T22:29:16.225 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.235 7fcc1b7fe700 1 -- 172.21.15.102:0/1040911362 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fcc0800d940 con 0x7fcc1c10b640 2024-04-17T22:29:16.225 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.235 7fcc23ee3700 1 -- 172.21.15.102:0/1040911362 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fcc1c10b640 msgr2=0x7fcc1c19faf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:16.226 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.235 7fcc23ee3700 1 --2- 172.21.15.102:0/1040911362 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fcc1c10b640 0x7fcc1c19faf0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7fcc08002f70 tx=0x7fcc0800b9a0).stop 2024-04-17T22:29:16.226 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.236 7f51df7fe700 1 --2- 172.21.15.102:0/2018475656 >> v2:172.21.15.102:3300/0 conn(0x7f51e010c720 0x7f51e01a0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:16.226 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.236 7fcc23ee3700 1 -- 172.21.15.102:0/1040911362 shutdown_connections 2024-04-17T22:29:16.226 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.236 7fcc23ee3700 1 --2- 172.21.15.102:0/1040911362 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fcc1c0f5840 0x7fcc1c10b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:16.226 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.236 7fcc23ee3700 1 --2- 172.21.15.102:0/1040911362 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fcc1c10b640 0x7fcc1c19faf0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:16.226 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.236 7fcc23ee3700 1 -- 172.21.15.102:0/1040911362 >> 172.21.15.102:0/1040911362 conn(0x7fcc1c1138a0 msgr2=0x7fcc1c0f4f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:29:16.227 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.236 7f51deffd700 1 -- 172.21.15.102:0/2018475656 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 596037946 0 0) 0x7f51d8001490 con 0x7f51e010cb70 2024-04-17T22:29:16.227 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.236 7f51deffd700 1 -- 172.21.15.102:0/2018475656 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f51cc003270 con 0x7f51e010cb70 2024-04-17T22:29:16.227 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.236 7f51df7fe700 1 -- 172.21.15.102:0/2018475656 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f51e00f5750 msgr2=0x7f51e010c0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:29:16.227 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.236 7f51df7fe700 1 --2- 172.21.15.102:0/2018475656 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f51e00f5750 0x7f51e010c0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:16.228 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.236 7f51df7fe700 1 -- 172.21.15.102:0/2018475656 >> v1:172.21.15.102:6789/0 conn(0x7f51e010cb70 legacy=0x7f51e01a1210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:16.228 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.236 7f51df7fe700 1 -- 172.21.15.102:0/2018475656 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51e01aa310 con 0x7f51e010c720 2024-04-17T22:29:16.228 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.236 7f51df7fe700 1 --2- 172.21.15.102:0/2018475656 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f51e010c720 0x7f51e01a0be0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0x7f51e00f9570 tx=0x7f51d400ba20).ready entity=mon.0 client_cookie=64031f0a01cac5b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:29:16.228 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.236 7f51deffd700 1 -- 172.21.15.102:0/2018475656 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f51e01aa310 con 0x7f51e010c720 2024-04-17T22:29:16.228 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.236 7f51deffd700 1 -- 172.21.15.102:0/2018475656 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51d400d5c0 con 0x7f51e010c720 2024-04-17T22:29:16.228 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.236 7f51deffd700 1 -- 172.21.15.102:0/2018475656 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f51d400d9e0 con 0x7f51e010c720 2024-04-17T22:29:16.228 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.236 7f51e7406700 1 -- 172.21.15.102:0/2018475656 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f51e010c720 msgr2=0x7f51e01a0be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:16.228 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.236 7f51e7406700 1 --2- 172.21.15.102:0/2018475656 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f51e010c720 0x7f51e01a0be0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0x7f51e00f9570 tx=0x7f51d400ba20).stop 2024-04-17T22:29:16.228 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.236 7fcc23ee3700 1 -- 172.21.15.102:0/1040911362 shutdown_connections 2024-04-17T22:29:16.229 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.236 7fcc23ee3700 1 -- 172.21.15.102:0/1040911362 wait complete. 2024-04-17T22:29:16.229 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.237 7f51e7406700 1 -- 172.21.15.102:0/2018475656 shutdown_connections 2024-04-17T22:29:16.229 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.237 7f51e7406700 1 --2- 172.21.15.102:0/2018475656 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f51e00f5750 0x7f51e010c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:16.229 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.237 7f51e7406700 1 --2- 172.21.15.102:0/2018475656 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f51e010c720 0x7f51e01a0be0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:16.229 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.237 7f51e7406700 1 -- 172.21.15.102:0/2018475656 >> 172.21.15.102:0/2018475656 conn(0x7f51e01138a0 msgr2=0x7f51e00f4f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:29:16.230 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.237 7f51e7406700 1 -- 172.21.15.102:0/2018475656 shutdown_connections 2024-04-17T22:29:16.230 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.237 7f51e7406700 1 -- 172.21.15.102:0/2018475656 wait complete. 2024-04-17T22:29:16.231 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.240 7f167bd24700 1 Processor -- start 2024-04-17T22:29:16.231 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.240 7f167bd24700 1 -- start start 2024-04-17T22:29:16.231 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.240 7f167bd24700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f1674101b50 0x7f16741a3580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:16.231 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.240 7f167bd24700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f16741a9260 0x7f16741ab6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:16.231 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.240 7f167bd24700 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1674113fc0 con 0x7f1674101700 2024-04-17T22:29:16.231 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.240 7f167bd24700 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1674107f50 con 0x7f16741a9260 2024-04-17T22:29:16.231 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.240 7f167bd24700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1674107be0 con 0x7f1674101b50 2024-04-17T22:29:16.231 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.240 7f1679320700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f16741a9260 0x7f16741ab6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:16.231 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.241 7f1679320700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f16741a9260 0x7f16741ab6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:40800/0 (socket says 172.21.15.102:40800) 2024-04-17T22:29:16.231 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.241 7f1679320700 1 -- 172.21.15.102:0/528358125 learned_addr learned my addr 172.21.15.102:0/528358125 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:29:16.231 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.241 7f1679320700 1 -- 172.21.15.102:0/528358125 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f1674101b50 msgr2=0x7f16741a3580 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-17T22:29:16.232 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.241 7f1679320700 1 --2- 172.21.15.102:0/528358125 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f1674101b50 0x7f16741a3580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2024-04-17T22:29:16.232 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.241 7f1679320700 1 -- 172.21.15.102:0/528358125 >> v1:172.21.15.102:6789/0 conn(0x7f1674101700 legacy=0x7f16741a2d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:16.232 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.241 7f1679320700 1 -- 172.21.15.102:0/528358125 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16741a3bb0 con 0x7f16741a9260 2024-04-17T22:29:16.232 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.241 7f1679320700 1 --2- 172.21.15.102:0/528358125 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f16741a9260 0x7f16741ab6c0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 rx=0x7f166c0032f0 tx=0x7f166c009de0).ready entity=mon.0 client_cookie=65fa62e19afbaa88 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:29:16.232 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.241 7f3added2700 1 Processor -- start 2024-04-17T22:29:16.232 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.242 7f16737fe700 1 -- 172.21.15.102:0/528358125 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f16741a3bb0 con 0x7f16741a9260 2024-04-17T22:29:16.233 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.242 7f16737fe700 1 -- 172.21.15.102:0/528358125 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f166c00b8d0 con 0x7f16741a9260 2024-04-17T22:29:16.233 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.242 7f16737fe700 1 -- 172.21.15.102:0/528358125 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f166c00bb40 con 0x7f16741a9260 2024-04-17T22:29:16.233 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.241 7f3added2700 1 -- start start 2024-04-17T22:29:16.233 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.242 7f3added2700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f3ad81a0eb0 0x7f3ad81ac880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:16.233 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.242 7f3added2700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f3ad81aceb0 0x7f3ad81af310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:16.233 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.242 7f3added2700 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ad80f9870 con 0x7f3ad80eff40 2024-04-17T22:29:16.234 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.242 7f3added2700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ad81a1300 con 0x7f3ad81a0eb0 2024-04-17T22:29:16.234 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.242 7f3added2700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3ad81b0950 con 0x7f3ad81aceb0 2024-04-17T22:29:16.234 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.242 7f3ad64c4700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f3ad81a0eb0 0x7f3ad81ac880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:16.234 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.242 7f3ad64c4700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f3ad81a0eb0 0x7f3ad81ac880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:40212/0 (socket says 172.21.15.102:40212) 2024-04-17T22:29:16.234 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.242 7f3ad64c4700 1 -- 172.21.15.102:0/3609011618 learned_addr learned my addr 172.21.15.102:0/3609011618 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:29:16.234 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.243 7f167bd24700 1 -- 172.21.15.102:0/528358125 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f16741a9260 msgr2=0x7f16741ab6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:16.234 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.243 7f167bd24700 1 --2- 172.21.15.102:0/528358125 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f16741a9260 0x7f16741ab6c0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 rx=0x7f166c0032f0 tx=0x7f166c009de0).stop 2024-04-17T22:29:16.234 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.242 7f3ad64c4700 1 -- 172.21.15.102:0/3609011618 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f3ad81aceb0 msgr2=0x7f3ad81af310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:29:16.235 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.242 7f3ad64c4700 1 --2- 172.21.15.102:0/3609011618 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f3ad81aceb0 0x7f3ad81af310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2024-04-17T22:29:16.235 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.242 7f3ad64c4700 1 -- 172.21.15.102:0/3609011618 >> v1:172.21.15.102:6789/0 conn(0x7f3ad80eff40 legacy=0x7f3ad81a06b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:16.235 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.243 7f3ad64c4700 1 -- 172.21.15.102:0/3609011618 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ad81b0b00 con 0x7f3ad81a0eb0 2024-04-17T22:29:16.235 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.243 7f167bd24700 1 -- 172.21.15.102:0/528358125 shutdown_connections 2024-04-17T22:29:16.235 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.243 7f167bd24700 1 --2- 172.21.15.102:0/528358125 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f16741a9260 0x7f16741ab6c0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:16.235 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.243 7f167bd24700 1 --2- 172.21.15.102:0/528358125 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f1674101b50 0x7f16741a3580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2024-04-17T22:29:16.235 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.243 7f167bd24700 1 -- 172.21.15.102:0/528358125 >> 172.21.15.102:0/528358125 conn(0x7f1674106190 msgr2=0x7f1674116570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:29:16.236 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.243 7f3ad64c4700 1 --2- 172.21.15.102:0/3609011618 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f3ad81a0eb0 0x7f3ad81ac880 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7f3ad80f4f20 tx=0x7f3acc00baa0).ready entity=mon.1 client_cookie=406560a67dc42ab server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:29:16.236 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.243 7f167bd24700 1 -- 172.21.15.102:0/528358125 shutdown_connections 2024-04-17T22:29:16.236 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.243 7f167bd24700 1 -- 172.21.15.102:0/528358125 wait complete. 2024-04-17T22:29:16.237 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.243 7f3ad5cc3700 1 -- 172.21.15.102:0/3609011618 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f3ad81b0b00 con 0x7f3ad81a0eb0 2024-04-17T22:29:16.237 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.245 7f3ad5cc3700 1 -- 172.21.15.102:0/3609011618 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3acc0095c0 con 0x7f3ad81a0eb0 2024-04-17T22:29:16.238 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.245 7f3ad5cc3700 1 -- 172.21.15.102:0/3609011618 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f3acc009990 con 0x7f3ad81a0eb0 2024-04-17T22:29:16.238 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.245 7f3added2700 1 -- 172.21.15.102:0/3609011618 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f3ad81a0eb0 msgr2=0x7f3ad81ac880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:16.238 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.245 7f3added2700 1 --2- 172.21.15.102:0/3609011618 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f3ad81a0eb0 0x7f3ad81ac880 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7f3ad80f4f20 tx=0x7f3acc00baa0).stop 2024-04-17T22:29:16.238 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.245 7f3added2700 1 -- 172.21.15.102:0/3609011618 shutdown_connections 2024-04-17T22:29:16.238 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.245 7f3added2700 1 --2- 172.21.15.102:0/3609011618 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f3ad81aceb0 0x7f3ad81af310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2024-04-17T22:29:16.238 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.245 7f3added2700 1 --2- 172.21.15.102:0/3609011618 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f3ad81a0eb0 0x7f3ad81ac880 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:16.238 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.245 7f3added2700 1 -- 172.21.15.102:0/3609011618 >> 172.21.15.102:0/3609011618 conn(0x7f3ad8113020 msgr2=0x7f3ad80f46d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:29:16.238 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.245 7f3added2700 1 -- 172.21.15.102:0/3609011618 shutdown_connections 2024-04-17T22:29:16.238 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.245 7f3added2700 1 -- 172.21.15.102:0/3609011618 wait complete. 2024-04-17T22:29:16.253 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.266 7f167bd24700 1 Processor -- start 2024-04-17T22:29:16.253 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.266 7f167bd24700 1 -- start start 2024-04-17T22:29:16.255 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.267 7f167bd24700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f16740d7510 0x7f16740de9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:16.255 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.267 7f167bd24700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f16740df010 0x7f16741ad670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:16.255 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.267 7f167bd24700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f16740df460 0x7f16741adca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:16.255 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.267 7f167bd24700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16740dff00 con 0x7f16740d7510 2024-04-17T22:29:16.255 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.267 7f167bd24700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16740db220 con 0x7f16740df460 2024-04-17T22:29:16.255 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.267 7f167bd24700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f16741ae2d0 con 0x7f16740df010 2024-04-17T22:29:16.256 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.267 7f1678b1f700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f16740d7510 0x7f16740de9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:16.256 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.267 7f1673fff700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f16740df010 0x7f16741ad670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:16.256 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.267 7f1678b1f700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f16740d7510 0x7f16740de9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:40812/0 (socket says 172.21.15.102:40812) 2024-04-17T22:29:16.256 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.267 7f1678b1f700 1 -- 172.21.15.102:0/3639017345 learned_addr learned my addr 172.21.15.102:0/3639017345 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:29:16.256 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.267 7f1679320700 1 --2- 172.21.15.102:0/3639017345 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f16740df460 0x7f16741adca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:16.256 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.267 7f1678b1f700 1 -- 172.21.15.102:0/3639017345 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f16740df010 msgr2=0x7f16741ad670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:16.256 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.267 7f1678b1f700 1 --2- 172.21.15.102:0/3639017345 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f16740df010 0x7f16741ad670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:16.256 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.267 7f1678b1f700 1 -- 172.21.15.102:0/3639017345 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f16740df460 msgr2=0x7f16741adca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:16.256 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.267 7f1678b1f700 1 --2- 172.21.15.102:0/3639017345 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f16740df460 0x7f16741adca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:16.256 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.267 7f1678b1f700 1 -- 172.21.15.102:0/3639017345 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16741bb820 con 0x7f16740d7510 2024-04-17T22:29:16.256 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.267 7f1673fff700 1 --2- 172.21.15.102:0/3639017345 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f16740df010 0x7f16741ad670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:29:16.256 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.267 7f1678b1f700 1 --2- 172.21.15.102:0/3639017345 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f16740d7510 0x7f16740de9e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 rx=0x7f16740c70e0 tx=0x7f166800c810).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:29:16.256 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.268 7f1671ffb700 1 -- 172.21.15.102:0/3639017345 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f16741bb820 con 0x7f16740d7510 2024-04-17T22:29:16.256 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.268 7f1671ffb700 1 -- 172.21.15.102:0/3639017345 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1668007240 con 0x7f16740d7510 2024-04-17T22:29:16.256 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.268 7f167bd24700 1 -- 172.21.15.102:0/3639017345 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16741bba20 con 0x7f16740d7510 2024-04-17T22:29:16.257 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.268 7f1671ffb700 1 -- 172.21.15.102:0/3639017345 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f1668006710 con 0x7f16740d7510 2024-04-17T22:29:16.257 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.268 7f167bd24700 1 -- 172.21.15.102:0/3639017345 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16741bbe00 con 0x7f16740d7510 2024-04-17T22:29:16.257 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.268 7f1671ffb700 1 -- 172.21.15.102:0/3639017345 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7f166801a020 con 0x7f16740d7510 2024-04-17T22:29:16.257 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.268 7f1671ffb700 1 -- 172.21.15.102:0/3639017345 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 3194+0+0 (secure 0 0 0) 0x7f166800bdc0 con 0x7f16740d7510 2024-04-17T22:29:16.258 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.269 7f3added2700 1 Processor -- start 2024-04-17T22:29:16.259 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.269 7f3added2700 1 -- start start 2024-04-17T22:29:16.259 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.269 7f3added2700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3ad80c4550 0x7f3ad80cc680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:16.259 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.270 7f3added2700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f3ad80cccb0 0x7f3ad81b8010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:16.259 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.270 7f3added2700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f3ad80cd100 0x7f3ad81b8640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:16.259 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.270 7f3added2700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ad80c4ca0 con 0x7f3ad80c4550 2024-04-17T22:29:16.259 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.270 7f3added2700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ad81b8c70 con 0x7f3ad80cd100 2024-04-17T22:29:16.259 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.270 7f3added2700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3ad81b8ea0 con 0x7f3ad80cccb0 2024-04-17T22:29:16.259 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.270 7f3ad74c6700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f3ad80cd100 0x7f3ad81b8640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:16.259 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.270 7f3ad6cc5700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3ad80c4550 0x7f3ad80cc680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:16.259 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.270 7f3ad6cc5700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3ad80c4550 0x7f3ad80cc680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:40820/0 (socket says 172.21.15.102:40820) 2024-04-17T22:29:16.259 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.270 7f3ad74c6700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f3ad80cd100 0x7f3ad81b8640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:40232/0 (socket says 172.21.15.102:40232) 2024-04-17T22:29:16.259 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.270 7f3ad6cc5700 1 -- 172.21.15.102:0/863475035 learned_addr learned my addr 172.21.15.102:0/863475035 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:29:16.260 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.270 7f3ad64c4700 1 --2- 172.21.15.102:0/863475035 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f3ad80cccb0 0x7f3ad81b8010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:16.260 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.270 7f3ad6cc5700 1 -- 172.21.15.102:0/863475035 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f3ad80cccb0 msgr2=0x7f3ad81b8010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:16.260 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.270 7f3ad6cc5700 1 --2- 172.21.15.102:0/863475035 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f3ad80cccb0 0x7f3ad81b8010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:16.260 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.270 7f3ad6cc5700 1 -- 172.21.15.102:0/863475035 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f3ad80cd100 msgr2=0x7f3ad81b8640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:16.260 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.270 7f3ad6cc5700 1 --2- 172.21.15.102:0/863475035 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f3ad80cd100 0x7f3ad81b8640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:16.260 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.270 7f3ad6cc5700 1 -- 172.21.15.102:0/863475035 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ad81bf780 con 0x7f3ad80c4550 2024-04-17T22:29:16.260 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.270 7f167bd24700 1 -- 172.21.15.102:0/3639017345 --> v1:172.21.15.102:6812/42955 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f167408a3b0 con 0x7f16741a8a70 2024-04-17T22:29:16.261 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.270 7f3ad6cc5700 1 --2- 172.21.15.102:0/863475035 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3ad80c4550 0x7f3ad80cc680 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 rx=0x7f3ac8000ed0 tx=0x7f3ac800c7b0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:29:16.261 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.270 7f3ac7fff700 1 -- 172.21.15.102:0/863475035 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f3ad81bf780 con 0x7f3ad80c4550 2024-04-17T22:29:16.261 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.271 7f3ac7fff700 1 -- 172.21.15.102:0/863475035 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ac800f470 con 0x7f3ad80c4550 2024-04-17T22:29:16.261 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.271 7f3added2700 1 -- 172.21.15.102:0/863475035 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ad81bf980 con 0x7f3ad80c4550 2024-04-17T22:29:16.261 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.271 7f3ac7fff700 1 -- 172.21.15.102:0/863475035 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f3ac800f840 con 0x7f3ad80c4550 2024-04-17T22:29:16.262 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.271 7f1671ffb700 1 -- 172.21.15.102:0/3639017345 <== osd.0 v1:172.21.15.102:6812/42955 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+5151 (unknown 0 0 4057655604) 0x7f166c00b8d0 con 0x7f16741a8a70 2024-04-17T22:29:16.262 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.271 7f3added2700 1 -- 172.21.15.102:0/863475035 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ad81bfd60 con 0x7f3ad80c4550 2024-04-17T22:29:16.262 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.271 7f3ac7fff700 1 -- 172.21.15.102:0/863475035 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7f3ac800fa40 con 0x7f3ad80c4550 2024-04-17T22:29:16.262 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.272 7f3ac7fff700 1 -- 172.21.15.102:0/863475035 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 3194+0+0 (secure 0 0 0) 0x7f3ac800fd20 con 0x7f3ad80c4550 2024-04-17T22:29:16.262 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.272 7f3added2700 1 -- 172.21.15.102:0/863475035 --> v1:172.21.15.102:6804/42954 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3ad808a3b0 con 0x7f3ad80d8740 2024-04-17T22:29:16.264 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.278 7f167bd24700 1 -- 172.21.15.102:0/3639017345 --> v1:172.21.15.102:6812/42955 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f167408f6e0 con 0x7f16741a8a70 2024-04-17T22:29:16.264 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.278 7f1671ffb700 1 -- 172.21.15.102:0/3639017345 <== osd.0 v1:172.21.15.102:6812/42955 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (unknown 0 0 2652970471) 0x7f167408f6e0 con 0x7f16741a8a70 2024-04-17T22:29:16.265 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.278 7f3ac7fff700 1 -- 172.21.15.102:0/863475035 <== osd.2 v1:172.21.15.102:6804/42954 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+5151 (unknown 0 0 4057655604) 0x7f3ad0009660 con 0x7f3ad80d8740 2024-04-17T22:29:16.265 INFO:teuthology.orchestra.run.smithi102.stdout:38654705671 2024-04-17T22:29:16.266 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.280 7fcc23ee3700 1 Processor -- start 2024-04-17T22:29:16.267 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.281 7f51e7406700 1 Processor -- start 2024-04-17T22:29:16.267 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.281 7f51e7406700 1 -- start start 2024-04-17T22:29:16.268 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.280 7fcc23ee3700 1 -- start start 2024-04-17T22:29:16.268 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.281 7fcc23ee3700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fcc1c0dbdb0 0x7fcc1c1b1950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:16.268 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.281 7fcc23ee3700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcc1c1b1f80 0x7fcc1c0ca710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:16.268 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.281 7fcc23ee3700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fcc1c1b23d0 0x7fcc1c0cad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:16.268 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.281 7fcc23ee3700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc1c0cd5d0 con 0x7fcc1c1b1f80 2024-04-17T22:29:16.268 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.281 7fcc23ee3700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc1c0dc3e0 con 0x7fcc1c0dbdb0 2024-04-17T22:29:16.268 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.281 7fcc23ee3700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcc1c0dc610 con 0x7fcc1c1b23d0 2024-04-17T22:29:16.268 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.281 7fcc20cde700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fcc1c0dbdb0 0x7fcc1c1b1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:16.268 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.281 7fcc20cde700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fcc1c0dbdb0 0x7fcc1c1b1950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:40248/0 (socket says 172.21.15.102:40248) 2024-04-17T22:29:16.268 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.281 7fcc20cde700 1 -- 172.21.15.102:0/1131399471 learned_addr learned my addr 172.21.15.102:0/1131399471 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:29:16.268 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.281 7fcc1bfff700 1 --2- 172.21.15.102:0/1131399471 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcc1c1b1f80 0x7fcc1c0ca710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:16.268 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.281 7fcc214df700 1 --2- 172.21.15.102:0/1131399471 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fcc1c1b23d0 0x7fcc1c0cad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:16.269 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.281 7f51e7406700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f51e01a90a0 0x7f51e01a94f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:16.269 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.281 7f51e7406700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f51e01a9b20 0x7f51e01af8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:16.269 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.281 7f51e7406700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f51e01afed0 0x7f51e01b0320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:16.269 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.281 7f51e7406700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51e008a9c0 con 0x7f51e01afed0 2024-04-17T22:29:16.269 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.281 7f51e7406700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51e00c9070 con 0x7f51e01a9b20 2024-04-17T22:29:16.269 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.281 7f51e7406700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f51e00c8b00 con 0x7f51e01a90a0 2024-04-17T22:29:16.270 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.281 7fcc20cde700 1 -- 172.21.15.102:0/1131399471 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fcc1c1b23d0 msgr2=0x7fcc1c0cad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:16.270 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.281 7fcc20cde700 1 --2- 172.21.15.102:0/1131399471 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fcc1c1b23d0 0x7fcc1c0cad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:16.270 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.281 7fcc20cde700 1 -- 172.21.15.102:0/1131399471 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcc1c1b1f80 msgr2=0x7fcc1c0ca710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:16.270 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.281 7fcc20cde700 1 --2- 172.21.15.102:0/1131399471 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcc1c1b1f80 0x7fcc1c0ca710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:16.270 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.282 7fcc20cde700 1 -- 172.21.15.102:0/1131399471 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc1c0cb370 con 0x7fcc1c0dbdb0 2024-04-17T22:29:16.270 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.282 7fcc1bfff700 1 --2- 172.21.15.102:0/1131399471 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcc1c1b1f80 0x7fcc1c0ca710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:29:16.270 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.282 7fcc214df700 1 --2- 172.21.15.102:0/1131399471 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fcc1c1b23d0 0x7fcc1c0cad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:29:16.270 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.282 7f51df7fe700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f51e01a9b20 0x7f51e01af8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:16.271 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.282 7f51df7fe700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f51e01a9b20 0x7f51e01af8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:40252/0 (socket says 172.21.15.102:40252) 2024-04-17T22:29:16.271 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.282 7f51df7fe700 1 -- 172.21.15.102:0/1897302694 learned_addr learned my addr 172.21.15.102:0/1897302694 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:29:16.271 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.282 7f51e4a02700 1 --2- 172.21.15.102:0/1897302694 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f51e01afed0 0x7f51e01b0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:16.271 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.282 7fcc20cde700 1 --2- 172.21.15.102:0/1131399471 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fcc1c0dbdb0 0x7fcc1c1b1950 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7fcc1c0d6080 tx=0x7fcc1000ee70).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:29:16.271 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.282 7f51e4a02700 1 -- 172.21.15.102:0/1897302694 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f51e01a90a0 msgr2=0x7f51e01a94f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-17T22:29:16.272 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.282 7f51e4a02700 1 --2- 172.21.15.102:0/1897302694 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f51e01a90a0 0x7f51e01a94f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-04-17T22:29:16.272 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.282 7f51e4a02700 1 -- 172.21.15.102:0/1897302694 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f51e01a9b20 msgr2=0x7f51e01af8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:16.272 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.282 7f51e4a02700 1 --2- 172.21.15.102:0/1897302694 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f51e01a9b20 0x7f51e01af8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:16.272 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.282 7f51e4a02700 1 -- 172.21.15.102:0/1897302694 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51e00d9c50 con 0x7f51e01afed0 2024-04-17T22:29:16.272 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.282 7fcc19ffb700 1 -- 172.21.15.102:0/1131399471 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fcc1c0cb370 con 0x7fcc1c0dbdb0 2024-04-17T22:29:16.272 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.282 7fcc19ffb700 1 -- 172.21.15.102:0/1131399471 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc1000db50 con 0x7fcc1c0dbdb0 2024-04-17T22:29:16.272 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.282 7fcc19ffb700 1 -- 172.21.15.102:0/1131399471 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fcc10010440 con 0x7fcc1c0dbdb0 2024-04-17T22:29:16.273 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.282 7f51e4a02700 1 --2- 172.21.15.102:0/1897302694 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f51e01afed0 0x7f51e01b0320 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0x7f51e00df510 tx=0x7f51d800a340).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:29:16.273 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.282 7f51dd7fa700 1 -- 172.21.15.102:0/1897302694 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f51e00d9c50 con 0x7f51e01afed0 2024-04-17T22:29:16.273 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.283 7f51dd7fa700 1 -- 172.21.15.102:0/1897302694 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51d800ad50 con 0x7f51e01afed0 2024-04-17T22:29:16.273 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.283 7f51dd7fa700 1 -- 172.21.15.102:0/1897302694 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f51d8006a90 con 0x7f51e01afed0 2024-04-17T22:29:16.273 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.282 7fcc23ee3700 1 -- 172.21.15.102:0/1131399471 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc1c1bfad0 con 0x7fcc1c0dbdb0 2024-04-17T22:29:16.273 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.282 7fcc23ee3700 1 -- 172.21.15.102:0/1131399471 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc1c1bfeb0 con 0x7fcc1c0dbdb0 2024-04-17T22:29:16.274 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.283 7f167bd24700 1 -- 172.21.15.102:0/3639017345 >> v1:172.21.15.102:6812/42955 conn(0x7f16741a8a70 legacy=0x7f16740a7bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:16.274 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.283 7f51e7406700 1 -- 172.21.15.102:0/1897302694 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51e01b7010 con 0x7f51e01afed0 2024-04-17T22:29:16.274 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.283 7f51e7406700 1 -- 172.21.15.102:0/1897302694 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51e01b73f0 con 0x7f51e01afed0 2024-04-17T22:29:16.274 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.283 7f167bd24700 1 -- 172.21.15.102:0/3639017345 >> v1:172.21.15.102:6816/42360 conn(0x7f165c014540 legacy=0x7f165c016a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:16.274 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.283 7f167bd24700 1 -- 172.21.15.102:0/3639017345 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f16740d7510 msgr2=0x7f16740de9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:16.274 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.283 7f167bd24700 1 --2- 172.21.15.102:0/3639017345 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f16740d7510 0x7f16740de9e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 rx=0x7f16740c70e0 tx=0x7f166800c810).stop 2024-04-17T22:29:16.275 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.283 7fcc19ffb700 1 -- 172.21.15.102:0/1131399471 <== mon.1 v2:172.21.15.102:3301/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7fcc10020060 con 0x7fcc1c0dbdb0 2024-04-17T22:29:16.275 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.283 7fcc19ffb700 1 -- 172.21.15.102:0/1131399471 <== mon.1 v2:172.21.15.102:3301/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 3194+0+0 (secure 0 0 0) 0x7fcc10001bc0 con 0x7fcc1c0dbdb0 2024-04-17T22:29:16.275 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.283 7f1679320700 1 -- 172.21.15.102:0/3639017345 reap_dead start 2024-04-17T22:29:16.275 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.283 7fcc23ee3700 1 -- 172.21.15.102:0/1131399471 --> v1:172.21.15.102:6800/42953 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fcc1c08a3b0 con 0x7fcc1c0c9b60 2024-04-17T22:29:16.276 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.283 7f51dd7fa700 1 -- 172.21.15.102:0/1897302694 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7f51d8023060 con 0x7f51e01afed0 2024-04-17T22:29:16.276 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.283 7f51dd7fa700 1 -- 172.21.15.102:0/1897302694 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 3194+0+0 (secure 0 0 0) 0x7f51d80145d0 con 0x7f51e01afed0 2024-04-17T22:29:16.276 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.284 7f167bd24700 1 -- 172.21.15.102:0/3639017345 shutdown_connections 2024-04-17T22:29:16.276 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.284 7f167bd24700 1 -- 172.21.15.102:0/3639017345 >> 172.21.15.102:0/3639017345 conn(0x7f16740c6b70 msgr2=0x7f16741a8270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:29:16.276 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.285 7f167bd24700 1 -- 172.21.15.102:0/3639017345 shutdown_connections 2024-04-17T22:29:16.277 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.285 7f167bd24700 1 -- 172.21.15.102:0/3639017345 wait complete. 2024-04-17T22:29:16.277 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.285 7f51e7406700 1 -- 172.21.15.102:0/1897302694 --> v1:172.21.15.102:6808/42958 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f51bc0055f0 con 0x7f51bc001650 2024-04-17T22:29:16.277 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.286 7f51dd7fa700 1 -- 172.21.15.102:0/1897302694 <== osd.1 v1:172.21.15.102:6808/42958 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+5151 (unknown 0 0 4057655604) 0x7f51bc0055f0 con 0x7f51bc001650 2024-04-17T22:29:16.277 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.287 7f3added2700 1 -- 172.21.15.102:0/863475035 --> v1:172.21.15.102:6804/42954 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f3ad80d21f0 con 0x7f3ad80d8740 2024-04-17T22:29:16.277 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.287 7f3ac7fff700 1 -- 172.21.15.102:0/863475035 <== osd.2 v1:172.21.15.102:6804/42954 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (unknown 0 0 2652970471) 0x7f3ad80d21f0 con 0x7f3ad80d8740 2024-04-17T22:29:16.278 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.287 7fcc19ffb700 1 -- 172.21.15.102:0/1131399471 <== osd.3 v1:172.21.15.102:6800/42953 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+5151 (unknown 0 0 4057655604) 0x7fcc14009980 con 0x7fcc1c0c9b60 2024-04-17T22:29:16.278 INFO:teuthology.orchestra.run.smithi102.stdout:38654705671 2024-04-17T22:29:16.278 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.287 7f3added2700 1 -- 172.21.15.102:0/863475035 >> v1:172.21.15.102:6804/42954 conn(0x7f3ad80d8740 legacy=0x7f3ad80a6f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:16.279 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.287 7f3added2700 1 -- 172.21.15.102:0/863475035 >> v1:172.21.15.102:6816/42360 conn(0x7f3ab40145a0 legacy=0x7f3ab4016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:16.279 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.287 7f3added2700 1 -- 172.21.15.102:0/863475035 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3ad80c4550 msgr2=0x7f3ad80cc680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:16.279 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.287 7f3added2700 1 --2- 172.21.15.102:0/863475035 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3ad80c4550 0x7f3ad80cc680 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 rx=0x7f3ac8000ed0 tx=0x7f3ac800c7b0).stop 2024-04-17T22:29:16.279 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.287 7f3ad74c6700 1 -- 172.21.15.102:0/863475035 reap_dead start 2024-04-17T22:29:16.279 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.287 7f3added2700 1 -- 172.21.15.102:0/863475035 shutdown_connections 2024-04-17T22:29:16.279 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.287 7f3added2700 1 -- 172.21.15.102:0/863475035 >> 172.21.15.102:0/863475035 conn(0x7f3ad80c5530 msgr2=0x7f3ad80d7f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:29:16.279 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.287 7f3added2700 1 -- 172.21.15.102:0/863475035 shutdown_connections 2024-04-17T22:29:16.279 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.287 7f3added2700 1 -- 172.21.15.102:0/863475035 wait complete. 2024-04-17T22:29:16.280 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.293 7fcc23ee3700 1 -- 172.21.15.102:0/1131399471 --> v1:172.21.15.102:6800/42953 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fcc1c0f0800 con 0x7fcc1c0c9b60 2024-04-17T22:29:16.280 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.294 7fcc19ffb700 1 -- 172.21.15.102:0/1131399471 <== osd.3 v1:172.21.15.102:6800/42953 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (unknown 0 0 2652970471) 0x7fcc1c0f0800 con 0x7fcc1c0c9b60 2024-04-17T22:29:16.281 INFO:teuthology.orchestra.run.smithi102.stdout:38654705671 2024-04-17T22:29:16.281 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.294 7fcc23ee3700 1 -- 172.21.15.102:0/1131399471 >> v1:172.21.15.102:6800/42953 conn(0x7fcc1c0c9b60 legacy=0x7fcc1c0a7bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:16.281 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.294 7fcc23ee3700 1 -- 172.21.15.102:0/1131399471 >> v1:172.21.15.102:6816/42360 conn(0x7fcc040145a0 legacy=0x7fcc04016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:16.282 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.294 7fcc23ee3700 1 -- 172.21.15.102:0/1131399471 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fcc1c0dbdb0 msgr2=0x7fcc1c1b1950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:16.282 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.294 7fcc23ee3700 1 --2- 172.21.15.102:0/1131399471 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fcc1c0dbdb0 0x7fcc1c1b1950 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7fcc1c0d6080 tx=0x7fcc1000ee70).stop 2024-04-17T22:29:16.282 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.294 7fcc214df700 1 -- 172.21.15.102:0/1131399471 reap_dead start 2024-04-17T22:29:16.282 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.295 7fcc23ee3700 1 -- 172.21.15.102:0/1131399471 shutdown_connections 2024-04-17T22:29:16.282 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.295 7fcc23ee3700 1 -- 172.21.15.102:0/1131399471 >> 172.21.15.102:0/1131399471 conn(0x7fcc1c0e0eb0 msgr2=0x7fcc1c0c9360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:29:16.282 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.295 7fcc23ee3700 1 -- 172.21.15.102:0/1131399471 shutdown_connections 2024-04-17T22:29:16.282 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.295 7fcc23ee3700 1 -- 172.21.15.102:0/1131399471 wait complete. 2024-04-17T22:29:16.284 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.298 7f51e7406700 1 -- 172.21.15.102:0/1897302694 --> v1:172.21.15.102:6808/42958 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f51bc007010 con 0x7f51bc001650 2024-04-17T22:29:16.285 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.298 7f51dd7fa700 1 -- 172.21.15.102:0/1897302694 <== osd.1 v1:172.21.15.102:6808/42958 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (unknown 0 0 2374206864) 0x7f51bc007010 con 0x7f51bc001650 2024-04-17T22:29:16.285 INFO:teuthology.orchestra.run.smithi102.stdout:38654705670 2024-04-17T22:29:16.286 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.299 7f51e7406700 1 -- 172.21.15.102:0/1897302694 >> v1:172.21.15.102:6808/42958 conn(0x7f51bc001650 legacy=0x7f51bc003ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:16.286 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.299 7f51e7406700 1 -- 172.21.15.102:0/1897302694 >> v1:172.21.15.102:6816/42360 conn(0x7f51c00145f0 legacy=0x7f51c0016b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:16.286 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.299 7f51e7406700 1 -- 172.21.15.102:0/1897302694 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f51e01afed0 msgr2=0x7f51e01b0320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:16.286 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.299 7f51e7406700 1 --2- 172.21.15.102:0/1897302694 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f51e01afed0 0x7f51e01b0320 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0x7f51e00df510 tx=0x7f51d800a340).stop 2024-04-17T22:29:16.286 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.299 7f51e4a02700 1 -- 172.21.15.102:0/1897302694 reap_dead start 2024-04-17T22:29:16.286 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.299 7f51e7406700 1 -- 172.21.15.102:0/1897302694 shutdown_connections 2024-04-17T22:29:16.286 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.299 7f51e7406700 1 -- 172.21.15.102:0/1897302694 >> 172.21.15.102:0/1897302694 conn(0x7f51e00e0de0 msgr2=0x7f51e00de420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:29:16.286 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.299 7f51e7406700 1 -- 172.21.15.102:0/1897302694 shutdown_connections 2024-04-17T22:29:16.286 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.299 7f51e7406700 1 -- 172.21.15.102:0/1897302694 wait complete. 2024-04-17T22:29:16.288 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-17T22:29:16.289 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-17T22:29:16.294 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-17T22:29:16.298 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-17T22:29:16.426 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.438 7f82669f6700 1 Processor -- start 2024-04-17T22:29:16.426 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.438 7f82669f6700 1 -- start start 2024-04-17T22:29:16.426 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.438 7f82669f6700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f826010e1d0 0x7f826010e620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:16.426 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.439 7f82669f6700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f82600f1ec0 0x7f82601ab470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:16.426 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.439 7f82669f6700 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82600f2310 con 0x7f826010ec50 2024-04-17T22:29:16.426 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.439 7f82669f6700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82600f9ea0 con 0x7f826010e1d0 2024-04-17T22:29:16.426 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.439 7f82669f6700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f82600f8fc0 con 0x7f82600f1ec0 2024-04-17T22:29:16.426 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.439 7f825edea700 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f826010ec50 0x7f82600f16c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:34680/0 (socket says 172.21.15.102:34680) 2024-04-17T22:29:16.426 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.439 7f825f5eb700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f826010e1d0 0x7f826010e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:16.427 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.439 7f825edea700 1 -- 172.21.15.102:0/1015442665 learned_addr learned my addr 172.21.15.102:0/1015442665 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-17T22:29:16.427 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.439 7f825f5eb700 1 -- 172.21.15.102:0/1015442665 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f82600f1ec0 msgr2=0x7f82601ab470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:29:16.427 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.439 7f825f5eb700 1 --2- 172.21.15.102:0/1015442665 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f82600f1ec0 0x7f82601ab470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2024-04-17T22:29:16.427 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.439 7f825f5eb700 1 -- 172.21.15.102:0/1015442665 >> v1:172.21.15.102:6789/0 conn(0x7f826010ec50 legacy=0x7f82600f16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:29:16.427 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.439 7f825f5eb700 1 -- 172.21.15.102:0/1015442665 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82600f2530 con 0x7f826010e1d0 2024-04-17T22:29:16.427 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.439 7f825f5eb700 1 --2- 172.21.15.102:0/1015442665 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f826010e1d0 0x7f826010e620 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7f82600f4630 tx=0x7f825000ba50).ready entity=mon.1 client_cookie=db111f8f161cff2e server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:29:16.427 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.439 7f825e5e9700 1 -- 172.21.15.102:0/1015442665 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f82600f2530 con 0x7f826010e1d0 2024-04-17T22:29:16.427 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.439 7f825e5e9700 1 -- 172.21.15.102:0/1015442665 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f825000d570 con 0x7f826010e1d0 2024-04-17T22:29:16.427 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.439 7f825e5e9700 1 -- 172.21.15.102:0/1015442665 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f825000d940 con 0x7f826010e1d0 2024-04-17T22:29:16.427 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.439 7f82669f6700 1 -- 172.21.15.102:0/1015442665 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f826010e1d0 msgr2=0x7f826010e620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:16.427 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.439 7f82669f6700 1 --2- 172.21.15.102:0/1015442665 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f826010e1d0 0x7f826010e620 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7f82600f4630 tx=0x7f825000ba50).stop 2024-04-17T22:29:16.428 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.440 7f82669f6700 1 -- 172.21.15.102:0/1015442665 shutdown_connections 2024-04-17T22:29:16.428 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.440 7f82669f6700 1 --2- 172.21.15.102:0/1015442665 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f82600f1ec0 0x7f82601ab470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2024-04-17T22:29:16.428 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.440 7f82669f6700 1 --2- 172.21.15.102:0/1015442665 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f826010e1d0 0x7f826010e620 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:16.428 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.440 7f82669f6700 1 -- 172.21.15.102:0/1015442665 >> 172.21.15.102:0/1015442665 conn(0x7f8260113830 msgr2=0x7f82600f4ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:29:16.428 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.440 7f82669f6700 1 -- 172.21.15.102:0/1015442665 shutdown_connections 2024-04-17T22:29:16.428 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.440 7f82669f6700 1 -- 172.21.15.102:0/1015442665 wait complete. 2024-04-17T22:29:16.450 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.464 7f82669f6700 1 Processor -- start 2024-04-17T22:29:16.450 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.464 7f82669f6700 1 -- start start 2024-04-17T22:29:16.452 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.464 7f82669f6700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f82600d52a0 0x7f82600d56f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:16.452 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.464 7f82669f6700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f82600d5d20 0x7f82601b3d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:16.452 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.464 7f82669f6700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f82601b4350 0x7f82601b47a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:16.453 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.464 7f82669f6700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82600fa140 con 0x7f82600d5d20 2024-04-17T22:29:16.453 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.464 7f82669f6700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82600c47c0 con 0x7f82601b4350 2024-04-17T22:29:16.453 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.464 7f82669f6700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f82600cb450 con 0x7f82600d52a0 2024-04-17T22:29:16.453 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.465 7f825edea700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f82600d5d20 0x7f82601b3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:16.453 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.465 7f825edea700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f82600d5d20 0x7f82601b3d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:40850/0 (socket says 172.21.15.102:40850) 2024-04-17T22:29:16.453 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.465 7f825edea700 1 -- 172.21.15.102:0/1077725347 learned_addr learned my addr 172.21.15.102:0/1077725347 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:29:16.453 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.465 7f825f5eb700 1 --2- 172.21.15.102:0/1077725347 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f82600d52a0 0x7f82600d56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:16.453 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.465 7f825fdec700 1 --2- 172.21.15.102:0/1077725347 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f82601b4350 0x7f82601b47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:16.453 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.465 7f825edea700 1 -- 172.21.15.102:0/1077725347 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f82600d52a0 msgr2=0x7f82600d56f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:16.453 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.465 7f825edea700 1 --2- 172.21.15.102:0/1077725347 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f82600d52a0 0x7f82600d56f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:16.453 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.465 7f825edea700 1 -- 172.21.15.102:0/1077725347 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f82601b4350 msgr2=0x7f82601b47a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:16.454 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.465 7f825edea700 1 --2- 172.21.15.102:0/1077725347 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f82601b4350 0x7f82601b47a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:16.454 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.465 7f825edea700 1 -- 172.21.15.102:0/1077725347 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82600da940 con 0x7f82600d5d20 2024-04-17T22:29:16.454 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.465 7f825edea700 1 --2- 172.21.15.102:0/1077725347 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f82600d5d20 0x7f82601b3d20 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 rx=0x7f82600c25b0 tx=0x7f8254009810).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:29:16.454 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.465 7f825cde6700 1 -- 172.21.15.102:0/1077725347 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f82600da940 con 0x7f82600d5d20 2024-04-17T22:29:16.454 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.465 7f825cde6700 1 -- 172.21.15.102:0/1077725347 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f825400f510 con 0x7f82600d5d20 2024-04-17T22:29:16.454 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.465 7f825cde6700 1 -- 172.21.15.102:0/1077725347 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f825400f6f0 con 0x7f82600d5d20 2024-04-17T22:29:16.454 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.465 7f82669f6700 1 -- 172.21.15.102:0/1077725347 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82601bb410 con 0x7f82600d5d20 2024-04-17T22:29:16.454 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.465 7f82669f6700 1 -- 172.21.15.102:0/1077725347 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82601bb610 con 0x7f82600d5d20 2024-04-17T22:29:16.455 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.465 7fd13897d700 1 Processor -- start 2024-04-17T22:29:16.456 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.466 7fd13897d700 1 -- start start 2024-04-17T22:29:16.456 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.466 7fd13897d700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd1340f5870 0x7fd1341a08b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:16.456 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.466 7fd13897d700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd1341a0ee0 0x7fd1340f1730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:16.456 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.466 7fd13897d700 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1340f2560 con 0x7fd1341a1330 2024-04-17T22:29:16.456 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.466 7fd13897d700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1340fa1d0 con 0x7fd1341a0ee0 2024-04-17T22:29:16.456 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.466 7fd13897d700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd1340f9500 con 0x7fd1340f5870 2024-04-17T22:29:16.456 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.466 7fd131dfc700 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7fd1341a1330 0x7fd1340f1d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:34690/0 (socket says 172.21.15.102:34690) 2024-04-17T22:29:16.456 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.466 7fd131dfc700 1 -- 172.21.15.102:0/1637801531 learned_addr learned my addr 172.21.15.102:0/1637801531 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-17T22:29:16.456 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.466 7fd130dfa700 1 --2- 172.21.15.102:0/1637801531 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd1341a0ee0 0x7fd1340f1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:16.456 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.467 7fd1315fb700 1 --2- 172.21.15.102:0/1637801531 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd1340f5870 0x7fd1341a08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:16.456 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.467 7fd130dfa700 1 -- 172.21.15.102:0/1637801531 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd1340f5870 msgr2=0x7fd1341a08b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:29:16.456 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.467 7fd130dfa700 1 --2- 172.21.15.102:0/1637801531 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd1340f5870 0x7fd1341a08b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:16.456 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.467 7fd130dfa700 1 -- 172.21.15.102:0/1637801531 >> v1:172.21.15.102:6789/0 conn(0x7fd1341a1330 legacy=0x7fd1340f1d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:16.457 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.467 7fd123fff700 1 -- 172.21.15.102:0/1637801531 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2623288278 0 0) 0x7fd124001110 con 0x7fd1341a1330 2024-04-17T22:29:16.457 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.467 7fd130dfa700 1 -- 172.21.15.102:0/1637801531 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1341b0940 con 0x7fd1341a0ee0 2024-04-17T22:29:16.457 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.467 7fd130dfa700 1 --2- 172.21.15.102:0/1637801531 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd1341a0ee0 0x7fd1340f1730 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7fd1340f2780 tx=0x7fd12800ba50).ready entity=mon.1 client_cookie=c1c1777402103ac4 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:29:16.457 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.467 7f825cde6700 1 -- 172.21.15.102:0/1077725347 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7f8254018030 con 0x7f82600d5d20 2024-04-17T22:29:16.457 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.467 7f825cde6700 1 -- 172.21.15.102:0/1077725347 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 3194+0+0 (secure 0 0 0) 0x7f825401bd50 con 0x7f82600d5d20 2024-04-17T22:29:16.458 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.467 7fd123fff700 1 -- 172.21.15.102:0/1637801531 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd1341b0940 con 0x7fd1341a0ee0 2024-04-17T22:29:16.458 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.467 7fd123fff700 1 -- 172.21.15.102:0/1637801531 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd12800d570 con 0x7fd1341a0ee0 2024-04-17T22:29:16.458 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.467 7fd123fff700 1 -- 172.21.15.102:0/1637801531 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd12800d990 con 0x7fd1341a0ee0 2024-04-17T22:29:16.458 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.467 7fd13897d700 1 -- 172.21.15.102:0/1637801531 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd1341a0ee0 msgr2=0x7fd1340f1730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:16.458 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.467 7fd13897d700 1 --2- 172.21.15.102:0/1637801531 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd1341a0ee0 0x7fd1340f1730 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7fd1340f2780 tx=0x7fd12800ba50).stop 2024-04-17T22:29:16.458 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.468 7f82669f6700 1 -- 172.21.15.102:0/1077725347 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82600f0790 con 0x7f82600d5d20 2024-04-17T22:29:16.459 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.468 7fd13897d700 1 -- 172.21.15.102:0/1637801531 shutdown_connections 2024-04-17T22:29:16.459 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.468 7fd13897d700 1 --2- 172.21.15.102:0/1637801531 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd1340f5870 0x7fd1341a08b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:16.459 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.468 7fd13897d700 1 --2- 172.21.15.102:0/1637801531 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd1341a0ee0 0x7fd1340f1730 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:16.459 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.468 7fd13897d700 1 -- 172.21.15.102:0/1637801531 >> 172.21.15.102:0/1637801531 conn(0x7fd1341138a0 msgr2=0x7fd1340f4f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:29:16.459 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.468 7fd13897d700 1 -- 172.21.15.102:0/1637801531 shutdown_connections 2024-04-17T22:29:16.459 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.468 7fd13897d700 1 -- 172.21.15.102:0/1637801531 wait complete. 2024-04-17T22:29:16.460 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.471 7f825cde6700 1 -- 172.21.15.102:0/1077725347 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+110455 (secure 0 0 0) 0x7f825400f300 con 0x7f82600d5d20 2024-04-17T22:29:16.472 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.485 7f395ef2f700 1 Processor -- start 2024-04-17T22:29:16.472 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.485 7f395ef2f700 1 -- start start 2024-04-17T22:29:16.472 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.485 7f395ef2f700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f39580f5840 0x7f395810b010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:16.474 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.485 7f395ef2f700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f395810ba90 0x7f39581a02f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:16.474 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.485 7f395ef2f700 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3958103b40 con 0x7f395810b640 2024-04-17T22:29:16.474 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.485 7f395ef2f700 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f395810bee0 con 0x7f395810ba90 2024-04-17T22:29:16.474 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.485 7f395ef2f700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39580fa370 con 0x7f39580f5840 2024-04-17T22:29:16.474 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.486 7f3957fff700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f395810ba90 0x7f39581a02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:16.474 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.486 7f39577fe700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f39580f5840 0x7f395810b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:16.474 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.486 7f3957fff700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f395810ba90 0x7f39581a02f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:40852/0 (socket says 172.21.15.102:40852) 2024-04-17T22:29:16.474 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.486 7f39577fe700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f39580f5840 0x7f395810b010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:40284/0 (socket says 172.21.15.102:40284) 2024-04-17T22:29:16.474 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.486 7f3957fff700 1 -- 172.21.15.102:0/1460276075 learned_addr learned my addr 172.21.15.102:0/1460276075 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:29:16.474 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.486 7f39577fe700 1 -- 172.21.15.102:0/1460276075 >> v2:172.21.15.102:3300/0 conn(0x7f395810ba90 msgr2=0x7f39581a02f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:29:16.474 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.486 7f39577fe700 1 --2- 172.21.15.102:0/1460276075 >> v2:172.21.15.102:3300/0 conn(0x7f395810ba90 0x7f39581a02f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:16.475 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.486 7f39577fe700 1 -- 172.21.15.102:0/1460276075 >> v1:172.21.15.102:6789/0 conn(0x7f395810b640 legacy=0x7f395819faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:29:16.475 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.486 7f39577fe700 1 -- 172.21.15.102:0/1460276075 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39581a0920 con 0x7f39580f5840 2024-04-17T22:29:16.475 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.486 7f3957fff700 1 --2- 172.21.15.102:0/1460276075 >> v2:172.21.15.102:3300/0 conn(0x7f395810ba90 0x7f39581a02f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:29:16.475 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.486 7f39577fe700 1 --2- 172.21.15.102:0/1460276075 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f39580f5840 0x7f395810b010 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7f3948002f70 tx=0x7f394800b970).ready entity=mon.1 client_cookie=a032a8bbbaff6a36 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:29:16.475 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.486 7f39567fc700 1 -- 172.21.15.102:0/1460276075 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f39581a0920 con 0x7f39580f5840 2024-04-17T22:29:16.475 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.486 7f39567fc700 1 -- 172.21.15.102:0/1460276075 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f394800d570 con 0x7f39580f5840 2024-04-17T22:29:16.475 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.486 7f39567fc700 1 -- 172.21.15.102:0/1460276075 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f394800d940 con 0x7f39580f5840 2024-04-17T22:29:16.475 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.486 7f395ef2f700 1 -- 172.21.15.102:0/1460276075 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f39580f5840 msgr2=0x7f395810b010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:16.475 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.486 7f395ef2f700 1 --2- 172.21.15.102:0/1460276075 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f39580f5840 0x7f395810b010 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7f3948002f70 tx=0x7f394800b970).stop 2024-04-17T22:29:16.475 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.487 7f395ef2f700 1 -- 172.21.15.102:0/1460276075 shutdown_connections 2024-04-17T22:29:16.475 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.487 7f395ef2f700 1 --2- 172.21.15.102:0/1460276075 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f39580f5840 0x7f395810b010 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:16.475 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.487 7f395ef2f700 1 --2- 172.21.15.102:0/1460276075 >> v2:172.21.15.102:3300/0 conn(0x7f395810ba90 0x7f39581a02f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:16.475 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.487 7f395ef2f700 1 -- 172.21.15.102:0/1460276075 >> 172.21.15.102:0/1460276075 conn(0x7f39581138a0 msgr2=0x7f39580f4f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:29:16.475 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.487 7f395ef2f700 1 -- 172.21.15.102:0/1460276075 shutdown_connections 2024-04-17T22:29:16.475 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.487 7f395ef2f700 1 -- 172.21.15.102:0/1460276075 wait complete. 2024-04-17T22:29:16.483 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.495 7fd13897d700 1 Processor -- start 2024-04-17T22:29:16.483 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.495 7fd13897d700 1 -- start start 2024-04-17T22:29:16.483 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.495 7fd13897d700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd1340d51d0 0x7fd1340d5620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:16.483 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.495 7fd13897d700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd1340d5c50 0x7fd1341b85d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:16.483 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.495 7fd13897d700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd1341b8c00 0x7fd1341b9050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:16.483 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.495 7fd13897d700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1340dd010 con 0x7fd1340d51d0 2024-04-17T22:29:16.483 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.495 7fd13897d700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1340dc100 con 0x7fd1341b8c00 2024-04-17T22:29:16.483 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.495 7fd13897d700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd1340ddec0 con 0x7fd1340d5c50 2024-04-17T22:29:16.483 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.495 7fd131dfc700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd1341b8c00 0x7fd1341b9050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:16.483 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.495 7fd131dfc700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd1341b8c00 0x7fd1341b9050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:40298/0 (socket says 172.21.15.102:40298) 2024-04-17T22:29:16.483 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.495 7fd131dfc700 1 -- 172.21.15.102:0/2600849999 learned_addr learned my addr 172.21.15.102:0/2600849999 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:29:16.484 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.495 7fd131dfc700 1 -- 172.21.15.102:0/2600849999 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd1340d5c50 msgr2=0x7fd1341b85d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-17T22:29:16.484 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.496 7fd1315fb700 1 --2- 172.21.15.102:0/2600849999 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd1340d51d0 0x7fd1340d5620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:16.484 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.496 7fd131dfc700 1 --2- 172.21.15.102:0/2600849999 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd1340d5c50 0x7fd1341b85d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-04-17T22:29:16.484 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.496 7fd131dfc700 1 -- 172.21.15.102:0/2600849999 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd1340d51d0 msgr2=0x7fd1340d5620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:16.484 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.496 7fd131dfc700 1 --2- 172.21.15.102:0/2600849999 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd1340d51d0 0x7fd1340d5620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:16.484 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.496 7fd131dfc700 1 -- 172.21.15.102:0/2600849999 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1340ce5e0 con 0x7fd1341b8c00 2024-04-17T22:29:16.484 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.496 7fd1315fb700 1 --2- 172.21.15.102:0/2600849999 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd1340d51d0 0x7fd1340d5620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-04-17T22:29:16.484 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.496 7fd131dfc700 1 --2- 172.21.15.102:0/2600849999 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd1341b8c00 0x7fd1341b9050 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7fd124000bc0 tx=0x7fd12400c830).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:29:16.484 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.496 7fd122ffd700 1 -- 172.21.15.102:0/2600849999 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd1340ce5e0 con 0x7fd1341b8c00 2024-04-17T22:29:16.484 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.496 7fd122ffd700 1 -- 172.21.15.102:0/2600849999 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd12400ce20 con 0x7fd1341b8c00 2024-04-17T22:29:16.484 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.496 7fd122ffd700 1 -- 172.21.15.102:0/2600849999 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd124005620 con 0x7fd1341b8c00 2024-04-17T22:29:16.484 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.496 7fd13897d700 1 -- 172.21.15.102:0/2600849999 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1341bfdd0 con 0x7fd1341b8c00 2024-04-17T22:29:16.484 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.496 7fd13897d700 1 -- 172.21.15.102:0/2600849999 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1341c01b0 con 0x7fd1341b8c00 2024-04-17T22:29:16.485 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.497 7fd122ffd700 1 -- 172.21.15.102:0/2600849999 <== mon.1 v2:172.21.15.102:3301/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7fd124005820 con 0x7fd1341b8c00 2024-04-17T22:29:16.485 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.497 7fd122ffd700 1 -- 172.21.15.102:0/2600849999 <== mon.1 v2:172.21.15.102:3301/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 3194+0+0 (secure 0 0 0) 0x7fd124014d10 con 0x7fd1341b8c00 2024-04-17T22:29:16.485 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.498 7fd13897d700 1 -- 172.21.15.102:0/2600849999 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1340f0800 con 0x7fd1341b8c00 2024-04-17T22:29:16.487 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.501 7f2824eaf700 1 Processor -- start 2024-04-17T22:29:16.488 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.501 7f2824eaf700 1 -- start start 2024-04-17T22:29:16.488 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.502 7f2824eaf700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f282010c8f0 0x7f28201a62e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:16.488 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.502 7f2824eaf700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f28201a6910 0x7f28201a8d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:16.488 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.502 7f2824eaf700 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28201a1bf0 con 0x7f28200f5750 2024-04-17T22:29:16.488 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.502 7f2824eaf700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28200f93c0 con 0x7f282010c8f0 2024-04-17T22:29:16.489 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.502 7f2824eaf700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f282010cd40 con 0x7f28201a6910 2024-04-17T22:29:16.489 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.502 7f281db64700 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f28200f5750 0x7f282010c0f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:34698/0 (socket says 172.21.15.102:34698) 2024-04-17T22:29:16.489 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.502 7f281db64700 1 -- 172.21.15.102:0/1495558420 learned_addr learned my addr 172.21.15.102:0/1495558420 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-17T22:29:16.489 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.502 7f281d363700 1 --2- 172.21.15.102:0/1495558420 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f282010c8f0 0x7f28201a62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:16.489 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.502 7f281cb62700 1 -- 172.21.15.102:0/1495558420 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3436564078 0 0) 0x7f28201a1bf0 con 0x7f28200f5750 2024-04-17T22:29:16.489 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.502 7f281cb62700 1 -- 172.21.15.102:0/1495558420 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2800003270 con 0x7f28200f5750 2024-04-17T22:29:16.489 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.502 7f281d363700 1 -- 172.21.15.102:0/1495558420 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f28201a6910 msgr2=0x7f28201a8d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:29:16.490 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.502 7f281d363700 1 --2- 172.21.15.102:0/1495558420 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f28201a6910 0x7f28201a8d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2024-04-17T22:29:16.490 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.502 7f281d363700 1 -- 172.21.15.102:0/1495558420 >> v1:172.21.15.102:6789/0 conn(0x7f28200f5750 legacy=0x7f282010c0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:16.490 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.502 7f281d363700 1 -- 172.21.15.102:0/1495558420 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28201aa3b0 con 0x7f282010c8f0 2024-04-17T22:29:16.490 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.502 7f281d363700 1 --2- 172.21.15.102:0/1495558420 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f282010c8f0 0x7f28201a62e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7f2808002f70 tx=0x7f280800ba20).ready entity=mon.1 client_cookie=5864844e7cc3cb75 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:29:16.491 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.504 7f281cb62700 1 -- 172.21.15.102:0/1495558420 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f28201aa3b0 con 0x7f282010c8f0 2024-04-17T22:29:16.491 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.504 7f281cb62700 1 -- 172.21.15.102:0/1495558420 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f280800d570 con 0x7f282010c8f0 2024-04-17T22:29:16.491 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.504 7f281cb62700 1 -- 172.21.15.102:0/1495558420 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f280800d940 con 0x7f282010c8f0 2024-04-17T22:29:16.491 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.504 7f2824eaf700 1 -- 172.21.15.102:0/1495558420 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f282010c8f0 msgr2=0x7f28201a62e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:16.491 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.504 7f2824eaf700 1 --2- 172.21.15.102:0/1495558420 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f282010c8f0 0x7f28201a62e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7f2808002f70 tx=0x7f280800ba20).stop 2024-04-17T22:29:16.491 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.504 7fd122ffd700 1 -- 172.21.15.102:0/2600849999 <== mon.1 v2:172.21.15.102:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+110455 (secure 0 0 0) 0x7fd124000c30 con 0x7fd1341b8c00 2024-04-17T22:29:16.492 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.505 7f2824eaf700 1 -- 172.21.15.102:0/1495558420 shutdown_connections 2024-04-17T22:29:16.492 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.505 7f2824eaf700 1 --2- 172.21.15.102:0/1495558420 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f28201a6910 0x7f28201a8d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2024-04-17T22:29:16.492 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.505 7f2824eaf700 1 --2- 172.21.15.102:0/1495558420 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f282010c8f0 0x7f28201a62e0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:16.492 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.505 7f2824eaf700 1 -- 172.21.15.102:0/1495558420 >> 172.21.15.102:0/1495558420 conn(0x7f28201138a0 msgr2=0x7f28200f4f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:29:16.492 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.505 7f2824eaf700 1 -- 172.21.15.102:0/1495558420 shutdown_connections 2024-04-17T22:29:16.492 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.505 7f2824eaf700 1 -- 172.21.15.102:0/1495558420 wait complete. 2024-04-17T22:29:16.500 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.513 7f395ef2f700 1 Processor -- start 2024-04-17T22:29:16.501 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.514 7f395ef2f700 1 -- start start 2024-04-17T22:29:16.502 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.514 7f395ef2f700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f39581a9070 0x7f39581a94c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:16.502 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.514 7f395ef2f700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f39581a9af0 0x7f39581af8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:16.502 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.514 7f395ef2f700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f39581afee0 0x7f39581b0330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:16.502 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.514 7f395ef2f700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39580e2ce0 con 0x7f39581afee0 2024-04-17T22:29:16.502 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.514 7f395ef2f700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39580c3040 con 0x7f39581a9070 2024-04-17T22:29:16.502 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.514 7f395ef2f700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f39581b0960 con 0x7f39581a9af0 2024-04-17T22:29:16.502 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.514 7f39577fe700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f39581a9070 0x7f39581a94c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:16.502 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.514 7f39577fe700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f39581a9070 0x7f39581a94c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:40308/0 (socket says 172.21.15.102:40308) 2024-04-17T22:29:16.502 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.514 7f39577fe700 1 -- 172.21.15.102:0/2889067889 learned_addr learned my addr 172.21.15.102:0/2889067889 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:29:16.502 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.514 7f3957fff700 1 --2- 172.21.15.102:0/2889067889 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f39581afee0 0x7f39581b0330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:16.502 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.514 7f39577fe700 1 -- 172.21.15.102:0/2889067889 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f39581a9af0 msgr2=0x7f39581af8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:16.502 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.514 7f39577fe700 1 --2- 172.21.15.102:0/2889067889 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f39581a9af0 0x7f39581af8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-04-17T22:29:16.503 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.514 7f39577fe700 1 -- 172.21.15.102:0/2889067889 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f39581afee0 msgr2=0x7f39581b0330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:16.503 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.514 7f39577fe700 1 --2- 172.21.15.102:0/2889067889 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f39581afee0 0x7f39581b0330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:16.503 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.514 7f39577fe700 1 -- 172.21.15.102:0/2889067889 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39580dc950 con 0x7f39581a9070 2024-04-17T22:29:16.503 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.515 7f39577fe700 1 --2- 172.21.15.102:0/2889067889 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f39581a9070 0x7f39581a94c0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7f39580e1210 tx=0x7f39480068c0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:29:16.503 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.515 7f3954ff9700 1 -- 172.21.15.102:0/2889067889 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f39580dc950 con 0x7f39581a9070 2024-04-17T22:29:16.503 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.515 7f3954ff9700 1 -- 172.21.15.102:0/2889067889 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3948011820 con 0x7f39581a9070 2024-04-17T22:29:16.503 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.515 7f395ef2f700 1 -- 172.21.15.102:0/2889067889 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39580dc650 con 0x7f39581a9070 2024-04-17T22:29:16.503 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.515 7f3954ff9700 1 -- 172.21.15.102:0/2889067889 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f3948011bf0 con 0x7f39581a9070 2024-04-17T22:29:16.503 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.515 7f395ef2f700 1 -- 172.21.15.102:0/2889067889 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39581b7020 con 0x7f39581a9070 2024-04-17T22:29:16.504 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.516 7f3954ff9700 1 -- 172.21.15.102:0/2889067889 <== mon.1 v2:172.21.15.102:3301/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7f3948027460 con 0x7f39581a9070 2024-04-17T22:29:16.504 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.516 7f3954ff9700 1 -- 172.21.15.102:0/2889067889 <== mon.1 v2:172.21.15.102:3301/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 3194+0+0 (secure 0 0 0) 0x7f394802b160 con 0x7f39581a9070 2024-04-17T22:29:16.504 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.518 7f395ef2f700 1 -- 172.21.15.102:0/2889067889 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39580f0800 con 0x7f39581a9070 2024-04-17T22:29:16.507 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.521 7f3954ff9700 1 -- 172.21.15.102:0/2889067889 <== mon.1 v2:172.21.15.102:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+110455 (secure 0 0 0) 0x7f3948011610 con 0x7f39581a9070 2024-04-17T22:29:16.516 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.529 7f2824eaf700 1 Processor -- start 2024-04-17T22:29:16.516 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.530 7f2824eaf700 1 -- start start 2024-04-17T22:29:16.516 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.530 7f2824eaf700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f28201aaf10 0x7f28201ab360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:16.516 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.530 7f2824eaf700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f28201ab990 0x7f28201b19c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:16.516 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.530 7f2824eaf700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f28201b1ff0 0x7f28201b2440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:16.517 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.530 7f2824eaf700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28201a3cc0 con 0x7f28201aaf10 2024-04-17T22:29:16.517 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.530 7f2824eaf700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28201a3f00 con 0x7f28201b1ff0 2024-04-17T22:29:16.517 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.530 7f2824eaf700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f28201a34d0 con 0x7f28201ab990 2024-04-17T22:29:16.518 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.530 7f281e365700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f28201b1ff0 0x7f28201b2440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:16.518 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.530 7f281e365700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f28201b1ff0 0x7f28201b2440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:40318/0 (socket says 172.21.15.102:40318) 2024-04-17T22:29:16.518 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.530 7f281e365700 1 -- 172.21.15.102:0/3256353095 learned_addr learned my addr 172.21.15.102:0/3256353095 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:29:16.518 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.530 7f281d363700 1 --2- 172.21.15.102:0/3256353095 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f28201ab990 0x7f28201b19c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:16.518 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.530 7f281db64700 1 --2- 172.21.15.102:0/3256353095 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f28201aaf10 0x7f28201ab360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:16.518 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.530 7f281e365700 1 -- 172.21.15.102:0/3256353095 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f28201ab990 msgr2=0x7f28201b19c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:16.519 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.530 7f281e365700 1 --2- 172.21.15.102:0/3256353095 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f28201ab990 0x7f28201b19c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:16.519 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.530 7f281e365700 1 -- 172.21.15.102:0/3256353095 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f28201aaf10 msgr2=0x7f28201ab360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:16.519 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.530 7f281e365700 1 --2- 172.21.15.102:0/3256353095 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f28201aaf10 0x7f28201ab360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:16.519 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.530 7f281e365700 1 -- 172.21.15.102:0/3256353095 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28200f61c0 con 0x7f28201b1ff0 2024-04-17T22:29:16.519 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.531 7f281e365700 1 --2- 172.21.15.102:0/3256353095 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f28201b1ff0 0x7f28201b2440 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7f2810002f70 tx=0x7f2810009730).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:29:16.519 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.531 7f2816ffd700 1 -- 172.21.15.102:0/3256353095 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f28200f61c0 con 0x7f28201b1ff0 2024-04-17T22:29:16.519 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.531 7f2816ffd700 1 -- 172.21.15.102:0/3256353095 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2810013460 con 0x7f28201b1ff0 2024-04-17T22:29:16.519 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.531 7f2824eaf700 1 -- 172.21.15.102:0/3256353095 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28200f64e0 con 0x7f28201b1ff0 2024-04-17T22:29:16.519 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.531 7f2816ffd700 1 -- 172.21.15.102:0/3256353095 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f2810013830 con 0x7f28201b1ff0 2024-04-17T22:29:16.519 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.531 7f2824eaf700 1 -- 172.21.15.102:0/3256353095 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28200f69e0 con 0x7f28201b1ff0 2024-04-17T22:29:16.519 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.531 7f2816ffd700 1 -- 172.21.15.102:0/3256353095 <== mon.1 v2:172.21.15.102:3301/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7f2810013a30 con 0x7f28201b1ff0 2024-04-17T22:29:16.519 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.531 7f2816ffd700 1 -- 172.21.15.102:0/3256353095 <== mon.1 v2:172.21.15.102:3301/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 3194+0+0 (secure 0 0 0) 0x7f28100063a0 con 0x7f28201b1ff0 2024-04-17T22:29:16.520 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.532 7f2824eaf700 1 -- 172.21.15.102:0/3256353095 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28200f0800 con 0x7f28201b1ff0 2024-04-17T22:29:16.522 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.535 7f2816ffd700 1 -- 172.21.15.102:0/3256353095 <== mon.1 v2:172.21.15.102:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+110455 (secure 0 0 0) 0x7f281000d1c0 con 0x7f28201b1ff0 2024-04-17T22:29:16.611 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.625 7f82669f6700 1 -- 172.21.15.102:0/1077725347 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f82601ae440 con 0x7f82600d5d20 2024-04-17T22:29:16.612 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.625 7f825cde6700 1 -- 172.21.15.102:0/1077725347 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f82600f13d0 con 0x7f82600d5d20 2024-04-17T22:29:16.612 INFO:teuthology.orchestra.run.smithi102.stdout:38654705669 2024-04-17T22:29:16.613 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.626 7f82669f6700 1 -- 172.21.15.102:0/1077725347 >> v1:172.21.15.102:6816/42360 conn(0x7f82400145a0 legacy=0x7f8240016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:16.613 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.626 7f82669f6700 1 -- 172.21.15.102:0/1077725347 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f82600d5d20 msgr2=0x7f82601b3d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:16.613 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.626 7f82669f6700 1 --2- 172.21.15.102:0/1077725347 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f82600d5d20 0x7f82601b3d20 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 rx=0x7f82600c25b0 tx=0x7f8254009810).stop 2024-04-17T22:29:16.614 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.626 7f82669f6700 1 -- 172.21.15.102:0/1077725347 shutdown_connections 2024-04-17T22:29:16.614 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.626 7f82669f6700 1 --2- 172.21.15.102:0/1077725347 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f82601b4350 0x7f82601b47a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:16.614 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.626 7f82669f6700 1 --2- 172.21.15.102:0/1077725347 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f82600d52a0 0x7f82600d56f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:16.614 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.626 7f82669f6700 1 --2- 172.21.15.102:0/1077725347 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f82600d5d20 0x7f82601b3d20 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:16.614 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.627 7f82669f6700 1 -- 172.21.15.102:0/1077725347 >> 172.21.15.102:0/1077725347 conn(0x7f82600c6a60 msgr2=0x7f82600d6380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:29:16.614 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.627 7f82669f6700 1 -- 172.21.15.102:0/1077725347 shutdown_connections 2024-04-17T22:29:16.614 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.627 7f82669f6700 1 -- 172.21.15.102:0/1077725347 wait complete. 2024-04-17T22:29:16.619 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.633 7f395ef2f700 1 -- 172.21.15.102:0/2889067889 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f39580abc90 con 0x7f39581a9070 2024-04-17T22:29:16.620 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.633 7f3954ff9700 1 -- 172.21.15.102:0/2889067889 <== mon.1 v2:172.21.15.102:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f394800d360 con 0x7f39581a9070 2024-04-17T22:29:16.620 INFO:teuthology.orchestra.run.smithi102.stdout:38654705668 2024-04-17T22:29:16.621 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.635 7f395ef2f700 1 -- 172.21.15.102:0/2889067889 >> v1:172.21.15.102:6816/42360 conn(0x7f39380145a0 legacy=0x7f3938016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:16.622 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.635 7f395ef2f700 1 -- 172.21.15.102:0/2889067889 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f39581a9070 msgr2=0x7f39581a94c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:16.622 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.635 7f395ef2f700 1 --2- 172.21.15.102:0/2889067889 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f39581a9070 0x7f39581a94c0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7f39580e1210 tx=0x7f39480068c0).stop 2024-04-17T22:29:16.622 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.636 7f395ef2f700 1 -- 172.21.15.102:0/2889067889 shutdown_connections 2024-04-17T22:29:16.622 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.636 7f395ef2f700 1 --2- 172.21.15.102:0/2889067889 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f39581afee0 0x7f39581b0330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:16.623 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.636 7f395ef2f700 1 --2- 172.21.15.102:0/2889067889 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f39581a9070 0x7f39581a94c0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:16.623 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.636 7f395ef2f700 1 --2- 172.21.15.102:0/2889067889 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f39581a9af0 0x7f39581af8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-04-17T22:29:16.623 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.636 7f395ef2f700 1 -- 172.21.15.102:0/2889067889 >> 172.21.15.102:0/2889067889 conn(0x7f39580c5ac0 msgr2=0x7f39580c4810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:29:16.623 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.636 7f395ef2f700 1 -- 172.21.15.102:0/2889067889 shutdown_connections 2024-04-17T22:29:16.623 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.636 7f395ef2f700 1 -- 172.21.15.102:0/2889067889 wait complete. 2024-04-17T22:29:16.624 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705671 got 38654705669 for osd.2 2024-04-17T22:29:16.631 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.645 7f2824eaf700 1 -- 172.21.15.102:0/3256353095 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f28200abc90 con 0x7f28201b1ff0 2024-04-17T22:29:16.632 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.645 7f2816ffd700 1 -- 172.21.15.102:0/3256353095 <== mon.1 v2:172.21.15.102:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f2820114420 con 0x7f28201b1ff0 2024-04-17T22:29:16.632 INFO:teuthology.orchestra.run.smithi102.stdout:38654705669 2024-04-17T22:29:16.633 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705670 got 38654705668 for osd.1 2024-04-17T22:29:16.634 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.647 7f2824eaf700 1 -- 172.21.15.102:0/3256353095 >> v1:172.21.15.102:6816/42360 conn(0x7f28040145a0 legacy=0x7f2804016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:16.634 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.647 7f2824eaf700 1 -- 172.21.15.102:0/3256353095 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f28201b1ff0 msgr2=0x7f28201b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:16.634 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.647 7f2824eaf700 1 --2- 172.21.15.102:0/3256353095 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f28201b1ff0 0x7f28201b2440 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7f2810002f70 tx=0x7f2810009730).stop 2024-04-17T22:29:16.634 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.647 7f2824eaf700 1 -- 172.21.15.102:0/3256353095 shutdown_connections 2024-04-17T22:29:16.635 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.647 7f2824eaf700 1 --2- 172.21.15.102:0/3256353095 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f28201b1ff0 0x7f28201b2440 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:16.635 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.647 7f2824eaf700 1 --2- 172.21.15.102:0/3256353095 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f28201aaf10 0x7f28201ab360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:16.635 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.647 7f2824eaf700 1 --2- 172.21.15.102:0/3256353095 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f28201ab990 0x7f28201b19c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:16.635 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.647 7f2824eaf700 1 -- 172.21.15.102:0/3256353095 >> 172.21.15.102:0/3256353095 conn(0x7f28201a4e70 msgr2=0x7f28201a5990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:29:16.635 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.647 7f2824eaf700 1 -- 172.21.15.102:0/3256353095 shutdown_connections 2024-04-17T22:29:16.635 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.647 7f2824eaf700 1 -- 172.21.15.102:0/3256353095 wait complete. 2024-04-17T22:29:16.642 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.655 7fd13897d700 1 -- 172.21.15.102:0/2600849999 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fd1341b2cd0 con 0x7fd1341b8c00 2024-04-17T22:29:16.642 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.656 7fd122ffd700 1 -- 172.21.15.102:0/2600849999 <== mon.1 v2:172.21.15.102:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fd1240070d0 con 0x7fd1341b8c00 2024-04-17T22:29:16.642 INFO:teuthology.orchestra.run.smithi102.stdout:38654705669 2024-04-17T22:29:16.644 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.657 7fd13897d700 1 -- 172.21.15.102:0/2600849999 >> v1:172.21.15.102:6816/42360 conn(0x7fd10c0145f0 legacy=0x7fd10c016b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:16.644 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.657 7fd13897d700 1 -- 172.21.15.102:0/2600849999 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd1341b8c00 msgr2=0x7fd1341b9050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:16.644 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.657 7fd13897d700 1 --2- 172.21.15.102:0/2600849999 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd1341b8c00 0x7fd1341b9050 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7fd124000bc0 tx=0x7fd12400c830).stop 2024-04-17T22:29:16.644 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.657 7fd13897d700 1 -- 172.21.15.102:0/2600849999 shutdown_connections 2024-04-17T22:29:16.644 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.657 7fd13897d700 1 --2- 172.21.15.102:0/2600849999 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd1341b8c00 0x7fd1341b9050 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:16.644 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.657 7fd13897d700 1 --2- 172.21.15.102:0/2600849999 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd1340d51d0 0x7fd1340d5620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:16.644 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.657 7fd13897d700 1 --2- 172.21.15.102:0/2600849999 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd1340d5c50 0x7fd1341b85d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-04-17T22:29:16.645 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.657 7fd13897d700 1 -- 172.21.15.102:0/2600849999 >> 172.21.15.102:0/2600849999 conn(0x7fd1340c6a60 msgr2=0x7fd1340ced20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:29:16.645 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.657 7fd13897d700 1 -- 172.21.15.102:0/2600849999 shutdown_connections 2024-04-17T22:29:16.645 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:16.657 7fd13897d700 1 -- 172.21.15.102:0/2600849999 wait complete. 2024-04-17T22:29:16.645 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705671 got 38654705669 for osd.3 2024-04-17T22:29:16.654 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705671 got 38654705669 for osd.0 2024-04-17T22:29:17.625 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-17T22:29:17.634 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-17T22:29:17.646 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-17T22:29:17.655 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-17T22:29:17.775 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.786 7f668f71a700 1 Processor -- start 2024-04-17T22:29:17.775 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.786 7f668f71a700 1 -- start start 2024-04-17T22:29:17.775 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.786 7f668f71a700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f668810c8f0 0x7f66881a62e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:17.775 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.786 7f668f71a700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f66881a6910 0x7f66881a8d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:17.775 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.786 7f668f71a700 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66881a1bf0 con 0x7f66880f5750 2024-04-17T22:29:17.776 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.786 7f668f71a700 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66880f93c0 con 0x7f668810c8f0 2024-04-17T22:29:17.776 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.786 7f668f71a700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f668810cd40 con 0x7f66881a6910 2024-04-17T22:29:17.776 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.787 7f66877fe700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f668810c8f0 0x7f66881a62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:17.776 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.787 7f668cd16700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f66881a6910 0x7f66881a8d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:17.776 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.787 7f66877fe700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f668810c8f0 0x7f66881a62e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:40902/0 (socket says 172.21.15.102:40902) 2024-04-17T22:29:17.776 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.787 7f66877fe700 1 -- 172.21.15.102:0/1691402478 learned_addr learned my addr 172.21.15.102:0/1691402478 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:29:17.776 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.787 7f6686ffd700 1 -- 172.21.15.102:0/1691402478 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1591095316 0 0) 0x7f66881a1bf0 con 0x7f66880f5750 2024-04-17T22:29:17.776 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.787 7f6686ffd700 1 -- 172.21.15.102:0/1691402478 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6674003270 con 0x7f66880f5750 2024-04-17T22:29:17.776 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.787 7f668cd16700 1 -- 172.21.15.102:0/1691402478 >> v2:172.21.15.102:3300/0 conn(0x7f668810c8f0 msgr2=0x7f66881a62e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:29:17.776 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.787 7f668cd16700 1 --2- 172.21.15.102:0/1691402478 >> v2:172.21.15.102:3300/0 conn(0x7f668810c8f0 0x7f66881a62e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:17.776 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.787 7f668cd16700 1 -- 172.21.15.102:0/1691402478 >> v1:172.21.15.102:6789/0 conn(0x7f66880f5750 legacy=0x7f668810c0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:17.776 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.787 7f668cd16700 1 -- 172.21.15.102:0/1691402478 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66881aa3b0 con 0x7f66881a6910 2024-04-17T22:29:17.776 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.787 7f66877fe700 1 --2- 172.21.15.102:0/1691402478 >> v2:172.21.15.102:3300/0 conn(0x7f668810c8f0 0x7f66881a62e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:29:17.776 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.787 7f668cd16700 1 --2- 172.21.15.102:0/1691402478 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f66881a6910 0x7f66881a8d70 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7f66880f45f0 tx=0x7f6680009cc0).ready entity=mon.2 client_cookie=4266c6b4c9651862 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:29:17.777 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.787 7f6686ffd700 1 -- 172.21.15.102:0/1691402478 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f66881aa3b0 con 0x7f66881a6910 2024-04-17T22:29:17.777 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.788 7f6686ffd700 1 -- 172.21.15.102:0/1691402478 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f668000b840 con 0x7f66881a6910 2024-04-17T22:29:17.777 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.788 7f6686ffd700 1 -- 172.21.15.102:0/1691402478 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f668000ba20 con 0x7f66881a6910 2024-04-17T22:29:17.777 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.788 7f668f71a700 1 -- 172.21.15.102:0/1691402478 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f66881a6910 msgr2=0x7f66881a8d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:17.777 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.788 7f668f71a700 1 --2- 172.21.15.102:0/1691402478 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f66881a6910 0x7f66881a8d70 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7f66880f45f0 tx=0x7f6680009cc0).stop 2024-04-17T22:29:17.777 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.788 7f668f71a700 1 -- 172.21.15.102:0/1691402478 shutdown_connections 2024-04-17T22:29:17.777 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.788 7f668f71a700 1 --2- 172.21.15.102:0/1691402478 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f66881a6910 0x7f66881a8d70 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:17.777 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.788 7f668f71a700 1 --2- 172.21.15.102:0/1691402478 >> v2:172.21.15.102:3300/0 conn(0x7f668810c8f0 0x7f66881a62e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:17.777 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.788 7f668f71a700 1 -- 172.21.15.102:0/1691402478 >> 172.21.15.102:0/1691402478 conn(0x7f66881138a0 msgr2=0x7f66880f4f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:29:17.777 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.788 7f668f71a700 1 -- 172.21.15.102:0/1691402478 shutdown_connections 2024-04-17T22:29:17.777 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.788 7f668f71a700 1 -- 172.21.15.102:0/1691402478 wait complete. 2024-04-17T22:29:17.802 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.815 7f668f71a700 1 Processor -- start 2024-04-17T22:29:17.802 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.815 7f668f71a700 1 -- start start 2024-04-17T22:29:17.803 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.816 7f668f71a700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f66880cd820 0x7f66880d5e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:17.804 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.816 7f668f71a700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f66880d64c0 0x7f66881b1be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:17.804 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.816 7f668f71a700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f66880d6910 0x7f66881b2210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:17.804 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.816 7f668f71a700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66880cd1c0 con 0x7f66880d64c0 2024-04-17T22:29:17.804 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.816 7f668f71a700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66880cdc70 con 0x7f66880cd820 2024-04-17T22:29:17.804 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.816 7f668f71a700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f66881b2840 con 0x7f66880d6910 2024-04-17T22:29:17.804 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.816 7f668cd16700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f66880d6910 0x7f66881b2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:17.804 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.816 7f668cd16700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f66880d6910 0x7f66881b2210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:44292/0 (socket says 172.21.15.102:44292) 2024-04-17T22:29:17.804 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.816 7f668cd16700 1 -- 172.21.15.102:0/1277886520 learned_addr learned my addr 172.21.15.102:0/1277886520 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:29:17.804 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.816 7f6687fff700 1 --2- 172.21.15.102:0/1277886520 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f66880cd820 0x7f66880d5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:17.804 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.816 7f668cd16700 1 -- 172.21.15.102:0/1277886520 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f66880cd820 msgr2=0x7f66880d5e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:17.804 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.816 7f668cd16700 1 --2- 172.21.15.102:0/1277886520 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f66880cd820 0x7f66880d5e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:17.804 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.816 7f668cd16700 1 -- 172.21.15.102:0/1277886520 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f66880d64c0 msgr2=0x7f66881b1be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:17.804 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.816 7f668cd16700 1 --2- 172.21.15.102:0/1277886520 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f66880d64c0 0x7f66881b1be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-04-17T22:29:17.804 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.816 7f668cd16700 1 -- 172.21.15.102:0/1277886520 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66881b2a70 con 0x7f66880d6910 2024-04-17T22:29:17.805 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.816 7f6687fff700 1 --2- 172.21.15.102:0/1277886520 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f66880cd820 0x7f66880d5e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:29:17.805 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.816 7f668cd16700 1 --2- 172.21.15.102:0/1277886520 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f66880d6910 0x7f66881b2210 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7f66880dac90 tx=0x7f6680006c40).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:29:17.805 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.817 7f66857fa700 1 -- 172.21.15.102:0/1277886520 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f66881b2a70 con 0x7f66880d6910 2024-04-17T22:29:17.805 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.817 7f66857fa700 1 -- 172.21.15.102:0/1277886520 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66800119e0 con 0x7f66880d6910 2024-04-17T22:29:17.805 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.817 7f66857fa700 1 -- 172.21.15.102:0/1277886520 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f6680017040 con 0x7f66880d6910 2024-04-17T22:29:17.805 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.817 7f668f71a700 1 -- 172.21.15.102:0/1277886520 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66881b9350 con 0x7f66880d6910 2024-04-17T22:29:17.805 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.817 7f668f71a700 1 -- 172.21.15.102:0/1277886520 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66881b9730 con 0x7f66880d6910 2024-04-17T22:29:17.806 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.817 7f66857fa700 1 -- 172.21.15.102:0/1277886520 <== mon.2 v2:172.21.15.102:3302/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7f668001c470 con 0x7f66880d6910 2024-04-17T22:29:17.806 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.818 7f66857fa700 1 -- 172.21.15.102:0/1277886520 <== mon.2 v2:172.21.15.102:3302/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 3194+0+0 (secure 0 0 0) 0x7f6680018030 con 0x7f66880d6910 2024-04-17T22:29:17.806 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.818 7f668f71a700 1 -- 172.21.15.102:0/1277886520 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66880f0800 con 0x7f66880d6910 2024-04-17T22:29:17.807 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.821 7f66857fa700 1 -- 172.21.15.102:0/1277886520 <== mon.2 v2:172.21.15.102:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+110455 (secure 0 0 0) 0x7f66800117d0 con 0x7f66880d6910 2024-04-17T22:29:17.816 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.829 7f5f21842700 1 Processor -- start 2024-04-17T22:29:17.816 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.829 7f5f21842700 1 -- start start 2024-04-17T22:29:17.817 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.830 7f5f21842700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5f1c0f5750 0x7f5f1c10c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:17.818 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.830 7f5f21842700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f5f1c10cb70 0x7f5f1c1a13e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:17.818 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.830 7f5f21842700 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f1c1a1a10 con 0x7f5f1c10c720 2024-04-17T22:29:17.818 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.830 7f5f21842700 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f1c0f93c0 con 0x7f5f1c10cb70 2024-04-17T22:29:17.818 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.830 7f5f21842700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f1c1a1c30 con 0x7f5f1c0f5750 2024-04-17T22:29:17.818 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.830 7f5f1a5c5700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5f1c0f5750 0x7f5f1c10c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:17.818 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.830 7f5f1a5c5700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5f1c0f5750 0x7f5f1c10c0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:40338/0 (socket says 172.21.15.102:40338) 2024-04-17T22:29:17.818 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.830 7f5f1a5c5700 1 -- 172.21.15.102:0/165431208 learned_addr learned my addr 172.21.15.102:0/165431208 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:29:17.818 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.830 7f5f1adc6700 1 --2- 172.21.15.102:0/165431208 >> v2:172.21.15.102:3300/0 conn(0x7f5f1c10cb70 0x7f5f1c1a13e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:17.818 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.830 7f5f195c3700 1 -- 172.21.15.102:0/165431208 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3446482631 0 0) 0x7f5f10001110 con 0x7f5f1c10c720 2024-04-17T22:29:17.818 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.830 7f5f195c3700 1 -- 172.21.15.102:0/165431208 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5f00003270 con 0x7f5f1c10c720 2024-04-17T22:29:17.818 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.830 7f5f1a5c5700 1 -- 172.21.15.102:0/165431208 >> v2:172.21.15.102:3300/0 conn(0x7f5f1c10cb70 msgr2=0x7f5f1c1a13e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:29:17.818 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.830 7f5f1a5c5700 1 --2- 172.21.15.102:0/165431208 >> v2:172.21.15.102:3300/0 conn(0x7f5f1c10cb70 0x7f5f1c1a13e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:17.819 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.830 7f5f1a5c5700 1 -- 172.21.15.102:0/165431208 >> v1:172.21.15.102:6789/0 conn(0x7f5f1c10c720 legacy=0x7f5f1c1a0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:17.819 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.830 7f5f1a5c5700 1 -- 172.21.15.102:0/165431208 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f1c1aa310 con 0x7f5f1c0f5750 2024-04-17T22:29:17.819 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.831 7f5f1a5c5700 1 --2- 172.21.15.102:0/165431208 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5f1c0f5750 0x7f5f1c10c0f0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7f5f140032f0 tx=0x7f5f14009d60).ready entity=mon.1 client_cookie=cfaad27f711b7cdc server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:29:17.819 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.831 7f5f195c3700 1 -- 172.21.15.102:0/165431208 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f5f1c1aa310 con 0x7f5f1c0f5750 2024-04-17T22:29:17.819 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.831 7f5f195c3700 1 -- 172.21.15.102:0/165431208 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f1400b820 con 0x7f5f1c0f5750 2024-04-17T22:29:17.819 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.831 7f5f195c3700 1 -- 172.21.15.102:0/165431208 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f5f1400ba00 con 0x7f5f1c0f5750 2024-04-17T22:29:17.819 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.831 7f5f21842700 1 -- 172.21.15.102:0/165431208 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5f1c0f5750 msgr2=0x7f5f1c10c0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:17.819 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.831 7f5f21842700 1 --2- 172.21.15.102:0/165431208 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5f1c0f5750 0x7f5f1c10c0f0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7f5f140032f0 tx=0x7f5f14009d60).stop 2024-04-17T22:29:17.820 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.832 7f5f21842700 1 -- 172.21.15.102:0/165431208 shutdown_connections 2024-04-17T22:29:17.820 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.832 7f5f21842700 1 --2- 172.21.15.102:0/165431208 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5f1c0f5750 0x7f5f1c10c0f0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:17.820 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.832 7f5f21842700 1 --2- 172.21.15.102:0/165431208 >> v2:172.21.15.102:3300/0 conn(0x7f5f1c10cb70 0x7f5f1c1a13e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:17.820 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.832 7f5f21842700 1 -- 172.21.15.102:0/165431208 >> 172.21.15.102:0/165431208 conn(0x7f5f1c1138a0 msgr2=0x7f5f1c0f4f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:29:17.820 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.832 7f5f21842700 1 -- 172.21.15.102:0/165431208 shutdown_connections 2024-04-17T22:29:17.820 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.832 7f5f21842700 1 -- 172.21.15.102:0/165431208 wait complete. 2024-04-17T22:29:17.831 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.844 7f9d2fc83700 1 Processor -- start 2024-04-17T22:29:17.831 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.844 7f9d2fc83700 1 -- start start 2024-04-17T22:29:17.833 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.845 7f9d2fc83700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f9d2810e240 0x7f9d2810e690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:17.833 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.845 7f9d2fc83700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f9d2810ecc0 0x7f9d280f1730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:17.833 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.845 7f9d2fc83700 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d280f21b0 con 0x7f9d280f1d60 2024-04-17T22:29:17.833 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.845 7f9d2fc83700 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d280f9f10 con 0x7f9d2810ecc0 2024-04-17T22:29:17.833 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.845 7f9d2fc83700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d280f9030 con 0x7f9d2810e240 2024-04-17T22:29:17.833 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.845 7f9d27fff700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f9d2810ecc0 0x7f9d280f1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:17.833 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.845 7f9d27fff700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f9d2810ecc0 0x7f9d280f1730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:40930/0 (socket says 172.21.15.102:40930) 2024-04-17T22:29:17.833 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.845 7f9d27fff700 1 -- 172.21.15.102:0/3750135120 learned_addr learned my addr 172.21.15.102:0/3750135120 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:29:17.834 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.845 7f9d2ca7e700 1 --2- 172.21.15.102:0/3750135120 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f9d2810e240 0x7f9d2810e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:17.834 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.845 7f9d27fff700 1 -- 172.21.15.102:0/3750135120 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f9d2810e240 msgr2=0x7f9d2810e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:29:17.834 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.845 7f9d27fff700 1 --2- 172.21.15.102:0/3750135120 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f9d2810e240 0x7f9d2810e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:17.834 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.845 7f9d27fff700 1 -- 172.21.15.102:0/3750135120 >> v1:172.21.15.102:6789/0 conn(0x7f9d280f1d60 legacy=0x7f9d281aa4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:29:17.834 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.845 7f9d27fff700 1 -- 172.21.15.102:0/3750135120 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d280f23d0 con 0x7f9d2810ecc0 2024-04-17T22:29:17.834 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.845 7f9d27fff700 1 --2- 172.21.15.102:0/3750135120 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9d2810ecc0 0x7f9d280f1730 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0x7f9d28103580 tx=0x7f9d14009a80).ready entity=mon.0 client_cookie=17ba3aaea7dcdd94 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:29:17.834 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.846 7f9d277fe700 1 -- 172.21.15.102:0/3750135120 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9d280f23d0 con 0x7f9d2810ecc0 2024-04-17T22:29:17.834 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.846 7f9d277fe700 1 -- 172.21.15.102:0/3750135120 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d1400f510 con 0x7f9d2810ecc0 2024-04-17T22:29:17.834 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.846 7f9d277fe700 1 -- 172.21.15.102:0/3750135120 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9d1400f780 con 0x7f9d2810ecc0 2024-04-17T22:29:17.834 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.846 7f9d2fc83700 1 -- 172.21.15.102:0/3750135120 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9d2810ecc0 msgr2=0x7f9d280f1730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:17.834 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.846 7f9d2fc83700 1 --2- 172.21.15.102:0/3750135120 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9d2810ecc0 0x7f9d280f1730 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0x7f9d28103580 tx=0x7f9d14009a80).stop 2024-04-17T22:29:17.834 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.846 7f9d2fc83700 1 -- 172.21.15.102:0/3750135120 shutdown_connections 2024-04-17T22:29:17.834 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.846 7f9d2fc83700 1 --2- 172.21.15.102:0/3750135120 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f9d2810e240 0x7f9d2810e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:17.835 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.846 7f9d2fc83700 1 --2- 172.21.15.102:0/3750135120 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9d2810ecc0 0x7f9d280f1730 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:17.835 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.846 7f9d2fc83700 1 -- 172.21.15.102:0/3750135120 >> 172.21.15.102:0/3750135120 conn(0x7f9d281138a0 msgr2=0x7f9d280f4f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:29:17.835 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.846 7f9d2fc83700 1 -- 172.21.15.102:0/3750135120 shutdown_connections 2024-04-17T22:29:17.835 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.847 7f9d2fc83700 1 -- 172.21.15.102:0/3750135120 wait complete. 2024-04-17T22:29:17.837 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.847 7fa439b46700 1 Processor -- start 2024-04-17T22:29:17.837 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.847 7fa439b46700 1 -- start start 2024-04-17T22:29:17.837 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.848 7fa439b46700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fa4341a10b0 0x7fa4341ac910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:17.837 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.848 7fa439b46700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fa4341acf40 0x7fa4341af3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:17.837 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.848 7fa439b46700 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4341a1500 con 0x7fa4340f5870 2024-04-17T22:29:17.837 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.848 7fa439b46700 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4340fa1d0 con 0x7fa4341acf40 2024-04-17T22:29:17.837 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.848 7fa439b46700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa4340f9500 con 0x7fa4341a10b0 2024-04-17T22:29:17.837 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.848 7fa43285d700 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7fa4340f5870 0x7fa4341a08b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:34722/0 (socket says 172.21.15.102:34722) 2024-04-17T22:29:17.837 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.848 7fa43285d700 1 -- 172.21.15.102:0/3104315249 learned_addr learned my addr 172.21.15.102:0/3104315249 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-17T22:29:17.837 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.848 7fa43205c700 1 --2- 172.21.15.102:0/3104315249 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fa4341a10b0 0x7fa4341ac910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:17.838 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.848 7fa43305e700 1 --2- 172.21.15.102:0/3104315249 >> v2:172.21.15.102:3300/0 conn(0x7fa4341acf40 0x7fa4341af3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:17.838 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.848 7fa43205c700 1 -- 172.21.15.102:0/3104315249 >> v2:172.21.15.102:3300/0 conn(0x7fa4341acf40 msgr2=0x7fa4341af3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:29:17.838 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.848 7fa43205c700 1 --2- 172.21.15.102:0/3104315249 >> v2:172.21.15.102:3300/0 conn(0x7fa4341acf40 0x7fa4341af3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:17.838 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.848 7fa43185b700 1 -- 172.21.15.102:0/3104315249 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3648031769 0 0) 0x7fa424001110 con 0x7fa4340f5870 2024-04-17T22:29:17.838 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.848 7fa43205c700 1 -- 172.21.15.102:0/3104315249 >> v1:172.21.15.102:6789/0 conn(0x7fa4340f5870 legacy=0x7fa4341a08b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:17.838 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.848 7fa43205c700 1 -- 172.21.15.102:0/3104315249 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4341b09e0 con 0x7fa4341a10b0 2024-04-17T22:29:17.838 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.849 7fa43205c700 1 --2- 172.21.15.102:0/3104315249 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fa4341a10b0 0x7fa4341ac910 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7fa428002f70 tx=0x7fa42800b970).ready entity=mon.2 client_cookie=65c2f6382e5e570 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:29:17.838 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.849 7fa43185b700 1 -- 172.21.15.102:0/3104315249 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa4341b09e0 con 0x7fa4341a10b0 2024-04-17T22:29:17.838 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.849 7fa43185b700 1 -- 172.21.15.102:0/3104315249 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa42800d5c0 con 0x7fa4341a10b0 2024-04-17T22:29:17.838 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.849 7fa43185b700 1 -- 172.21.15.102:0/3104315249 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa42800d990 con 0x7fa4341a10b0 2024-04-17T22:29:17.838 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.849 7fa439b46700 1 -- 172.21.15.102:0/3104315249 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fa4341a10b0 msgr2=0x7fa4341ac910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:17.838 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.849 7fa439b46700 1 --2- 172.21.15.102:0/3104315249 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fa4341a10b0 0x7fa4341ac910 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7fa428002f70 tx=0x7fa42800b970).stop 2024-04-17T22:29:17.838 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.850 7fa439b46700 1 -- 172.21.15.102:0/3104315249 shutdown_connections 2024-04-17T22:29:17.838 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.850 7fa439b46700 1 --2- 172.21.15.102:0/3104315249 >> v2:172.21.15.102:3300/0 conn(0x7fa4341acf40 0x7fa4341af3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:17.839 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.850 7fa439b46700 1 --2- 172.21.15.102:0/3104315249 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fa4341a10b0 0x7fa4341ac910 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:17.839 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.850 7fa439b46700 1 -- 172.21.15.102:0/3104315249 >> 172.21.15.102:0/3104315249 conn(0x7fa4341138a0 msgr2=0x7fa4340f4f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:29:17.839 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.850 7fa439b46700 1 -- 172.21.15.102:0/3104315249 shutdown_connections 2024-04-17T22:29:17.839 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.850 7fa439b46700 1 -- 172.21.15.102:0/3104315249 wait complete. 2024-04-17T22:29:17.843 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.856 7f5f21842700 1 Processor -- start 2024-04-17T22:29:17.843 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.856 7f5f21842700 1 -- start start 2024-04-17T22:29:17.843 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.856 7f5f21842700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5f1c0cdd30 0x7f5f1c0d5f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:17.843 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.856 7f5f21842700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5f1c0d6570 0x7f5f1c1b1e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:17.843 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.856 7f5f21842700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5f1c0d69c0 0x7f5f1c1b2480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:17.843 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.856 7f5f21842700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f1c0cf810 con 0x7f5f1c0d69c0 2024-04-17T22:29:17.843 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.856 7f5f21842700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f1c1b2ab0 con 0x7f5f1c0cdd30 2024-04-17T22:29:17.843 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.856 7f5f21842700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5f1c1b2ce0 con 0x7f5f1c0d6570 2024-04-17T22:29:17.843 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.856 7f5f19dc4700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5f1c0d6570 0x7f5f1c1b1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:17.843 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.856 7f5f19dc4700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5f1c0d6570 0x7f5f1c1b1e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:44320/0 (socket says 172.21.15.102:44320) 2024-04-17T22:29:17.843 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.856 7f5f19dc4700 1 -- 172.21.15.102:0/3673792439 learned_addr learned my addr 172.21.15.102:0/3673792439 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:29:17.844 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.856 7f5f1a5c5700 1 --2- 172.21.15.102:0/3673792439 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5f1c0cdd30 0x7f5f1c0d5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:17.844 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.857 7f5f1adc6700 1 --2- 172.21.15.102:0/3673792439 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5f1c0d69c0 0x7f5f1c1b2480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:17.844 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.857 7f5f19dc4700 1 -- 172.21.15.102:0/3673792439 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5f1c0cdd30 msgr2=0x7f5f1c0d5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:17.844 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.857 7f5f19dc4700 1 --2- 172.21.15.102:0/3673792439 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5f1c0cdd30 0x7f5f1c0d5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:17.844 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.857 7f5f19dc4700 1 -- 172.21.15.102:0/3673792439 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5f1c0d69c0 msgr2=0x7f5f1c1b2480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:17.844 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.857 7f5f19dc4700 1 --2- 172.21.15.102:0/3673792439 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5f1c0d69c0 0x7f5f1c1b2480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:17.845 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.857 7f5f19dc4700 1 -- 172.21.15.102:0/3673792439 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f1c1b9650 con 0x7f5f1c0d6570 2024-04-17T22:29:17.845 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.857 7f5f19dc4700 1 --2- 172.21.15.102:0/3673792439 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5f1c0d6570 0x7f5f1c1b1e50 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7f5f10000bc0 tx=0x7f5f1000c760).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:29:17.845 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.859 7f5f0b7fe700 1 -- 172.21.15.102:0/3673792439 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f5f1c1b9650 con 0x7f5f1c0d6570 2024-04-17T22:29:17.845 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.859 7f5f0b7fe700 1 -- 172.21.15.102:0/3673792439 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f1000f470 con 0x7f5f1c0d6570 2024-04-17T22:29:17.845 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.859 7f5f21842700 1 -- 172.21.15.102:0/3673792439 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f1c1b9850 con 0x7f5f1c0d6570 2024-04-17T22:29:17.845 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.859 7f5f0b7fe700 1 -- 172.21.15.102:0/3673792439 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f5f1000f840 con 0x7f5f1c0d6570 2024-04-17T22:29:17.846 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.859 7f5f21842700 1 -- 172.21.15.102:0/3673792439 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f1c1b9a50 con 0x7f5f1c0d6570 2024-04-17T22:29:17.846 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.860 7f5f0b7fe700 1 -- 172.21.15.102:0/3673792439 <== mon.2 v2:172.21.15.102:3302/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7f5f10019020 con 0x7f5f1c0d6570 2024-04-17T22:29:17.846 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.860 7f5f0b7fe700 1 -- 172.21.15.102:0/3673792439 <== mon.2 v2:172.21.15.102:3302/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 3194+0+0 (secure 0 0 0) 0x7f5f100130f0 con 0x7f5f1c0d6570 2024-04-17T22:29:17.848 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.862 7f5f21842700 1 -- 172.21.15.102:0/3673792439 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f1c0f0800 con 0x7f5f1c0d6570 2024-04-17T22:29:17.851 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.865 7f5f0b7fe700 1 -- 172.21.15.102:0/3673792439 <== mon.2 v2:172.21.15.102:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+110455 (secure 0 0 0) 0x7f5f1c0f1440 con 0x7f5f1c0d6570 2024-04-17T22:29:17.875 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.889 7f9d2fc83700 1 Processor -- start 2024-04-17T22:29:17.875 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.889 7f9d2fc83700 1 -- start start 2024-04-17T22:29:17.878 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.889 7f9d2fc83700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f9d280c7200 0x7f9d280c7650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:17.878 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.889 7f9d2fc83700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9d280c7c80 0x7f9d280d60a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:17.878 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.889 7f9d2fc83700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f9d280d66d0 0x7f9d281bb890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:17.878 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.889 7f9d2fc83700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d280e2c10 con 0x7f9d280c7c80 2024-04-17T22:29:17.878 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.889 7f9d2fc83700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d280fa1b0 con 0x7f9d280d66d0 2024-04-17T22:29:17.878 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.889 7f9d2fc83700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9d280cf6f0 con 0x7f9d280c7200 2024-04-17T22:29:17.878 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.890 7f9d2ca7e700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f9d280c7200 0x7f9d280c7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:17.878 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.890 7f9d2ca7e700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f9d280c7200 0x7f9d280c7650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:44328/0 (socket says 172.21.15.102:44328) 2024-04-17T22:29:17.878 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.890 7f9d2ca7e700 1 -- 172.21.15.102:0/3436163482 learned_addr learned my addr 172.21.15.102:0/3436163482 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:29:17.878 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.890 7f9d27fff700 1 --2- 172.21.15.102:0/3436163482 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9d280c7c80 0x7f9d280d60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:17.878 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.890 7f9d2d27f700 1 --2- 172.21.15.102:0/3436163482 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f9d280d66d0 0x7f9d281bb890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:17.879 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.890 7f9d27fff700 1 -- 172.21.15.102:0/3436163482 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f9d280c7200 msgr2=0x7f9d280c7650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:17.879 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.890 7f9d27fff700 1 --2- 172.21.15.102:0/3436163482 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f9d280c7200 0x7f9d280c7650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:17.879 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.890 7f9d27fff700 1 -- 172.21.15.102:0/3436163482 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f9d280d66d0 msgr2=0x7f9d281bb890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:17.879 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.890 7f9d27fff700 1 --2- 172.21.15.102:0/3436163482 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f9d280d66d0 0x7f9d281bb890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:17.879 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.890 7f9d27fff700 1 -- 172.21.15.102:0/3436163482 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d280e2dc0 con 0x7f9d280c7c80 2024-04-17T22:29:17.879 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.890 7f9d2ca7e700 1 --2- 172.21.15.102:0/3436163482 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f9d280c7200 0x7f9d280c7650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:29:17.879 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.890 7f9d27fff700 1 --2- 172.21.15.102:0/3436163482 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9d280c7c80 0x7f9d280d60a0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0x7f9d280c5e00 tx=0x7f9d140068c0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:29:17.879 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.890 7f9d25ffb700 1 -- 172.21.15.102:0/3436163482 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9d280e2dc0 con 0x7f9d280c7c80 2024-04-17T22:29:17.879 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.890 7f9d25ffb700 1 -- 172.21.15.102:0/3436163482 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d1400f510 con 0x7f9d280c7c80 2024-04-17T22:29:17.879 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.890 7f9d2fc83700 1 -- 172.21.15.102:0/3436163482 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d280dd920 con 0x7f9d280c7c80 2024-04-17T22:29:17.879 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.890 7f9d25ffb700 1 -- 172.21.15.102:0/3436163482 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9d1401c440 con 0x7f9d280c7c80 2024-04-17T22:29:17.879 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.891 7f9d2fc83700 1 -- 172.21.15.102:0/3436163482 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d280d6b20 con 0x7f9d280c7c80 2024-04-17T22:29:17.879 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.891 7f9d25ffb700 1 -- 172.21.15.102:0/3436163482 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7f9d14019030 con 0x7f9d280c7c80 2024-04-17T22:29:17.879 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.891 7f9d25ffb700 1 -- 172.21.15.102:0/3436163482 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 3194+0+0 (secure 0 0 0) 0x7f9d14018040 con 0x7f9d280c7c80 2024-04-17T22:29:17.880 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.892 7fa439b46700 1 Processor -- start 2024-04-17T22:29:17.880 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.892 7fa439b46700 1 -- start start 2024-04-17T22:29:17.880 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.893 7fa439b46700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fa43410d640 0x7fa43410da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:17.880 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.893 7fa439b46700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa43410e0c0 0x7fa4341c59a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:17.880 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.893 7fa439b46700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa4341c5fd0 0x7fa4341c8430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:17.880 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.893 7fa439b46700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa43410f080 con 0x7fa43410e0c0 2024-04-17T22:29:17.881 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.893 7fa439b46700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4341ac110 con 0x7fa4341c5fd0 2024-04-17T22:29:17.881 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.893 7fa439b46700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa43410ebe0 con 0x7fa43410d640 2024-04-17T22:29:17.881 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.893 7fa43285d700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fa43410d640 0x7fa43410da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:17.881 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.893 7fa43285d700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fa43410d640 0x7fa43410da90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:44334/0 (socket says 172.21.15.102:44334) 2024-04-17T22:29:17.881 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.893 7fa43285d700 1 -- 172.21.15.102:0/3969271193 learned_addr learned my addr 172.21.15.102:0/3969271193 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:29:17.881 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.893 7f9d2fc83700 1 -- 172.21.15.102:0/3436163482 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d280f0800 con 0x7f9d280c7c80 2024-04-17T22:29:17.881 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.893 7fa43205c700 1 --2- 172.21.15.102:0/3969271193 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa43410e0c0 0x7fa4341c59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:17.882 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.893 7fa43305e700 1 --2- 172.21.15.102:0/3969271193 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa4341c5fd0 0x7fa4341c8430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:17.882 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.893 7fa43205c700 1 -- 172.21.15.102:0/3969271193 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fa43410d640 msgr2=0x7fa43410da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:17.882 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.893 7fa43205c700 1 --2- 172.21.15.102:0/3969271193 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fa43410d640 0x7fa43410da90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:17.882 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.893 7fa43205c700 1 -- 172.21.15.102:0/3969271193 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa4341c5fd0 msgr2=0x7fa4341c8430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:17.882 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.893 7fa43205c700 1 --2- 172.21.15.102:0/3969271193 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa4341c5fd0 0x7fa4341c8430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:17.882 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.893 7fa43205c700 1 -- 172.21.15.102:0/3969271193 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa43410e510 con 0x7fa43410e0c0 2024-04-17T22:29:17.882 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.893 7fa43205c700 1 --2- 172.21.15.102:0/3969271193 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa43410e0c0 0x7fa4341c59a0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 rx=0x7fa4340d7870 tx=0x7fa4280068c0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:29:17.882 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.896 7fa423fff700 1 -- 172.21.15.102:0/3969271193 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa43410e510 con 0x7fa43410e0c0 2024-04-17T22:29:17.883 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.896 7fa423fff700 1 -- 172.21.15.102:0/3969271193 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa428011860 con 0x7fa43410e0c0 2024-04-17T22:29:17.883 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.896 7fa423fff700 1 -- 172.21.15.102:0/3969271193 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa428017040 con 0x7fa43410e0c0 2024-04-17T22:29:17.883 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.896 7f9d25ffb700 1 -- 172.21.15.102:0/3436163482 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+110455 (secure 0 0 0) 0x7f9d14011ae0 con 0x7f9d280c7c80 2024-04-17T22:29:17.884 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.897 7fa439b46700 1 -- 172.21.15.102:0/3969271193 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4340d8c70 con 0x7fa43410e0c0 2024-04-17T22:29:17.884 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.897 7fa439b46700 1 -- 172.21.15.102:0/3969271193 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4340de3b0 con 0x7fa43410e0c0 2024-04-17T22:29:17.884 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.898 7fa439b46700 1 -- 172.21.15.102:0/3969271193 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4340f0800 con 0x7fa43410e0c0 2024-04-17T22:29:17.884 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.898 7fa423fff700 1 -- 172.21.15.102:0/3969271193 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7fa428011c30 con 0x7fa43410e0c0 2024-04-17T22:29:17.885 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.898 7fa423fff700 1 -- 172.21.15.102:0/3969271193 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 3194+0+0 (secure 0 0 0) 0x7fa428018070 con 0x7fa43410e0c0 2024-04-17T22:29:17.890 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.904 7fa423fff700 1 -- 172.21.15.102:0/3969271193 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+110455 (secure 0 0 0) 0x7fa42800d3b0 con 0x7fa43410e0c0 2024-04-17T22:29:17.971 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.984 7f668f71a700 1 -- 172.21.15.102:0/1277886520 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f66881ac300 con 0x7f66880d6910 2024-04-17T22:29:17.971 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.984 7f66857fa700 1 -- 172.21.15.102:0/1277886520 <== mon.2 v2:172.21.15.102:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f668000b630 con 0x7f66880d6910 2024-04-17T22:29:17.971 INFO:teuthology.orchestra.run.smithi102.stdout:38654705671 2024-04-17T22:29:17.972 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.985 7f668f71a700 1 -- 172.21.15.102:0/1277886520 >> v1:172.21.15.102:6816/42360 conn(0x7f66680145a0 legacy=0x7f6668016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:17.972 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.985 7f668f71a700 1 -- 172.21.15.102:0/1277886520 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f66880d6910 msgr2=0x7f66881b2210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:17.972 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.985 7f668f71a700 1 --2- 172.21.15.102:0/1277886520 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f66880d6910 0x7f66881b2210 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7f66880dac90 tx=0x7f6680006c40).stop 2024-04-17T22:29:17.973 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.986 7f668f71a700 1 -- 172.21.15.102:0/1277886520 shutdown_connections 2024-04-17T22:29:17.973 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.986 7f668f71a700 1 --2- 172.21.15.102:0/1277886520 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f66880d6910 0x7f66881b2210 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:17.973 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.986 7f668f71a700 1 --2- 172.21.15.102:0/1277886520 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f66880cd820 0x7f66880d5e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:17.973 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.986 7f668f71a700 1 --2- 172.21.15.102:0/1277886520 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f66880d64c0 0x7f66881b1be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-04-17T22:29:17.973 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.986 7f668f71a700 1 -- 172.21.15.102:0/1277886520 >> 172.21.15.102:0/1277886520 conn(0x7f66880e0fd0 msgr2=0x7f66880dacf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:29:17.973 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.986 7f668f71a700 1 -- 172.21.15.102:0/1277886520 shutdown_connections 2024-04-17T22:29:17.973 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:17.986 7f668f71a700 1 -- 172.21.15.102:0/1277886520 wait complete. 2024-04-17T22:29:17.987 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705671 got 38654705671 for osd.2 2024-04-17T22:29:17.987 DEBUG:teuthology.parallel:result is None 2024-04-17T22:29:18.001 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.015 7fa439b46700 1 -- 172.21.15.102:0/3969271193 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fa4340cff30 con 0x7fa43410e0c0 2024-04-17T22:29:18.002 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.015 7fa423fff700 1 -- 172.21.15.102:0/3969271193 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fa428011650 con 0x7fa43410e0c0 2024-04-17T22:29:18.002 INFO:teuthology.orchestra.run.smithi102.stdout:38654705671 2024-04-17T22:29:18.004 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.017 7fa439b46700 1 -- 172.21.15.102:0/3969271193 >> v1:172.21.15.102:6816/42360 conn(0x7fa4100145a0 legacy=0x7fa410016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:18.004 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.017 7fa439b46700 1 -- 172.21.15.102:0/3969271193 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa43410e0c0 msgr2=0x7fa4341c59a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:18.004 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.017 7fa439b46700 1 --2- 172.21.15.102:0/3969271193 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa43410e0c0 0x7fa4341c59a0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 rx=0x7fa4340d7870 tx=0x7fa4280068c0).stop 2024-04-17T22:29:18.004 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.017 7fa439b46700 1 -- 172.21.15.102:0/3969271193 shutdown_connections 2024-04-17T22:29:18.004 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.017 7fa439b46700 1 --2- 172.21.15.102:0/3969271193 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa4341c5fd0 0x7fa4341c8430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:18.004 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.017 7fa439b46700 1 --2- 172.21.15.102:0/3969271193 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fa43410d640 0x7fa43410da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:18.004 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.017 7fa439b46700 1 --2- 172.21.15.102:0/3969271193 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa43410e0c0 0x7fa4341c59a0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:18.005 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.017 7fa439b46700 1 -- 172.21.15.102:0/3969271193 >> 172.21.15.102:0/3969271193 conn(0x7fa4340c6b50 msgr2=0x7fa434106b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:29:18.005 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.017 7fa439b46700 1 -- 172.21.15.102:0/3969271193 shutdown_connections 2024-04-17T22:29:18.005 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.017 7fa439b46700 1 -- 172.21.15.102:0/3969271193 wait complete. 2024-04-17T22:29:18.005 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.017 7f9d2fc83700 1 -- 172.21.15.102:0/3436163482 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f9d280c80d0 con 0x7f9d280c7c80 2024-04-17T22:29:18.005 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.018 7f9d25ffb700 1 -- 172.21.15.102:0/3436163482 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f9d1400f300 con 0x7f9d280c7c80 2024-04-17T22:29:18.006 INFO:teuthology.orchestra.run.smithi102.stdout:38654705671 2024-04-17T22:29:18.006 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.019 7f9d2fc83700 1 -- 172.21.15.102:0/3436163482 >> v1:172.21.15.102:6816/42360 conn(0x7f9d100145a0 legacy=0x7f9d10016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:18.007 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.019 7f9d2fc83700 1 -- 172.21.15.102:0/3436163482 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9d280c7c80 msgr2=0x7f9d280d60a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:18.007 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.019 7f9d2fc83700 1 --2- 172.21.15.102:0/3436163482 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9d280c7c80 0x7f9d280d60a0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0x7f9d280c5e00 tx=0x7f9d140068c0).stop 2024-04-17T22:29:18.007 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.019 7f9d2fc83700 1 -- 172.21.15.102:0/3436163482 shutdown_connections 2024-04-17T22:29:18.007 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.019 7f9d2fc83700 1 --2- 172.21.15.102:0/3436163482 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f9d280d66d0 0x7f9d281bb890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:18.007 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.019 7f9d2fc83700 1 --2- 172.21.15.102:0/3436163482 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f9d280c7200 0x7f9d280c7650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:18.007 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.019 7f9d2fc83700 1 --2- 172.21.15.102:0/3436163482 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9d280c7c80 0x7f9d280d60a0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:18.007 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.019 7f9d2fc83700 1 -- 172.21.15.102:0/3436163482 >> 172.21.15.102:0/3436163482 conn(0x7f9d280c6a60 msgr2=0x7f9d280c3020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:29:18.007 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.019 7f9d2fc83700 1 -- 172.21.15.102:0/3436163482 shutdown_connections 2024-04-17T22:29:18.007 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.019 7f9d2fc83700 1 -- 172.21.15.102:0/3436163482 wait complete. 2024-04-17T22:29:18.011 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.025 7f5f21842700 1 -- 172.21.15.102:0/3673792439 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f5f1c1ac570 con 0x7f5f1c0d6570 2024-04-17T22:29:18.012 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.025 7f5f0b7fe700 1 -- 172.21.15.102:0/3673792439 <== mon.2 v2:172.21.15.102:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f5f1000f260 con 0x7f5f1c0d6570 2024-04-17T22:29:18.012 INFO:teuthology.orchestra.run.smithi102.stdout:38654705670 2024-04-17T22:29:18.014 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.027 7f5f21842700 1 -- 172.21.15.102:0/3673792439 >> v1:172.21.15.102:6816/42360 conn(0x7f5ef80145a0 legacy=0x7f5ef8016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:18.014 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.027 7f5f21842700 1 -- 172.21.15.102:0/3673792439 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5f1c0d6570 msgr2=0x7f5f1c1b1e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:18.014 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.027 7f5f21842700 1 --2- 172.21.15.102:0/3673792439 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5f1c0d6570 0x7f5f1c1b1e50 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7f5f10000bc0 tx=0x7f5f1000c760).stop 2024-04-17T22:29:18.014 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.027 7f5f21842700 1 -- 172.21.15.102:0/3673792439 shutdown_connections 2024-04-17T22:29:18.014 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.027 7f5f21842700 1 --2- 172.21.15.102:0/3673792439 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5f1c0d69c0 0x7f5f1c1b2480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:18.014 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.027 7f5f21842700 1 --2- 172.21.15.102:0/3673792439 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5f1c0cdd30 0x7f5f1c0d5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:18.014 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.027 7f5f21842700 1 --2- 172.21.15.102:0/3673792439 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5f1c0d6570 0x7f5f1c1b1e50 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:18.015 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.027 7f5f21842700 1 -- 172.21.15.102:0/3673792439 >> 172.21.15.102:0/3673792439 conn(0x7f5f1c0c3810 msgr2=0x7f5f1c0d76c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:29:18.015 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.027 7f5f21842700 1 -- 172.21.15.102:0/3673792439 shutdown_connections 2024-04-17T22:29:18.015 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.027 7f5f21842700 1 -- 172.21.15.102:0/3673792439 wait complete. 2024-04-17T22:29:18.016 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705671 got 38654705671 for osd.0 2024-04-17T22:29:18.016 DEBUG:teuthology.parallel:result is None 2024-04-17T22:29:18.018 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705671 got 38654705671 for osd.3 2024-04-17T22:29:18.018 DEBUG:teuthology.parallel:result is None 2024-04-17T22:29:18.025 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705670 got 38654705670 for osd.1 2024-04-17T22:29:18.025 DEBUG:teuthology.parallel:result is None 2024-04-17T22:29:18.025 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-17T22:29:18.025 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-17T22:29:18.193 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.204 7f3c01b68700 1 Processor -- start 2024-04-17T22:29:18.193 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.204 7f3c01b68700 1 -- start start 2024-04-17T22:29:18.193 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.204 7f3c01b68700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f3bfc10e260 0x7f3bfc10e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:18.193 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.204 7f3c01b68700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f3bfc10ece0 0x7f3bfc0f1750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:18.193 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.204 7f3c01b68700 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3bfc0f21d0 con 0x7f3bfc0f1d80 2024-04-17T22:29:18.193 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.204 7f3c01b68700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bfc0f9f30 con 0x7f3bfc10ece0 2024-04-17T22:29:18.194 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.204 7f3c01b68700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3bfc0f9050 con 0x7f3bfc10e260 2024-04-17T22:29:18.194 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.204 7f3bf3fff700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f3bfc10ece0 0x7f3bfc0f1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:18.194 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.204 7f3bfa85d700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f3bfc10e260 0x7f3bfc10e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:18.194 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.204 7f3bf3fff700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f3bfc10ece0 0x7f3bfc0f1750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:40388/0 (socket says 172.21.15.102:40388) 2024-04-17T22:29:18.194 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.204 7f3bf3fff700 1 -- 172.21.15.102:0/3722602948 learned_addr learned my addr 172.21.15.102:0/3722602948 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:29:18.194 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.204 7f3bfb05e700 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f3bfc0f1d80 0x7f3bfc1aa4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:34730/0 (socket says 172.21.15.102:34730) 2024-04-17T22:29:18.194 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.204 7f3bfa85d700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f3bfc10e260 0x7f3bfc10e6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:44346/0 (socket says 172.21.15.102:44346) 2024-04-17T22:29:18.194 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.205 7f3bfa05c700 1 -- 172.21.15.102:0/3722602948 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1715768512 0 0) 0x7f3bf4001490 con 0x7f3bfc0f1d80 2024-04-17T22:29:18.194 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.205 7f3bfa05c700 1 -- 172.21.15.102:0/3722602948 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3bdc003270 con 0x7f3bfc0f1d80 2024-04-17T22:29:18.194 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.205 7f3bf3fff700 1 -- 172.21.15.102:0/3722602948 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f3bfc10e260 msgr2=0x7f3bfc10e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:29:18.194 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.205 7f3bf3fff700 1 --2- 172.21.15.102:0/3722602948 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f3bfc10e260 0x7f3bfc10e6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:18.194 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.205 7f3bf3fff700 1 -- 172.21.15.102:0/3722602948 >> v1:172.21.15.102:6789/0 conn(0x7f3bfc0f1d80 legacy=0x7f3bfc1aa4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:18.195 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.205 7f3bf3fff700 1 -- 172.21.15.102:0/3722602948 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bfc0f23f0 con 0x7f3bfc10ece0 2024-04-17T22:29:18.195 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.205 7f3bfa85d700 1 --2- 172.21.15.102:0/3722602948 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f3bfc10e260 0x7f3bfc10e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:29:18.195 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.205 7f3bf3fff700 1 --2- 172.21.15.102:0/3722602948 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f3bfc10ece0 0x7f3bfc0f1750 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7f3bfc1035a0 tx=0x7f3be4009ab0).ready entity=mon.1 client_cookie=8cce66d6524bd97b server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:29:18.195 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.205 7f3bfa05c700 1 -- 172.21.15.102:0/3722602948 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f3bfc0f23f0 con 0x7f3bfc10ece0 2024-04-17T22:29:18.195 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.205 7f3bfa05c700 1 -- 172.21.15.102:0/3722602948 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3be400f510 con 0x7f3bfc10ece0 2024-04-17T22:29:18.195 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.205 7f3bfa05c700 1 -- 172.21.15.102:0/3722602948 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f3be400f6f0 con 0x7f3bfc10ece0 2024-04-17T22:29:18.195 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.205 7f3c01b68700 1 -- 172.21.15.102:0/3722602948 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f3bfc10ece0 msgr2=0x7f3bfc0f1750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:18.195 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.205 7f3c01b68700 1 --2- 172.21.15.102:0/3722602948 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f3bfc10ece0 0x7f3bfc0f1750 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7f3bfc1035a0 tx=0x7f3be4009ab0).stop 2024-04-17T22:29:18.195 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.205 7f3c01b68700 1 -- 172.21.15.102:0/3722602948 shutdown_connections 2024-04-17T22:29:18.195 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.206 7f3c01b68700 1 --2- 172.21.15.102:0/3722602948 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f3bfc10e260 0x7f3bfc10e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:18.195 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.206 7f3c01b68700 1 --2- 172.21.15.102:0/3722602948 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f3bfc10ece0 0x7f3bfc0f1750 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:18.195 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.206 7f3c01b68700 1 -- 172.21.15.102:0/3722602948 >> 172.21.15.102:0/3722602948 conn(0x7f3bfc1138c0 msgr2=0x7f3bfc0f4f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:29:18.196 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.206 7f3c01b68700 1 -- 172.21.15.102:0/3722602948 shutdown_connections 2024-04-17T22:29:18.196 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.206 7f3c01b68700 1 -- 172.21.15.102:0/3722602948 wait complete. 2024-04-17T22:29:18.215 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.228 7f3c01b68700 1 Processor -- start 2024-04-17T22:29:18.215 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.228 7f3c01b68700 1 -- start start 2024-04-17T22:29:18.217 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.229 7f3c01b68700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3bfc0d51d0 0x7f3bfc0d5620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:18.217 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.229 7f3c01b68700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f3bfc0d5c50 0x7f3bfc1b42a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:18.217 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.229 7f3c01b68700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f3bfc1b48d0 0x7f3bfc1b4d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:18.217 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.229 7f3c01b68700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bfc0fa1d0 con 0x7f3bfc0d51d0 2024-04-17T22:29:18.217 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.229 7f3c01b68700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bfc0cbd30 con 0x7f3bfc1b48d0 2024-04-17T22:29:18.217 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.229 7f3c01b68700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3bfc1b5350 con 0x7f3bfc0d5c50 2024-04-17T22:29:18.217 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.229 7f3bfa85d700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3bfc0d51d0 0x7f3bfc0d5620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:18.217 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.229 7f3bf3fff700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f3bfc0d5c50 0x7f3bfc1b42a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:18.218 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.229 7f3bfa85d700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3bfc0d51d0 0x7f3bfc0d5620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:40974/0 (socket says 172.21.15.102:40974) 2024-04-17T22:29:18.218 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.229 7f3bf3fff700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f3bfc0d5c50 0x7f3bfc1b42a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:44348/0 (socket says 172.21.15.102:44348) 2024-04-17T22:29:18.218 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.229 7f3bfa85d700 1 -- 172.21.15.102:0/798149455 learned_addr learned my addr 172.21.15.102:0/798149455 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:29:18.218 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.229 7f3bfb05e700 1 --2- 172.21.15.102:0/798149455 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f3bfc1b48d0 0x7f3bfc1b4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:18.218 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.229 7f3bf3fff700 1 -- 172.21.15.102:0/798149455 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f3bfc1b48d0 msgr2=0x7f3bfc1b4d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:18.218 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.229 7f3bf3fff700 1 --2- 172.21.15.102:0/798149455 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f3bfc1b48d0 0x7f3bfc1b4d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:18.218 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.229 7f3bf3fff700 1 -- 172.21.15.102:0/798149455 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3bfc0d51d0 msgr2=0x7f3bfc0d5620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:18.218 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.229 7f3bf3fff700 1 --2- 172.21.15.102:0/798149455 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3bfc0d51d0 0x7f3bfc0d5620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:18.218 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.229 7f3bf3fff700 1 -- 172.21.15.102:0/798149455 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bfc0c4720 con 0x7f3bfc0d5c50 2024-04-17T22:29:18.218 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.229 7f3bfa85d700 1 --2- 172.21.15.102:0/798149455 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3bfc0d51d0 0x7f3bfc0d5620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:29:18.218 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.229 7f3bf3fff700 1 --2- 172.21.15.102:0/798149455 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f3bfc0d5c50 0x7f3bfc1b42a0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7f3bfc0c4b30 tx=0x7f3be40068c0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:29:18.218 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.229 7f3bf8859700 1 -- 172.21.15.102:0/798149455 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f3be401c650 con 0x7f3bfc0d5c50 2024-04-17T22:29:18.218 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.230 7f3bf8859700 1 -- 172.21.15.102:0/798149455 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3be401c850 con 0x7f3bfc0d5c50 2024-04-17T22:29:18.219 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.230 7f3c01b68700 1 -- 172.21.15.102:0/798149455 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bfc0cc540 con 0x7f3bfc0d5c50 2024-04-17T22:29:18.219 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.230 7f3bf8859700 1 -- 172.21.15.102:0/798149455 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f3be401cc20 con 0x7f3bfc0d5c50 2024-04-17T22:29:18.219 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.230 7f3c01b68700 1 -- 172.21.15.102:0/798149455 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bfc0dabc0 con 0x7f3bfc0d5c50 2024-04-17T22:29:18.219 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.230 7f3bf8859700 1 -- 172.21.15.102:0/798149455 <== mon.2 v2:172.21.15.102:3302/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7f3be4019030 con 0x7f3bfc0d5c50 2024-04-17T22:29:18.219 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.230 7f3bf8859700 1 -- 172.21.15.102:0/798149455 <== mon.2 v2:172.21.15.102:3302/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 3194+0+0 (secure 0 0 0) 0x7f3be4018040 con 0x7f3bfc0d5c50 2024-04-17T22:29:18.219 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.230 7f3c01b68700 1 -- 172.21.15.102:0/798149455 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3bfc0f0820 con 0x7f3bfc0d5c50 2024-04-17T22:29:18.220 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.234 7f3bf8859700 1 -- 172.21.15.102:0/798149455 <== mon.2 v2:172.21.15.102:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+110455 (secure 0 0 0) 0x7f3be401c440 con 0x7f3bfc0d5c50 2024-04-17T22:29:18.322 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.336 7f3c01b68700 1 -- 172.21.15.102:0/798149455 --> v1:172.21.15.102:6816/42360 -- command(tid 0: {"prefix": "pg dump", "target": ["mgr", ""], "format": "json"}) v1 -- 0x7f3bfc08a300 con 0x7f3be00145a0 2024-04-17T22:29:18.330 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.344 7f3bf8859700 1 -- 172.21.15.102:0/798149455 <== mgr.4104 v1:172.21.15.102:6816/42360 1 ==== command_reply(tid 0: 0 dumped all) v1 ==== 18+0+306120 (unknown 2965378022 0 1331125451) 0x7f3bec00bac0 con 0x7f3be00145a0 2024-04-17T22:29:18.334 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-17T22:29:18.336 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.346 7f3c01b68700 1 -- 172.21.15.102:0/798149455 >> v1:172.21.15.102:6816/42360 conn(0x7f3be00145a0 legacy=0x7f3be0016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:18.336 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.346 7f3c01b68700 1 -- 172.21.15.102:0/798149455 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f3bfc0d5c50 msgr2=0x7f3bfc1b42a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:18.336 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.346 7f3c01b68700 1 --2- 172.21.15.102:0/798149455 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f3bfc0d5c50 0x7f3bfc1b42a0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7f3bfc0c4b30 tx=0x7f3be40068c0).stop 2024-04-17T22:29:18.336 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.346 7f3c01b68700 1 -- 172.21.15.102:0/798149455 shutdown_connections 2024-04-17T22:29:18.337 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.346 7f3c01b68700 1 --2- 172.21.15.102:0/798149455 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f3bfc1b48d0 0x7f3bfc1b4d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:18.337 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.346 7f3c01b68700 1 --2- 172.21.15.102:0/798149455 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3bfc0d51d0 0x7f3bfc0d5620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:18.337 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.346 7f3c01b68700 1 --2- 172.21.15.102:0/798149455 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f3bfc0d5c50 0x7f3bfc1b42a0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:18.337 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.346 7f3c01b68700 1 -- 172.21.15.102:0/798149455 >> 172.21.15.102:0/798149455 conn(0x7f3bfc0c6af0 msgr2=0x7f3bfc0d62b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:29:18.337 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.346 7f3c01b68700 1 -- 172.21.15.102:0/798149455 shutdown_connections 2024-04-17T22:29:18.337 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.346 7f3c01b68700 1 -- 172.21.15.102:0/798149455 wait complete. 2024-04-17T22:29:18.337 INFO:teuthology.orchestra.run.smithi102.stderr:dumped all 2024-04-17T22:29:18.343 INFO:teuthology.orchestra.run.smithi102.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2024-04-17 22:29:17.188866","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2286,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":272,"acting":272,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":272,"num_osds":4,"num_per_pool_osds":4,"kb":377487360,"kb_used":4194944,"kb_used_data":608,"kb_used_omap":0,"kb_used_meta":4194304,"kb_avail":373292416,"statfs":{"total":386547056640,"available":382251433984,"internally_reserved":4294967296,"allocated":622592,"data_stored":161692,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4294967296},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001302"},"pg_stats":[{"pgid":"3.1a","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.820762","last_change":"2024-04-17 22:29:07.814049","last_active":"2024-04-17 22:29:07.820762","last_peered":"2024-04-17 22:29:07.820762","last_clean":"2024-04-17 22:29:07.820762","last_became_active":"2024-04-17 22:29:07.811049","last_became_peered":"2024-04-17 22:29:07.811049","last_unstale":"2024-04-17 22:29:07.820762","last_undegraded":"2024-04-17 22:29:07.820762","last_fullsized":"2024-04-17 22:29:07.820762","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.807568","last_change":"2024-04-17 22:29:07.477053","last_active":"2024-04-17 22:29:07.807568","last_peered":"2024-04-17 22:29:07.807568","last_clean":"2024-04-17 22:29:07.807568","last_became_active":"2024-04-17 22:29:07.476568","last_became_peered":"2024-04-17 22:29:07.476568","last_unstale":"2024-04-17 22:29:07.807568","last_undegraded":"2024-04-17 22:29:07.807568","last_fullsized":"2024-04-17 22:29:07.807568","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.828628","last_change":"2024-04-17 22:29:07.818787","last_active":"2024-04-17 22:29:07.828628","last_peered":"2024-04-17 22:29:07.828628","last_clean":"2024-04-17 22:29:07.828628","last_became_active":"2024-04-17 22:29:07.818347","last_became_peered":"2024-04-17 22:29:07.818347","last_unstale":"2024-04-17 22:29:07.828628","last_undegraded":"2024-04-17 22:29:07.828628","last_fullsized":"2024-04-17 22:29:07.828628","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.805356","last_change":"2024-04-17 22:29:07.477933","last_active":"2024-04-17 22:29:07.805356","last_peered":"2024-04-17 22:29:07.805356","last_clean":"2024-04-17 22:29:07.805356","last_became_active":"2024-04-17 22:29:07.472128","last_became_peered":"2024-04-17 22:29:07.472128","last_unstale":"2024-04-17 22:29:07.805356","last_undegraded":"2024-04-17 22:29:07.805356","last_fullsized":"2024-04-17 22:29:07.805356","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.834368","last_change":"2024-04-17 22:29:07.833107","last_active":"2024-04-17 22:29:07.834368","last_peered":"2024-04-17 22:29:07.834368","last_clean":"2024-04-17 22:29:07.834368","last_became_active":"2024-04-17 22:29:07.832822","last_became_peered":"2024-04-17 22:29:07.832822","last_unstale":"2024-04-17 22:29:07.834368","last_undegraded":"2024-04-17 22:29:07.834368","last_fullsized":"2024-04-17 22:29:07.834368","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.802997","last_change":"2024-04-17 22:29:07.471612","last_active":"2024-04-17 22:29:07.802997","last_peered":"2024-04-17 22:29:07.802997","last_clean":"2024-04-17 22:29:07.802997","last_became_active":"2024-04-17 22:29:07.471322","last_became_peered":"2024-04-17 22:29:07.471322","last_unstale":"2024-04-17 22:29:07.802997","last_undegraded":"2024-04-17 22:29:07.802997","last_fullsized":"2024-04-17 22:29:07.802997","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.812548","last_change":"2024-04-17 22:29:07.810081","last_active":"2024-04-17 22:29:07.812548","last_peered":"2024-04-17 22:29:07.812548","last_clean":"2024-04-17 22:29:07.812548","last_became_active":"2024-04-17 22:29:07.809858","last_became_peered":"2024-04-17 22:29:07.809858","last_unstale":"2024-04-17 22:29:07.812548","last_undegraded":"2024-04-17 22:29:07.812548","last_fullsized":"2024-04-17 22:29:07.812548","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.797655","last_change":"2024-04-17 22:29:07.474346","last_active":"2024-04-17 22:29:07.797655","last_peered":"2024-04-17 22:29:07.797655","last_clean":"2024-04-17 22:29:07.797655","last_became_active":"2024-04-17 22:29:07.473783","last_became_peered":"2024-04-17 22:29:07.473783","last_unstale":"2024-04-17 22:29:07.797655","last_undegraded":"2024-04-17 22:29:07.797655","last_fullsized":"2024-04-17 22:29:07.797655","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.825049","last_change":"2024-04-17 22:29:07.811047","last_active":"2024-04-17 22:29:07.825049","last_peered":"2024-04-17 22:29:07.825049","last_clean":"2024-04-17 22:29:07.825049","last_became_active":"2024-04-17 22:29:07.810747","last_became_peered":"2024-04-17 22:29:07.810747","last_unstale":"2024-04-17 22:29:07.825049","last_undegraded":"2024-04-17 22:29:07.825049","last_fullsized":"2024-04-17 22:29:07.825049","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.798589","last_change":"2024-04-17 22:29:07.460228","last_active":"2024-04-17 22:29:07.798589","last_peered":"2024-04-17 22:29:07.798589","last_clean":"2024-04-17 22:29:07.798589","last_became_active":"2024-04-17 22:29:07.459910","last_became_peered":"2024-04-17 22:29:07.459910","last_unstale":"2024-04-17 22:29:07.798589","last_undegraded":"2024-04-17 22:29:07.798589","last_fullsized":"2024-04-17 22:29:07.798589","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.830592","last_change":"2024-04-17 22:29:07.827143","last_active":"2024-04-17 22:29:07.830592","last_peered":"2024-04-17 22:29:07.830592","last_clean":"2024-04-17 22:29:07.830592","last_became_active":"2024-04-17 22:29:07.826842","last_became_peered":"2024-04-17 22:29:07.826842","last_unstale":"2024-04-17 22:29:07.830592","last_undegraded":"2024-04-17 22:29:07.830592","last_fullsized":"2024-04-17 22:29:07.830592","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.807458","last_change":"2024-04-17 22:29:07.477248","last_active":"2024-04-17 22:29:07.807458","last_peered":"2024-04-17 22:29:07.807458","last_clean":"2024-04-17 22:29:07.807458","last_became_active":"2024-04-17 22:29:07.476944","last_became_peered":"2024-04-17 22:29:07.476944","last_unstale":"2024-04-17 22:29:07.807458","last_undegraded":"2024-04-17 22:29:07.807458","last_fullsized":"2024-04-17 22:29:07.807458","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.826061","last_change":"2024-04-17 22:29:07.822961","last_active":"2024-04-17 22:29:07.826061","last_peered":"2024-04-17 22:29:07.826061","last_clean":"2024-04-17 22:29:07.826061","last_became_active":"2024-04-17 22:29:07.822592","last_became_peered":"2024-04-17 22:29:07.822592","last_unstale":"2024-04-17 22:29:07.826061","last_undegraded":"2024-04-17 22:29:07.826061","last_fullsized":"2024-04-17 22:29:07.826061","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.15","version":"15'3","reported_seq":"13","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.826924","last_change":"2024-04-17 22:29:07.485166","last_active":"2024-04-17 22:29:07.826924","last_peered":"2024-04-17 22:29:07.826924","last_clean":"2024-04-17 22:29:07.826924","last_became_active":"2024-04-17 22:29:07.484870","last_became_peered":"2024-04-17 22:29:07.484870","last_unstale":"2024-04-17 22:29:07.826924","last_undegraded":"2024-04-17 22:29:07.826924","last_fullsized":"2024-04-17 22:29:07.826924","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.825665","last_change":"2024-04-17 22:29:07.814509","last_active":"2024-04-17 22:29:07.825665","last_peered":"2024-04-17 22:29:07.825665","last_clean":"2024-04-17 22:29:07.825665","last_became_active":"2024-04-17 22:29:07.814119","last_became_peered":"2024-04-17 22:29:07.814119","last_unstale":"2024-04-17 22:29:07.825665","last_undegraded":"2024-04-17 22:29:07.825665","last_fullsized":"2024-04-17 22:29:07.825665","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.14","version":"15'2","reported_seq":"12","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.823699","last_change":"2024-04-17 22:29:07.473309","last_active":"2024-04-17 22:29:07.823699","last_peered":"2024-04-17 22:29:07.823699","last_clean":"2024-04-17 22:29:07.823699","last_became_active":"2024-04-17 22:29:07.472976","last_became_peered":"2024-04-17 22:29:07.472976","last_unstale":"2024-04-17 22:29:07.823699","last_undegraded":"2024-04-17 22:29:07.823699","last_fullsized":"2024-04-17 22:29:07.823699","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":634,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.832027","last_change":"2024-04-17 22:29:07.827915","last_active":"2024-04-17 22:29:07.832027","last_peered":"2024-04-17 22:29:07.832027","last_clean":"2024-04-17 22:29:07.832027","last_became_active":"2024-04-17 22:29:07.827670","last_became_peered":"2024-04-17 22:29:07.827670","last_unstale":"2024-04-17 22:29:07.832027","last_undegraded":"2024-04-17 22:29:07.832027","last_fullsized":"2024-04-17 22:29:07.832027","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"15'1","reported_seq":"11","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.827199","last_change":"2024-04-17 22:29:07.471458","last_active":"2024-04-17 22:29:07.827199","last_peered":"2024-04-17 22:29:07.827199","last_clean":"2024-04-17 22:29:07.827199","last_became_active":"2024-04-17 22:29:07.471085","last_became_peered":"2024-04-17 22:29:07.471085","last_unstale":"2024-04-17 22:29:07.827199","last_undegraded":"2024-04-17 22:29:07.827199","last_fullsized":"2024-04-17 22:29:07.827199","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.833535","last_change":"2024-04-17 22:29:07.829288","last_active":"2024-04-17 22:29:07.833535","last_peered":"2024-04-17 22:29:07.833535","last_clean":"2024-04-17 22:29:07.833535","last_became_active":"2024-04-17 22:29:07.828994","last_became_peered":"2024-04-17 22:29:07.828994","last_unstale":"2024-04-17 22:29:07.833535","last_undegraded":"2024-04-17 22:29:07.833535","last_fullsized":"2024-04-17 22:29:07.833535","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.805144","last_change":"2024-04-17 22:29:07.480194","last_active":"2024-04-17 22:29:07.805144","last_peered":"2024-04-17 22:29:07.805144","last_clean":"2024-04-17 22:29:07.805144","last_became_active":"2024-04-17 22:29:07.479728","last_became_peered":"2024-04-17 22:29:07.479728","last_unstale":"2024-04-17 22:29:07.805144","last_undegraded":"2024-04-17 22:29:07.805144","last_fullsized":"2024-04-17 22:29:07.805144","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.820915","last_change":"2024-04-17 22:29:07.815790","last_active":"2024-04-17 22:29:07.820915","last_peered":"2024-04-17 22:29:07.820915","last_clean":"2024-04-17 22:29:07.820915","last_became_active":"2024-04-17 22:29:07.815632","last_became_peered":"2024-04-17 22:29:07.815632","last_unstale":"2024-04-17 22:29:07.820915","last_undegraded":"2024-04-17 22:29:07.820915","last_fullsized":"2024-04-17 22:29:07.820915","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.804035","last_change":"2024-04-17 22:29:07.475830","last_active":"2024-04-17 22:29:07.804035","last_peered":"2024-04-17 22:29:07.804035","last_clean":"2024-04-17 22:29:07.804035","last_became_active":"2024-04-17 22:29:07.474411","last_became_peered":"2024-04-17 22:29:07.474411","last_unstale":"2024-04-17 22:29:07.804035","last_undegraded":"2024-04-17 22:29:07.804035","last_fullsized":"2024-04-17 22:29:07.804035","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.828447","last_change":"2024-04-17 22:29:07.826101","last_active":"2024-04-17 22:29:07.828447","last_peered":"2024-04-17 22:29:07.828447","last_clean":"2024-04-17 22:29:07.828447","last_became_active":"2024-04-17 22:29:07.822917","last_became_peered":"2024-04-17 22:29:07.822917","last_unstale":"2024-04-17 22:29:07.828447","last_undegraded":"2024-04-17 22:29:07.828447","last_fullsized":"2024-04-17 22:29:07.828447","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.10","version":"15'4","reported_seq":"14","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.817471","last_change":"2024-04-17 22:29:07.457017","last_active":"2024-04-17 22:29:07.817471","last_peered":"2024-04-17 22:29:07.817471","last_clean":"2024-04-17 22:29:07.817471","last_became_active":"2024-04-17 22:29:07.456705","last_became_peered":"2024-04-17 22:29:07.456705","last_unstale":"2024-04-17 22:29:07.817471","last_undegraded":"2024-04-17 22:29:07.817471","last_fullsized":"2024-04-17 22:29:07.817471","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.825122","last_change":"2024-04-17 22:29:07.820464","last_active":"2024-04-17 22:29:07.825122","last_peered":"2024-04-17 22:29:07.825122","last_clean":"2024-04-17 22:29:07.825122","last_became_active":"2024-04-17 22:29:07.820151","last_became_peered":"2024-04-17 22:29:07.820151","last_unstale":"2024-04-17 22:29:07.825122","last_undegraded":"2024-04-17 22:29:07.825122","last_fullsized":"2024-04-17 22:29:07.825122","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.f","version":"15'1","reported_seq":"11","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.828338","last_change":"2024-04-17 22:29:07.478464","last_active":"2024-04-17 22:29:07.828338","last_peered":"2024-04-17 22:29:07.828338","last_clean":"2024-04-17 22:29:07.828338","last_became_active":"2024-04-17 22:29:07.478144","last_became_peered":"2024-04-17 22:29:07.478144","last_unstale":"2024-04-17 22:29:07.828338","last_undegraded":"2024-04-17 22:29:07.828338","last_fullsized":"2024-04-17 22:29:07.828338","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.828463","last_change":"2024-04-17 22:29:07.815171","last_active":"2024-04-17 22:29:07.828463","last_peered":"2024-04-17 22:29:07.828463","last_clean":"2024-04-17 22:29:07.828463","last_became_active":"2024-04-17 22:29:07.814830","last_became_peered":"2024-04-17 22:29:07.814830","last_unstale":"2024-04-17 22:29:07.828463","last_undegraded":"2024-04-17 22:29:07.828463","last_fullsized":"2024-04-17 22:29:07.828463","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"15'2","reported_seq":"12","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.811237","last_change":"2024-04-17 22:29:07.481865","last_active":"2024-04-17 22:29:07.811237","last_peered":"2024-04-17 22:29:07.811237","last_clean":"2024-04-17 22:29:07.811237","last_became_active":"2024-04-17 22:29:07.481506","last_became_peered":"2024-04-17 22:29:07.481506","last_unstale":"2024-04-17 22:29:07.811237","last_undegraded":"2024-04-17 22:29:07.811237","last_fullsized":"2024-04-17 22:29:07.811237","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.828178","last_change":"2024-04-17 22:29:07.818238","last_active":"2024-04-17 22:29:07.828178","last_peered":"2024-04-17 22:29:07.828178","last_clean":"2024-04-17 22:29:07.828178","last_became_active":"2024-04-17 22:29:07.818060","last_became_peered":"2024-04-17 22:29:07.818060","last_unstale":"2024-04-17 22:29:07.828178","last_undegraded":"2024-04-17 22:29:07.828178","last_fullsized":"2024-04-17 22:29:07.828178","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.796955","last_change":"2024-04-17 22:29:07.484999","last_active":"2024-04-17 22:29:07.796955","last_peered":"2024-04-17 22:29:07.796955","last_clean":"2024-04-17 22:29:07.796955","last_became_active":"2024-04-17 22:29:07.484668","last_became_peered":"2024-04-17 22:29:07.484668","last_unstale":"2024-04-17 22:29:07.796955","last_undegraded":"2024-04-17 22:29:07.796955","last_fullsized":"2024-04-17 22:29:07.796955","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.828594","last_change":"2024-04-17 22:29:07.826426","last_active":"2024-04-17 22:29:07.828594","last_peered":"2024-04-17 22:29:07.828594","last_clean":"2024-04-17 22:29:07.828594","last_became_active":"2024-04-17 22:29:07.826109","last_became_peered":"2024-04-17 22:29:07.826109","last_unstale":"2024-04-17 22:29:07.828594","last_undegraded":"2024-04-17 22:29:07.828594","last_fullsized":"2024-04-17 22:29:07.828594","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.797207","last_change":"2024-04-17 22:29:07.462218","last_active":"2024-04-17 22:29:07.797207","last_peered":"2024-04-17 22:29:07.797207","last_clean":"2024-04-17 22:29:07.797207","last_became_active":"2024-04-17 22:29:07.461842","last_became_peered":"2024-04-17 22:29:07.461842","last_unstale":"2024-04-17 22:29:07.797207","last_undegraded":"2024-04-17 22:29:07.797207","last_fullsized":"2024-04-17 22:29:07.797207","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.820296","last_change":"2024-04-17 22:29:07.817144","last_active":"2024-04-17 22:29:07.820296","last_peered":"2024-04-17 22:29:07.820296","last_clean":"2024-04-17 22:29:07.820296","last_became_active":"2024-04-17 22:29:07.816859","last_became_peered":"2024-04-17 22:29:07.816859","last_unstale":"2024-04-17 22:29:07.820296","last_undegraded":"2024-04-17 22:29:07.820296","last_fullsized":"2024-04-17 22:29:07.820296","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"15'1","reported_seq":"11","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.821604","last_change":"2024-04-17 22:29:07.479816","last_active":"2024-04-17 22:29:07.821604","last_peered":"2024-04-17 22:29:07.821604","last_clean":"2024-04-17 22:29:07.821604","last_became_active":"2024-04-17 22:29:07.479251","last_became_peered":"2024-04-17 22:29:07.479251","last_unstale":"2024-04-17 22:29:07.821604","last_undegraded":"2024-04-17 22:29:07.821604","last_fullsized":"2024-04-17 22:29:07.821604","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.833516","last_change":"2024-04-17 22:29:07.829215","last_active":"2024-04-17 22:29:07.833516","last_peered":"2024-04-17 22:29:07.833516","last_clean":"2024-04-17 22:29:07.833516","last_became_active":"2024-04-17 22:29:07.828803","last_became_peered":"2024-04-17 22:29:07.828803","last_unstale":"2024-04-17 22:29:07.833516","last_undegraded":"2024-04-17 22:29:07.833516","last_fullsized":"2024-04-17 22:29:07.833516","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.796428","last_change":"2024-04-17 22:29:07.473128","last_active":"2024-04-17 22:29:07.796428","last_peered":"2024-04-17 22:29:07.796428","last_clean":"2024-04-17 22:29:07.796428","last_became_active":"2024-04-17 22:29:07.472806","last_became_peered":"2024-04-17 22:29:07.472806","last_unstale":"2024-04-17 22:29:07.796428","last_undegraded":"2024-04-17 22:29:07.796428","last_fullsized":"2024-04-17 22:29:07.796428","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.820555","last_change":"2024-04-17 22:29:07.817630","last_active":"2024-04-17 22:29:07.820555","last_peered":"2024-04-17 22:29:07.820555","last_clean":"2024-04-17 22:29:07.820555","last_became_active":"2024-04-17 22:29:07.817333","last_became_peered":"2024-04-17 22:29:07.817333","last_unstale":"2024-04-17 22:29:07.820555","last_undegraded":"2024-04-17 22:29:07.820555","last_fullsized":"2024-04-17 22:29:07.820555","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.792279","last_change":"2024-04-17 22:29:07.465053","last_active":"2024-04-17 22:29:07.792279","last_peered":"2024-04-17 22:29:07.792279","last_clean":"2024-04-17 22:29:07.792279","last_became_active":"2024-04-17 22:29:07.464766","last_became_peered":"2024-04-17 22:29:07.464766","last_unstale":"2024-04-17 22:29:07.792279","last_undegraded":"2024-04-17 22:29:07.792279","last_fullsized":"2024-04-17 22:29:07.792279","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.827066","last_change":"2024-04-17 22:29:07.823514","last_active":"2024-04-17 22:29:07.827066","last_peered":"2024-04-17 22:29:07.827066","last_clean":"2024-04-17 22:29:07.827066","last_became_active":"2024-04-17 22:29:07.823007","last_became_peered":"2024-04-17 22:29:07.823007","last_unstale":"2024-04-17 22:29:07.827066","last_undegraded":"2024-04-17 22:29:07.827066","last_fullsized":"2024-04-17 22:29:07.827066","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.801156","last_change":"2024-04-17 22:29:07.461568","last_active":"2024-04-17 22:29:07.801156","last_peered":"2024-04-17 22:29:07.801156","last_clean":"2024-04-17 22:29:07.801156","last_became_active":"2024-04-17 22:29:07.461249","last_became_peered":"2024-04-17 22:29:07.461249","last_unstale":"2024-04-17 22:29:07.801156","last_undegraded":"2024-04-17 22:29:07.801156","last_fullsized":"2024-04-17 22:29:07.801156","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.828426","last_change":"2024-04-17 22:29:07.826342","last_active":"2024-04-17 22:29:07.828426","last_peered":"2024-04-17 22:29:07.828426","last_clean":"2024-04-17 22:29:07.828426","last_became_active":"2024-04-17 22:29:07.826029","last_became_peered":"2024-04-17 22:29:07.826029","last_unstale":"2024-04-17 22:29:07.828426","last_undegraded":"2024-04-17 22:29:07.828426","last_fullsized":"2024-04-17 22:29:07.828426","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":"13","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.795463","last_change":"2024-04-17 22:28:59.253977","last_active":"2024-04-17 22:29:07.795463","last_peered":"2024-04-17 22:29:07.795463","last_clean":"2024-04-17 22:29:07.795463","last_became_active":"2024-04-17 22:28:59.253641","last_became_peered":"2024-04-17 22:28:59.253641","last_unstale":"2024-04-17 22:29:07.795463","last_undegraded":"2024-04-17 22:29:07.795463","last_fullsized":"2024-04-17 22:29:07.795463","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:28:58.239187","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:28:58.239187","last_clean_scrub_stamp":"2024-04-17 22:28:58.239187","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.796951","last_change":"2024-04-17 22:29:07.466907","last_active":"2024-04-17 22:29:07.796951","last_peered":"2024-04-17 22:29:07.796951","last_clean":"2024-04-17 22:29:07.796951","last_became_active":"2024-04-17 22:29:07.466583","last_became_peered":"2024-04-17 22:29:07.466583","last_unstale":"2024-04-17 22:29:07.796951","last_undegraded":"2024-04-17 22:29:07.796951","last_fullsized":"2024-04-17 22:29:07.796951","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.829316","last_change":"2024-04-17 22:29:07.824875","last_active":"2024-04-17 22:29:07.829316","last_peered":"2024-04-17 22:29:07.829316","last_clean":"2024-04-17 22:29:07.829316","last_became_active":"2024-04-17 22:29:07.824377","last_became_peered":"2024-04-17 22:29:07.824377","last_unstale":"2024-04-17 22:29:07.829316","last_undegraded":"2024-04-17 22:29:07.829316","last_fullsized":"2024-04-17 22:29:07.829316","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":"13","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.809532","last_change":"2024-04-17 22:28:59.255733","last_active":"2024-04-17 22:29:07.809532","last_peered":"2024-04-17 22:29:07.809532","last_clean":"2024-04-17 22:29:07.809532","last_became_active":"2024-04-17 22:28:59.255411","last_became_peered":"2024-04-17 22:28:59.255411","last_unstale":"2024-04-17 22:29:07.809532","last_undegraded":"2024-04-17 22:29:07.809532","last_fullsized":"2024-04-17 22:29:07.809532","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:28:58.239187","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:28:58.239187","last_clean_scrub_stamp":"2024-04-17 22:28:58.239187","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"15'2","reported_seq":"12","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.832132","last_change":"2024-04-17 22:29:07.478196","last_active":"2024-04-17 22:29:07.832132","last_peered":"2024-04-17 22:29:07.832132","last_clean":"2024-04-17 22:29:07.832132","last_became_active":"2024-04-17 22:29:07.477887","last_became_peered":"2024-04-17 22:29:07.477887","last_unstale":"2024-04-17 22:29:07.832132","last_undegraded":"2024-04-17 22:29:07.832132","last_fullsized":"2024-04-17 22:29:07.832132","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":"13","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.796430","last_change":"2024-04-17 22:28:59.252728","last_active":"2024-04-17 22:29:07.796430","last_peered":"2024-04-17 22:29:07.796430","last_clean":"2024-04-17 22:29:07.796430","last_became_active":"2024-04-17 22:28:59.252345","last_became_peered":"2024-04-17 22:28:59.252345","last_unstale":"2024-04-17 22:29:07.796430","last_undegraded":"2024-04-17 22:29:07.796430","last_fullsized":"2024-04-17 22:29:07.796430","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:28:58.239187","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:28:58.239187","last_clean_scrub_stamp":"2024-04-17 22:28:58.239187","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.797727","last_change":"2024-04-17 22:29:07.474520","last_active":"2024-04-17 22:29:07.797727","last_peered":"2024-04-17 22:29:07.797727","last_clean":"2024-04-17 22:29:07.797727","last_became_active":"2024-04-17 22:29:07.474029","last_became_peered":"2024-04-17 22:29:07.474029","last_unstale":"2024-04-17 22:29:07.797727","last_undegraded":"2024-04-17 22:29:07.797727","last_fullsized":"2024-04-17 22:29:07.797727","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.829302","last_change":"2024-04-17 22:29:07.825091","last_active":"2024-04-17 22:29:07.829302","last_peered":"2024-04-17 22:29:07.829302","last_clean":"2024-04-17 22:29:07.829302","last_became_active":"2024-04-17 22:29:07.824832","last_became_peered":"2024-04-17 22:29:07.824832","last_unstale":"2024-04-17 22:29:07.829302","last_undegraded":"2024-04-17 22:29:07.829302","last_fullsized":"2024-04-17 22:29:07.829302","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":"13","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.807580","last_change":"2024-04-17 22:28:59.253392","last_active":"2024-04-17 22:29:07.807580","last_peered":"2024-04-17 22:29:07.807580","last_clean":"2024-04-17 22:29:07.807580","last_became_active":"2024-04-17 22:28:59.253049","last_became_peered":"2024-04-17 22:28:59.253049","last_unstale":"2024-04-17 22:29:07.807580","last_undegraded":"2024-04-17 22:29:07.807580","last_fullsized":"2024-04-17 22:29:07.807580","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:28:58.239187","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:28:58.239187","last_clean_scrub_stamp":"2024-04-17 22:28:58.239187","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.796512","last_change":"2024-04-17 22:29:07.471913","last_active":"2024-04-17 22:29:07.796512","last_peered":"2024-04-17 22:29:07.796512","last_clean":"2024-04-17 22:29:07.796512","last_became_active":"2024-04-17 22:29:07.471686","last_became_peered":"2024-04-17 22:29:07.471686","last_unstale":"2024-04-17 22:29:07.796512","last_undegraded":"2024-04-17 22:29:07.796512","last_fullsized":"2024-04-17 22:29:07.796512","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.820450","last_change":"2024-04-17 22:29:07.812999","last_active":"2024-04-17 22:29:07.820450","last_peered":"2024-04-17 22:29:07.820450","last_clean":"2024-04-17 22:29:07.820450","last_became_active":"2024-04-17 22:29:07.812701","last_became_peered":"2024-04-17 22:29:07.812701","last_unstale":"2024-04-17 22:29:07.820450","last_undegraded":"2024-04-17 22:29:07.820450","last_fullsized":"2024-04-17 22:29:07.820450","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":"13","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.791848","last_change":"2024-04-17 22:28:59.254907","last_active":"2024-04-17 22:29:07.791848","last_peered":"2024-04-17 22:29:07.791848","last_clean":"2024-04-17 22:29:07.791848","last_became_active":"2024-04-17 22:28:59.254633","last_became_peered":"2024-04-17 22:28:59.254633","last_unstale":"2024-04-17 22:29:07.791848","last_undegraded":"2024-04-17 22:29:07.791848","last_fullsized":"2024-04-17 22:29:07.791848","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:28:58.239187","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:28:58.239187","last_clean_scrub_stamp":"2024-04-17 22:28:58.239187","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.807141","last_change":"2024-04-17 22:29:07.480020","last_active":"2024-04-17 22:29:07.807141","last_peered":"2024-04-17 22:29:07.807141","last_clean":"2024-04-17 22:29:07.807141","last_became_active":"2024-04-17 22:29:07.479492","last_became_peered":"2024-04-17 22:29:07.479492","last_unstale":"2024-04-17 22:29:07.807141","last_undegraded":"2024-04-17 22:29:07.807141","last_fullsized":"2024-04-17 22:29:07.807141","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.826030","last_change":"2024-04-17 22:29:07.821883","last_active":"2024-04-17 22:29:07.826030","last_peered":"2024-04-17 22:29:07.826030","last_clean":"2024-04-17 22:29:07.826030","last_became_active":"2024-04-17 22:29:07.821569","last_became_peered":"2024-04-17 22:29:07.821569","last_unstale":"2024-04-17 22:29:07.826030","last_undegraded":"2024-04-17 22:29:07.826030","last_fullsized":"2024-04-17 22:29:07.826030","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":"13","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.806466","last_change":"2024-04-17 22:28:59.254217","last_active":"2024-04-17 22:29:07.806466","last_peered":"2024-04-17 22:29:07.806466","last_clean":"2024-04-17 22:29:07.806466","last_became_active":"2024-04-17 22:28:59.253877","last_became_peered":"2024-04-17 22:28:59.253877","last_unstale":"2024-04-17 22:29:07.806466","last_undegraded":"2024-04-17 22:29:07.806466","last_fullsized":"2024-04-17 22:29:07.806466","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:28:58.239187","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:28:58.239187","last_clean_scrub_stamp":"2024-04-17 22:28:58.239187","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.805257","last_change":"2024-04-17 22:29:07.480378","last_active":"2024-04-17 22:29:07.805257","last_peered":"2024-04-17 22:29:07.805257","last_clean":"2024-04-17 22:29:07.805257","last_became_active":"2024-04-17 22:29:07.479866","last_became_peered":"2024-04-17 22:29:07.479866","last_unstale":"2024-04-17 22:29:07.805257","last_undegraded":"2024-04-17 22:29:07.805257","last_fullsized":"2024-04-17 22:29:07.805257","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.829882","last_change":"2024-04-17 22:29:07.821315","last_active":"2024-04-17 22:29:07.829882","last_peered":"2024-04-17 22:29:07.829882","last_clean":"2024-04-17 22:29:07.829882","last_became_active":"2024-04-17 22:29:07.820937","last_became_peered":"2024-04-17 22:29:07.820937","last_unstale":"2024-04-17 22:29:07.829882","last_undegraded":"2024-04-17 22:29:07.829882","last_fullsized":"2024-04-17 22:29:07.829882","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":"13","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.813317","last_change":"2024-04-17 22:28:59.254830","last_active":"2024-04-17 22:29:07.813317","last_peered":"2024-04-17 22:29:07.813317","last_clean":"2024-04-17 22:29:07.813317","last_became_active":"2024-04-17 22:28:59.254505","last_became_peered":"2024-04-17 22:28:59.254505","last_unstale":"2024-04-17 22:29:07.813317","last_undegraded":"2024-04-17 22:29:07.813317","last_fullsized":"2024-04-17 22:29:07.813317","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:28:58.239187","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:28:58.239187","last_clean_scrub_stamp":"2024-04-17 22:28:58.239187","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.809624","last_change":"2024-04-17 22:29:07.475017","last_active":"2024-04-17 22:29:07.809624","last_peered":"2024-04-17 22:29:07.809624","last_clean":"2024-04-17 22:29:07.809624","last_became_active":"2024-04-17 22:29:07.473559","last_became_peered":"2024-04-17 22:29:07.473559","last_unstale":"2024-04-17 22:29:07.809624","last_undegraded":"2024-04-17 22:29:07.809624","last_fullsized":"2024-04-17 22:29:07.809624","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.811282","last_change":"2024-04-17 22:29:07.808988","last_active":"2024-04-17 22:29:07.811282","last_peered":"2024-04-17 22:29:07.811282","last_clean":"2024-04-17 22:29:07.811282","last_became_active":"2024-04-17 22:29:07.808663","last_became_peered":"2024-04-17 22:29:07.808663","last_unstale":"2024-04-17 22:29:07.811282","last_undegraded":"2024-04-17 22:29:07.811282","last_fullsized":"2024-04-17 22:29:07.811282","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":"13","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.796932","last_change":"2024-04-17 22:28:59.254055","last_active":"2024-04-17 22:29:07.796932","last_peered":"2024-04-17 22:29:07.796932","last_clean":"2024-04-17 22:29:07.796932","last_became_active":"2024-04-17 22:28:59.253750","last_became_peered":"2024-04-17 22:28:59.253750","last_unstale":"2024-04-17 22:29:07.796932","last_undegraded":"2024-04-17 22:29:07.796932","last_fullsized":"2024-04-17 22:29:07.796932","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:28:58.239187","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:28:58.239187","last_clean_scrub_stamp":"2024-04-17 22:28:58.239187","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.796660","last_change":"2024-04-17 22:29:07.466238","last_active":"2024-04-17 22:29:07.796660","last_peered":"2024-04-17 22:29:07.796660","last_clean":"2024-04-17 22:29:07.796660","last_became_active":"2024-04-17 22:29:07.465806","last_became_peered":"2024-04-17 22:29:07.465806","last_unstale":"2024-04-17 22:29:07.796660","last_undegraded":"2024-04-17 22:29:07.796660","last_fullsized":"2024-04-17 22:29:07.796660","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.820702","last_change":"2024-04-17 22:29:07.809155","last_active":"2024-04-17 22:29:07.820702","last_peered":"2024-04-17 22:29:07.820702","last_clean":"2024-04-17 22:29:07.820702","last_became_active":"2024-04-17 22:29:07.808850","last_became_peered":"2024-04-17 22:29:07.808850","last_unstale":"2024-04-17 22:29:07.820702","last_undegraded":"2024-04-17 22:29:07.820702","last_fullsized":"2024-04-17 22:29:07.820702","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.833569","last_change":"2024-04-17 22:29:07.832794","last_active":"2024-04-17 22:29:07.833569","last_peered":"2024-04-17 22:29:07.833569","last_clean":"2024-04-17 22:29:07.833569","last_became_active":"2024-04-17 22:29:07.832467","last_became_peered":"2024-04-17 22:29:07.832467","last_unstale":"2024-04-17 22:29:07.833569","last_undegraded":"2024-04-17 22:29:07.833569","last_fullsized":"2024-04-17 22:29:07.833569","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.811305","last_change":"2024-04-17 22:29:07.477743","last_active":"2024-04-17 22:29:07.811305","last_peered":"2024-04-17 22:29:07.811305","last_clean":"2024-04-17 22:29:07.811305","last_became_active":"2024-04-17 22:29:07.477503","last_became_peered":"2024-04-17 22:29:07.477503","last_unstale":"2024-04-17 22:29:07.811305","last_undegraded":"2024-04-17 22:29:07.811305","last_fullsized":"2024-04-17 22:29:07.811305","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.830256","last_change":"2024-04-17 22:29:07.828401","last_active":"2024-04-17 22:29:07.830256","last_peered":"2024-04-17 22:29:07.830256","last_clean":"2024-04-17 22:29:07.830256","last_became_active":"2024-04-17 22:29:07.828093","last_became_peered":"2024-04-17 22:29:07.828093","last_unstale":"2024-04-17 22:29:07.830256","last_undegraded":"2024-04-17 22:29:07.830256","last_fullsized":"2024-04-17 22:29:07.830256","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"15'2","reported_seq":"12","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.824506","last_change":"2024-04-17 22:29:07.478361","last_active":"2024-04-17 22:29:07.824506","last_peered":"2024-04-17 22:29:07.824506","last_clean":"2024-04-17 22:29:07.824506","last_became_active":"2024-04-17 22:29:07.477793","last_became_peered":"2024-04-17 22:29:07.477793","last_unstale":"2024-04-17 22:29:07.824506","last_undegraded":"2024-04-17 22:29:07.824506","last_fullsized":"2024-04-17 22:29:07.824506","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.828482","last_change":"2024-04-17 22:29:07.815548","last_active":"2024-04-17 22:29:07.828482","last_peered":"2024-04-17 22:29:07.828482","last_clean":"2024-04-17 22:29:07.828482","last_became_active":"2024-04-17 22:29:07.815263","last_became_peered":"2024-04-17 22:29:07.815263","last_unstale":"2024-04-17 22:29:07.828482","last_undegraded":"2024-04-17 22:29:07.828482","last_fullsized":"2024-04-17 22:29:07.828482","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"15'1","reported_seq":"11","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.831868","last_change":"2024-04-17 22:29:07.482507","last_active":"2024-04-17 22:29:07.831868","last_peered":"2024-04-17 22:29:07.831868","last_clean":"2024-04-17 22:29:07.831868","last_became_active":"2024-04-17 22:29:07.481981","last_became_peered":"2024-04-17 22:29:07.481981","last_unstale":"2024-04-17 22:29:07.831868","last_undegraded":"2024-04-17 22:29:07.831868","last_fullsized":"2024-04-17 22:29:07.831868","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.833109","last_change":"2024-04-17 22:29:07.828459","last_active":"2024-04-17 22:29:07.833109","last_peered":"2024-04-17 22:29:07.833109","last_clean":"2024-04-17 22:29:07.833109","last_became_active":"2024-04-17 22:29:07.828157","last_became_peered":"2024-04-17 22:29:07.828157","last_unstale":"2024-04-17 22:29:07.833109","last_undegraded":"2024-04-17 22:29:07.833109","last_fullsized":"2024-04-17 22:29:07.833109","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"15'1","reported_seq":"11","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.828398","last_change":"2024-04-17 22:29:07.471751","last_active":"2024-04-17 22:29:07.828398","last_peered":"2024-04-17 22:29:07.828398","last_clean":"2024-04-17 22:29:07.828398","last_became_active":"2024-04-17 22:29:07.471385","last_became_peered":"2024-04-17 22:29:07.471385","last_unstale":"2024-04-17 22:29:07.828398","last_undegraded":"2024-04-17 22:29:07.828398","last_fullsized":"2024-04-17 22:29:07.828398","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.821701","last_change":"2024-04-17 22:29:07.812125","last_active":"2024-04-17 22:29:07.821701","last_peered":"2024-04-17 22:29:07.821701","last_clean":"2024-04-17 22:29:07.821701","last_became_active":"2024-04-17 22:29:07.811846","last_became_peered":"2024-04-17 22:29:07.811846","last_unstale":"2024-04-17 22:29:07.821701","last_undegraded":"2024-04-17 22:29:07.821701","last_fullsized":"2024-04-17 22:29:07.821701","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.20","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.798111","last_change":"2024-04-17 22:29:07.477127","last_active":"2024-04-17 22:29:07.798111","last_peered":"2024-04-17 22:29:07.798111","last_clean":"2024-04-17 22:29:07.798111","last_became_active":"2024-04-17 22:29:07.476773","last_became_peered":"2024-04-17 22:29:07.476773","last_unstale":"2024-04-17 22:29:07.798111","last_undegraded":"2024-04-17 22:29:07.798111","last_fullsized":"2024-04-17 22:29:07.798111","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.824583","last_change":"2024-04-17 22:29:07.821132","last_active":"2024-04-17 22:29:07.824583","last_peered":"2024-04-17 22:29:07.824583","last_clean":"2024-04-17 22:29:07.824583","last_became_active":"2024-04-17 22:29:07.820838","last_became_peered":"2024-04-17 22:29:07.820838","last_unstale":"2024-04-17 22:29:07.824583","last_undegraded":"2024-04-17 22:29:07.824583","last_fullsized":"2024-04-17 22:29:07.824583","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.21","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.796071","last_change":"2024-04-17 22:29:07.460110","last_active":"2024-04-17 22:29:07.796071","last_peered":"2024-04-17 22:29:07.796071","last_clean":"2024-04-17 22:29:07.796071","last_became_active":"2024-04-17 22:29:07.459611","last_became_peered":"2024-04-17 22:29:07.459611","last_unstale":"2024-04-17 22:29:07.796071","last_undegraded":"2024-04-17 22:29:07.796071","last_fullsized":"2024-04-17 22:29:07.796071","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.828609","last_change":"2024-04-17 22:29:07.818496","last_active":"2024-04-17 22:29:07.828609","last_peered":"2024-04-17 22:29:07.828609","last_clean":"2024-04-17 22:29:07.828609","last_became_active":"2024-04-17 22:29:07.818184","last_became_peered":"2024-04-17 22:29:07.818184","last_unstale":"2024-04-17 22:29:07.828609","last_undegraded":"2024-04-17 22:29:07.828609","last_fullsized":"2024-04-17 22:29:07.828609","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.22","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.805451","last_change":"2024-04-17 22:29:07.481695","last_active":"2024-04-17 22:29:07.805451","last_peered":"2024-04-17 22:29:07.805451","last_clean":"2024-04-17 22:29:07.805451","last_became_active":"2024-04-17 22:29:07.479982","last_became_peered":"2024-04-17 22:29:07.479982","last_unstale":"2024-04-17 22:29:07.805451","last_undegraded":"2024-04-17 22:29:07.805451","last_fullsized":"2024-04-17 22:29:07.805451","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.835093","last_change":"2024-04-17 22:29:07.833317","last_active":"2024-04-17 22:29:07.835093","last_peered":"2024-04-17 22:29:07.835093","last_clean":"2024-04-17 22:29:07.835093","last_became_active":"2024-04-17 22:29:07.827050","last_became_peered":"2024-04-17 22:29:07.827050","last_unstale":"2024-04-17 22:29:07.835093","last_undegraded":"2024-04-17 22:29:07.835093","last_fullsized":"2024-04-17 22:29:07.835093","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.23","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.807837","last_change":"2024-04-17 22:29:07.477449","last_active":"2024-04-17 22:29:07.807837","last_peered":"2024-04-17 22:29:07.807837","last_clean":"2024-04-17 22:29:07.807837","last_became_active":"2024-04-17 22:29:07.477155","last_became_peered":"2024-04-17 22:29:07.477155","last_unstale":"2024-04-17 22:29:07.807837","last_undegraded":"2024-04-17 22:29:07.807837","last_fullsized":"2024-04-17 22:29:07.807837","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.810313","last_change":"2024-04-17 22:29:07.808974","last_active":"2024-04-17 22:29:07.810313","last_peered":"2024-04-17 22:29:07.810313","last_clean":"2024-04-17 22:29:07.810313","last_became_active":"2024-04-17 22:29:07.808747","last_became_peered":"2024-04-17 22:29:07.808747","last_unstale":"2024-04-17 22:29:07.810313","last_undegraded":"2024-04-17 22:29:07.810313","last_fullsized":"2024-04-17 22:29:07.810313","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.24","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.796551","last_change":"2024-04-17 22:29:07.466052","last_active":"2024-04-17 22:29:07.796551","last_peered":"2024-04-17 22:29:07.796551","last_clean":"2024-04-17 22:29:07.796551","last_became_active":"2024-04-17 22:29:07.465576","last_became_peered":"2024-04-17 22:29:07.465576","last_unstale":"2024-04-17 22:29:07.796551","last_undegraded":"2024-04-17 22:29:07.796551","last_fullsized":"2024-04-17 22:29:07.796551","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.833471","last_change":"2024-04-17 22:29:07.829066","last_active":"2024-04-17 22:29:07.833471","last_peered":"2024-04-17 22:29:07.833471","last_clean":"2024-04-17 22:29:07.833471","last_became_active":"2024-04-17 22:29:07.828735","last_became_peered":"2024-04-17 22:29:07.828735","last_unstale":"2024-04-17 22:29:07.833471","last_undegraded":"2024-04-17 22:29:07.833471","last_fullsized":"2024-04-17 22:29:07.833471","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.25","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.795566","last_change":"2024-04-17 22:29:07.462011","last_active":"2024-04-17 22:29:07.795566","last_peered":"2024-04-17 22:29:07.795566","last_clean":"2024-04-17 22:29:07.795566","last_became_active":"2024-04-17 22:29:07.461693","last_became_peered":"2024-04-17 22:29:07.461693","last_unstale":"2024-04-17 22:29:07.795566","last_undegraded":"2024-04-17 22:29:07.795566","last_fullsized":"2024-04-17 22:29:07.795566","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.828447","last_change":"2024-04-17 22:29:07.826655","last_active":"2024-04-17 22:29:07.828447","last_peered":"2024-04-17 22:29:07.828447","last_clean":"2024-04-17 22:29:07.828447","last_became_active":"2024-04-17 22:29:07.823665","last_became_peered":"2024-04-17 22:29:07.823665","last_unstale":"2024-04-17 22:29:07.828447","last_undegraded":"2024-04-17 22:29:07.828447","last_fullsized":"2024-04-17 22:29:07.828447","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.26","version":"16'1","reported_seq":"11","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:08.802774","last_change":"2024-04-17 22:29:07.480854","last_active":"2024-04-17 22:29:08.802774","last_peered":"2024-04-17 22:29:08.802774","last_clean":"2024-04-17 22:29:08.802774","last_became_active":"2024-04-17 22:29:07.479975","last_became_peered":"2024-04-17 22:29:07.479975","last_unstale":"2024-04-17 22:29:08.802774","last_undegraded":"2024-04-17 22:29:08.802774","last_fullsized":"2024-04-17 22:29:08.802774","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.825070","last_change":"2024-04-17 22:29:07.813744","last_active":"2024-04-17 22:29:07.825070","last_peered":"2024-04-17 22:29:07.825070","last_clean":"2024-04-17 22:29:07.825070","last_became_active":"2024-04-17 22:29:07.810646","last_became_peered":"2024-04-17 22:29:07.810646","last_unstale":"2024-04-17 22:29:07.825070","last_undegraded":"2024-04-17 22:29:07.825070","last_fullsized":"2024-04-17 22:29:07.825070","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.27","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.815316","last_change":"2024-04-17 22:29:07.476428","last_active":"2024-04-17 22:29:07.815316","last_peered":"2024-04-17 22:29:07.815316","last_clean":"2024-04-17 22:29:07.815316","last_became_active":"2024-04-17 22:29:07.476128","last_became_peered":"2024-04-17 22:29:07.476128","last_unstale":"2024-04-17 22:29:07.815316","last_undegraded":"2024-04-17 22:29:07.815316","last_fullsized":"2024-04-17 22:29:07.815316","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.826418","last_change":"2024-04-17 22:29:07.823340","last_active":"2024-04-17 22:29:07.826418","last_peered":"2024-04-17 22:29:07.826418","last_clean":"2024-04-17 22:29:07.826418","last_became_active":"2024-04-17 22:29:07.822764","last_became_peered":"2024-04-17 22:29:07.822764","last_unstale":"2024-04-17 22:29:07.826418","last_undegraded":"2024-04-17 22:29:07.826418","last_fullsized":"2024-04-17 22:29:07.826418","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.28","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.798818","last_change":"2024-04-17 22:29:07.477137","last_active":"2024-04-17 22:29:07.798818","last_peered":"2024-04-17 22:29:07.798818","last_clean":"2024-04-17 22:29:07.798818","last_became_active":"2024-04-17 22:29:07.476796","last_became_peered":"2024-04-17 22:29:07.476796","last_unstale":"2024-04-17 22:29:07.798818","last_undegraded":"2024-04-17 22:29:07.798818","last_fullsized":"2024-04-17 22:29:07.798818","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.826239","last_change":"2024-04-17 22:29:07.823119","last_active":"2024-04-17 22:29:07.826239","last_peered":"2024-04-17 22:29:07.826239","last_clean":"2024-04-17 22:29:07.826239","last_became_active":"2024-04-17 22:29:07.822640","last_became_peered":"2024-04-17 22:29:07.822640","last_unstale":"2024-04-17 22:29:07.826239","last_undegraded":"2024-04-17 22:29:07.826239","last_fullsized":"2024-04-17 22:29:07.826239","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.29","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.792382","last_change":"2024-04-17 22:29:07.473467","last_active":"2024-04-17 22:29:07.792382","last_peered":"2024-04-17 22:29:07.792382","last_clean":"2024-04-17 22:29:07.792382","last_became_active":"2024-04-17 22:29:07.473028","last_became_peered":"2024-04-17 22:29:07.473028","last_unstale":"2024-04-17 22:29:07.792382","last_undegraded":"2024-04-17 22:29:07.792382","last_fullsized":"2024-04-17 22:29:07.792382","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.827332","last_change":"2024-04-17 22:29:07.817097","last_active":"2024-04-17 22:29:07.827332","last_peered":"2024-04-17 22:29:07.827332","last_clean":"2024-04-17 22:29:07.827332","last_became_active":"2024-04-17 22:29:07.816813","last_became_peered":"2024-04-17 22:29:07.816813","last_unstale":"2024-04-17 22:29:07.827332","last_undegraded":"2024-04-17 22:29:07.827332","last_fullsized":"2024-04-17 22:29:07.827332","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2a","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.796331","last_change":"2024-04-17 22:29:07.464330","last_active":"2024-04-17 22:29:07.796331","last_peered":"2024-04-17 22:29:07.796331","last_clean":"2024-04-17 22:29:07.796331","last_became_active":"2024-04-17 22:29:07.464002","last_became_peered":"2024-04-17 22:29:07.464002","last_unstale":"2024-04-17 22:29:07.796331","last_undegraded":"2024-04-17 22:29:07.796331","last_fullsized":"2024-04-17 22:29:07.796331","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.830305","last_change":"2024-04-17 22:29:07.828690","last_active":"2024-04-17 22:29:07.830305","last_peered":"2024-04-17 22:29:07.830305","last_clean":"2024-04-17 22:29:07.830305","last_became_active":"2024-04-17 22:29:07.828407","last_became_peered":"2024-04-17 22:29:07.828407","last_unstale":"2024-04-17 22:29:07.830305","last_undegraded":"2024-04-17 22:29:07.830305","last_fullsized":"2024-04-17 22:29:07.830305","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2b","version":"15'1","reported_seq":"11","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.810554","last_change":"2024-04-17 22:29:07.471300","last_active":"2024-04-17 22:29:07.810554","last_peered":"2024-04-17 22:29:07.810554","last_clean":"2024-04-17 22:29:07.810554","last_became_active":"2024-04-17 22:29:07.470886","last_became_peered":"2024-04-17 22:29:07.470886","last_unstale":"2024-04-17 22:29:07.810554","last_undegraded":"2024-04-17 22:29:07.810554","last_fullsized":"2024-04-17 22:29:07.810554","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.828614","last_change":"2024-04-17 22:29:07.826555","last_active":"2024-04-17 22:29:07.828614","last_peered":"2024-04-17 22:29:07.828614","last_clean":"2024-04-17 22:29:07.828614","last_became_active":"2024-04-17 22:29:07.826262","last_became_peered":"2024-04-17 22:29:07.826262","last_unstale":"2024-04-17 22:29:07.828614","last_undegraded":"2024-04-17 22:29:07.828614","last_fullsized":"2024-04-17 22:29:07.828614","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2c","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.797039","last_change":"2024-04-17 22:29:07.462176","last_active":"2024-04-17 22:29:07.797039","last_peered":"2024-04-17 22:29:07.797039","last_clean":"2024-04-17 22:29:07.797039","last_became_active":"2024-04-17 22:29:07.461594","last_became_peered":"2024-04-17 22:29:07.461594","last_unstale":"2024-04-17 22:29:07.797039","last_undegraded":"2024-04-17 22:29:07.797039","last_fullsized":"2024-04-17 22:29:07.797039","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.828130","last_change":"2024-04-17 22:29:07.816896","last_active":"2024-04-17 22:29:07.828130","last_peered":"2024-04-17 22:29:07.828130","last_clean":"2024-04-17 22:29:07.828130","last_became_active":"2024-04-17 22:29:07.816538","last_became_peered":"2024-04-17 22:29:07.816538","last_unstale":"2024-04-17 22:29:07.828130","last_undegraded":"2024-04-17 22:29:07.828130","last_fullsized":"2024-04-17 22:29:07.828130","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2d","version":"15'1","reported_seq":"11","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.827538","last_change":"2024-04-17 22:29:07.479214","last_active":"2024-04-17 22:29:07.827538","last_peered":"2024-04-17 22:29:07.827538","last_clean":"2024-04-17 22:29:07.827538","last_became_active":"2024-04-17 22:29:07.478790","last_became_peered":"2024-04-17 22:29:07.478790","last_unstale":"2024-04-17 22:29:07.827538","last_undegraded":"2024-04-17 22:29:07.827538","last_fullsized":"2024-04-17 22:29:07.827538","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.833543","last_change":"2024-04-17 22:29:07.832349","last_active":"2024-04-17 22:29:07.833543","last_peered":"2024-04-17 22:29:07.833543","last_clean":"2024-04-17 22:29:07.833543","last_became_active":"2024-04-17 22:29:07.831999","last_became_peered":"2024-04-17 22:29:07.831999","last_unstale":"2024-04-17 22:29:07.833543","last_undegraded":"2024-04-17 22:29:07.833543","last_fullsized":"2024-04-17 22:29:07.833543","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2e","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.797059","last_change":"2024-04-17 22:29:07.483244","last_active":"2024-04-17 22:29:07.797059","last_peered":"2024-04-17 22:29:07.797059","last_clean":"2024-04-17 22:29:07.797059","last_became_active":"2024-04-17 22:29:07.482893","last_became_peered":"2024-04-17 22:29:07.482893","last_unstale":"2024-04-17 22:29:07.797059","last_undegraded":"2024-04-17 22:29:07.797059","last_fullsized":"2024-04-17 22:29:07.797059","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.825028","last_change":"2024-04-17 22:29:07.806512","last_active":"2024-04-17 22:29:07.825028","last_peered":"2024-04-17 22:29:07.825028","last_clean":"2024-04-17 22:29:07.825028","last_became_active":"2024-04-17 22:29:07.806182","last_became_peered":"2024-04-17 22:29:07.806182","last_unstale":"2024-04-17 22:29:07.825028","last_undegraded":"2024-04-17 22:29:07.825028","last_fullsized":"2024-04-17 22:29:07.825028","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2f","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.815418","last_change":"2024-04-17 22:29:07.478655","last_active":"2024-04-17 22:29:07.815418","last_peered":"2024-04-17 22:29:07.815418","last_clean":"2024-04-17 22:29:07.815418","last_became_active":"2024-04-17 22:29:07.478346","last_became_peered":"2024-04-17 22:29:07.478346","last_unstale":"2024-04-17 22:29:07.815418","last_undegraded":"2024-04-17 22:29:07.815418","last_fullsized":"2024-04-17 22:29:07.815418","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.826848","last_change":"2024-04-17 22:29:07.825658","last_active":"2024-04-17 22:29:07.826848","last_peered":"2024-04-17 22:29:07.826848","last_clean":"2024-04-17 22:29:07.826848","last_became_active":"2024-04-17 22:29:07.825349","last_became_peered":"2024-04-17 22:29:07.825349","last_unstale":"2024-04-17 22:29:07.826848","last_undegraded":"2024-04-17 22:29:07.826848","last_fullsized":"2024-04-17 22:29:07.826848","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.30","version":"15'4","reported_seq":"14","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.824396","last_change":"2024-04-17 22:29:07.475470","last_active":"2024-04-17 22:29:07.824396","last_peered":"2024-04-17 22:29:07.824396","last_clean":"2024-04-17 22:29:07.824396","last_became_active":"2024-04-17 22:29:07.474917","last_became_peered":"2024-04-17 22:29:07.474917","last_unstale":"2024-04-17 22:29:07.824396","last_undegraded":"2024-04-17 22:29:07.824396","last_fullsized":"2024-04-17 22:29:07.824396","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.826262","last_change":"2024-04-17 22:29:07.823624","last_active":"2024-04-17 22:29:07.826262","last_peered":"2024-04-17 22:29:07.826262","last_clean":"2024-04-17 22:29:07.826262","last_became_active":"2024-04-17 22:29:07.822815","last_became_peered":"2024-04-17 22:29:07.822815","last_unstale":"2024-04-17 22:29:07.826262","last_undegraded":"2024-04-17 22:29:07.826262","last_fullsized":"2024-04-17 22:29:07.826262","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.31","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.804183","last_change":"2024-04-17 22:29:07.474318","last_active":"2024-04-17 22:29:07.804183","last_peered":"2024-04-17 22:29:07.804183","last_clean":"2024-04-17 22:29:07.804183","last_became_active":"2024-04-17 22:29:07.473754","last_became_peered":"2024-04-17 22:29:07.473754","last_unstale":"2024-04-17 22:29:07.804183","last_undegraded":"2024-04-17 22:29:07.804183","last_fullsized":"2024-04-17 22:29:07.804183","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.833495","last_change":"2024-04-17 22:29:07.829098","last_active":"2024-04-17 22:29:07.833495","last_peered":"2024-04-17 22:29:07.833495","last_clean":"2024-04-17 22:29:07.833495","last_became_active":"2024-04-17 22:29:07.828773","last_became_peered":"2024-04-17 22:29:07.828773","last_unstale":"2024-04-17 22:29:07.833495","last_undegraded":"2024-04-17 22:29:07.833495","last_fullsized":"2024-04-17 22:29:07.833495","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.32","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.795411","last_change":"2024-04-17 22:29:07.470666","last_active":"2024-04-17 22:29:07.795411","last_peered":"2024-04-17 22:29:07.795411","last_clean":"2024-04-17 22:29:07.795411","last_became_active":"2024-04-17 22:29:07.470363","last_became_peered":"2024-04-17 22:29:07.470363","last_unstale":"2024-04-17 22:29:07.795411","last_undegraded":"2024-04-17 22:29:07.795411","last_fullsized":"2024-04-17 22:29:07.795411","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.830343","last_change":"2024-04-17 22:29:07.827580","last_active":"2024-04-17 22:29:07.830343","last_peered":"2024-04-17 22:29:07.830343","last_clean":"2024-04-17 22:29:07.830343","last_became_active":"2024-04-17 22:29:07.827233","last_became_peered":"2024-04-17 22:29:07.827233","last_unstale":"2024-04-17 22:29:07.830343","last_undegraded":"2024-04-17 22:29:07.830343","last_fullsized":"2024-04-17 22:29:07.830343","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.33","version":"15'1","reported_seq":"11","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.811613","last_change":"2024-04-17 22:29:07.460978","last_active":"2024-04-17 22:29:07.811613","last_peered":"2024-04-17 22:29:07.811613","last_clean":"2024-04-17 22:29:07.811613","last_became_active":"2024-04-17 22:29:07.460659","last_became_peered":"2024-04-17 22:29:07.460659","last_unstale":"2024-04-17 22:29:07.811613","last_undegraded":"2024-04-17 22:29:07.811613","last_fullsized":"2024-04-17 22:29:07.811613","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.832128","last_change":"2024-04-17 22:29:07.816335","last_active":"2024-04-17 22:29:07.832128","last_peered":"2024-04-17 22:29:07.832128","last_clean":"2024-04-17 22:29:07.832128","last_became_active":"2024-04-17 22:29:07.816090","last_became_peered":"2024-04-17 22:29:07.816090","last_unstale":"2024-04-17 22:29:07.832128","last_undegraded":"2024-04-17 22:29:07.832128","last_fullsized":"2024-04-17 22:29:07.832128","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.34","version":"15'4","reported_seq":"14","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.835387","last_change":"2024-04-17 22:29:07.475702","last_active":"2024-04-17 22:29:07.835387","last_peered":"2024-04-17 22:29:07.835387","last_clean":"2024-04-17 22:29:07.835387","last_became_active":"2024-04-17 22:29:07.475395","last_became_peered":"2024-04-17 22:29:07.475395","last_unstale":"2024-04-17 22:29:07.835387","last_undegraded":"2024-04-17 22:29:07.835387","last_fullsized":"2024-04-17 22:29:07.835387","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":892,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.828199","last_change":"2024-04-17 22:29:07.818538","last_active":"2024-04-17 22:29:07.828199","last_peered":"2024-04-17 22:29:07.828199","last_clean":"2024-04-17 22:29:07.828199","last_became_active":"2024-04-17 22:29:07.817164","last_became_peered":"2024-04-17 22:29:07.817164","last_unstale":"2024-04-17 22:29:07.828199","last_undegraded":"2024-04-17 22:29:07.828199","last_fullsized":"2024-04-17 22:29:07.828199","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.35","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.797071","last_change":"2024-04-17 22:29:07.467651","last_active":"2024-04-17 22:29:07.797071","last_peered":"2024-04-17 22:29:07.797071","last_clean":"2024-04-17 22:29:07.797071","last_became_active":"2024-04-17 22:29:07.467327","last_became_peered":"2024-04-17 22:29:07.467327","last_unstale":"2024-04-17 22:29:07.797071","last_undegraded":"2024-04-17 22:29:07.797071","last_fullsized":"2024-04-17 22:29:07.797071","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.810303","last_change":"2024-04-17 22:29:07.809693","last_active":"2024-04-17 22:29:07.810303","last_peered":"2024-04-17 22:29:07.810303","last_clean":"2024-04-17 22:29:07.810303","last_became_active":"2024-04-17 22:29:07.809415","last_became_peered":"2024-04-17 22:29:07.809415","last_unstale":"2024-04-17 22:29:07.810303","last_undegraded":"2024-04-17 22:29:07.810303","last_fullsized":"2024-04-17 22:29:07.810303","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.36","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.796841","last_change":"2024-04-17 22:29:07.483402","last_active":"2024-04-17 22:29:07.796841","last_peered":"2024-04-17 22:29:07.796841","last_clean":"2024-04-17 22:29:07.796841","last_became_active":"2024-04-17 22:29:07.482914","last_became_peered":"2024-04-17 22:29:07.482914","last_unstale":"2024-04-17 22:29:07.796841","last_undegraded":"2024-04-17 22:29:07.796841","last_fullsized":"2024-04-17 22:29:07.796841","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.830317","last_change":"2024-04-17 22:29:07.827623","last_active":"2024-04-17 22:29:07.830317","last_peered":"2024-04-17 22:29:07.830317","last_clean":"2024-04-17 22:29:07.830317","last_became_active":"2024-04-17 22:29:07.827337","last_became_peered":"2024-04-17 22:29:07.827337","last_unstale":"2024-04-17 22:29:07.830317","last_undegraded":"2024-04-17 22:29:07.830317","last_fullsized":"2024-04-17 22:29:07.830317","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.37","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.808508","last_change":"2024-04-17 22:29:07.463449","last_active":"2024-04-17 22:29:07.808508","last_peered":"2024-04-17 22:29:07.808508","last_clean":"2024-04-17 22:29:07.808508","last_became_active":"2024-04-17 22:29:07.463110","last_became_peered":"2024-04-17 22:29:07.463110","last_unstale":"2024-04-17 22:29:07.808508","last_undegraded":"2024-04-17 22:29:07.808508","last_fullsized":"2024-04-17 22:29:07.808508","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.827399","last_change":"2024-04-17 22:29:07.815214","last_active":"2024-04-17 22:29:07.827399","last_peered":"2024-04-17 22:29:07.827399","last_clean":"2024-04-17 22:29:07.827399","last_became_active":"2024-04-17 22:29:07.814938","last_became_peered":"2024-04-17 22:29:07.814938","last_unstale":"2024-04-17 22:29:07.827399","last_undegraded":"2024-04-17 22:29:07.827399","last_fullsized":"2024-04-17 22:29:07.827399","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.38","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.801328","last_change":"2024-04-17 22:29:07.474682","last_active":"2024-04-17 22:29:07.801328","last_peered":"2024-04-17 22:29:07.801328","last_clean":"2024-04-17 22:29:07.801328","last_became_active":"2024-04-17 22:29:07.474305","last_became_peered":"2024-04-17 22:29:07.474305","last_unstale":"2024-04-17 22:29:07.801328","last_undegraded":"2024-04-17 22:29:07.801328","last_fullsized":"2024-04-17 22:29:07.801328","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.833512","last_change":"2024-04-17 22:29:07.829769","last_active":"2024-04-17 22:29:07.833512","last_peered":"2024-04-17 22:29:07.833512","last_clean":"2024-04-17 22:29:07.833512","last_became_active":"2024-04-17 22:29:07.829484","last_became_peered":"2024-04-17 22:29:07.829484","last_unstale":"2024-04-17 22:29:07.833512","last_undegraded":"2024-04-17 22:29:07.833512","last_fullsized":"2024-04-17 22:29:07.833512","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.39","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.794112","last_change":"2024-04-17 22:29:07.458386","last_active":"2024-04-17 22:29:07.794112","last_peered":"2024-04-17 22:29:07.794112","last_clean":"2024-04-17 22:29:07.794112","last_became_active":"2024-04-17 22:29:07.456695","last_became_peered":"2024-04-17 22:29:07.456695","last_unstale":"2024-04-17 22:29:07.794112","last_undegraded":"2024-04-17 22:29:07.794112","last_fullsized":"2024-04-17 22:29:07.794112","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.829903","last_change":"2024-04-17 22:29:07.821490","last_active":"2024-04-17 22:29:07.829903","last_peered":"2024-04-17 22:29:07.829903","last_clean":"2024-04-17 22:29:07.829903","last_became_active":"2024-04-17 22:29:07.821103","last_became_peered":"2024-04-17 22:29:07.821103","last_unstale":"2024-04-17 22:29:07.829903","last_undegraded":"2024-04-17 22:29:07.829903","last_fullsized":"2024-04-17 22:29:07.829903","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3a","version":"15'1","reported_seq":"11","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.819620","last_change":"2024-04-17 22:29:07.472610","last_active":"2024-04-17 22:29:07.819620","last_peered":"2024-04-17 22:29:07.819620","last_clean":"2024-04-17 22:29:07.819620","last_became_active":"2024-04-17 22:29:07.472297","last_became_peered":"2024-04-17 22:29:07.472297","last_unstale":"2024-04-17 22:29:07.819620","last_undegraded":"2024-04-17 22:29:07.819620","last_fullsized":"2024-04-17 22:29:07.819620","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.824029","last_change":"2024-04-17 22:29:07.820738","last_active":"2024-04-17 22:29:07.824029","last_peered":"2024-04-17 22:29:07.824029","last_clean":"2024-04-17 22:29:07.824029","last_became_active":"2024-04-17 22:29:07.820354","last_became_peered":"2024-04-17 22:29:07.820354","last_unstale":"2024-04-17 22:29:07.824029","last_undegraded":"2024-04-17 22:29:07.824029","last_fullsized":"2024-04-17 22:29:07.824029","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3b","version":"15'1","reported_seq":"11","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.824121","last_change":"2024-04-17 22:29:07.473963","last_active":"2024-04-17 22:29:07.824121","last_peered":"2024-04-17 22:29:07.824121","last_clean":"2024-04-17 22:29:07.824121","last_became_active":"2024-04-17 22:29:07.472018","last_became_peered":"2024-04-17 22:29:07.472018","last_unstale":"2024-04-17 22:29:07.824121","last_undegraded":"2024-04-17 22:29:07.824121","last_fullsized":"2024-04-17 22:29:07.824121","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.832188","last_change":"2024-04-17 22:29:07.824919","last_active":"2024-04-17 22:29:07.832188","last_peered":"2024-04-17 22:29:07.832188","last_clean":"2024-04-17 22:29:07.832188","last_became_active":"2024-04-17 22:29:07.824600","last_became_peered":"2024-04-17 22:29:07.824600","last_unstale":"2024-04-17 22:29:07.832188","last_undegraded":"2024-04-17 22:29:07.832188","last_fullsized":"2024-04-17 22:29:07.832188","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3c","version":"15'2","reported_seq":"12","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.825427","last_change":"2024-04-17 22:29:07.475485","last_active":"2024-04-17 22:29:07.825427","last_peered":"2024-04-17 22:29:07.825427","last_clean":"2024-04-17 22:29:07.825427","last_became_active":"2024-04-17 22:29:07.475182","last_became_peered":"2024-04-17 22:29:07.475182","last_unstale":"2024-04-17 22:29:07.825427","last_undegraded":"2024-04-17 22:29:07.825427","last_fullsized":"2024-04-17 22:29:07.825427","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.821750","last_change":"2024-04-17 22:29:07.810455","last_active":"2024-04-17 22:29:07.821750","last_peered":"2024-04-17 22:29:07.821750","last_clean":"2024-04-17 22:29:07.821750","last_became_active":"2024-04-17 22:29:07.810143","last_became_peered":"2024-04-17 22:29:07.810143","last_unstale":"2024-04-17 22:29:07.821750","last_undegraded":"2024-04-17 22:29:07.821750","last_fullsized":"2024-04-17 22:29:07.821750","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3d","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.797015","last_change":"2024-04-17 22:29:07.468187","last_active":"2024-04-17 22:29:07.797015","last_peered":"2024-04-17 22:29:07.797015","last_clean":"2024-04-17 22:29:07.797015","last_became_active":"2024-04-17 22:29:07.467849","last_became_peered":"2024-04-17 22:29:07.467849","last_unstale":"2024-04-17 22:29:07.797015","last_undegraded":"2024-04-17 22:29:07.797015","last_fullsized":"2024-04-17 22:29:07.797015","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.830571","last_change":"2024-04-17 22:29:07.822173","last_active":"2024-04-17 22:29:07.830571","last_peered":"2024-04-17 22:29:07.830571","last_clean":"2024-04-17 22:29:07.830571","last_became_active":"2024-04-17 22:29:07.821875","last_became_peered":"2024-04-17 22:29:07.821875","last_unstale":"2024-04-17 22:29:07.830571","last_undegraded":"2024-04-17 22:29:07.830571","last_fullsized":"2024-04-17 22:29:07.830571","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3e","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.807733","last_change":"2024-04-17 22:29:07.479547","last_active":"2024-04-17 22:29:07.807733","last_peered":"2024-04-17 22:29:07.807733","last_clean":"2024-04-17 22:29:07.807733","last_became_active":"2024-04-17 22:29:07.479004","last_became_peered":"2024-04-17 22:29:07.479004","last_unstale":"2024-04-17 22:29:07.807733","last_undegraded":"2024-04-17 22:29:07.807733","last_fullsized":"2024-04-17 22:29:07.807733","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.823835","last_change":"2024-04-17 22:29:07.819432","last_active":"2024-04-17 22:29:07.823835","last_peered":"2024-04-17 22:29:07.823835","last_clean":"2024-04-17 22:29:07.823835","last_became_active":"2024-04-17 22:29:07.819001","last_became_peered":"2024-04-17 22:29:07.819001","last_unstale":"2024-04-17 22:29:07.823835","last_undegraded":"2024-04-17 22:29:07.823835","last_fullsized":"2024-04-17 22:29:07.823835","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3f","version":"15'3","reported_seq":"13","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.829638","last_change":"2024-04-17 22:29:07.466765","last_active":"2024-04-17 22:29:07.829638","last_peered":"2024-04-17 22:29:07.829638","last_clean":"2024-04-17 22:29:07.829638","last_became_active":"2024-04-17 22:29:07.463177","last_became_peered":"2024-04-17 22:29:07.463177","last_unstale":"2024-04-17 22:29:07.829638","last_undegraded":"2024-04-17 22:29:07.829638","last_fullsized":"2024-04-17 22:29:07.829638","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":544,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":4},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":4},{"poolid":2,"num_pg":64,"stat_sum":{"num_bytes":2286,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":128,"acting":128,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705671,"num_pgs":82,"num_osds":1,"num_per_pool_osds":1,"kb":94371840,"kb_used":1048744,"kb_used_data":160,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323096,"statfs":{"total":96636764160,"available":95562850304,"internally_reserved":1073741824,"allocated":163840,"data_stored":41000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29899999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33200000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40699999999999997}]}]},{"osd":2,"up_from":9,"seq":38654705671,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"kb":94371840,"kb_used":1048732,"kb_used_data":148,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323108,"statfs":{"total":96636764160,"available":95562862592,"internally_reserved":1073741824,"allocated":151552,"data_stored":39984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27500000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.22900000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33500000000000002}]}]},{"osd":0,"up_from":9,"seq":38654705671,"num_pgs":78,"num_osds":1,"num_per_pool_osds":1,"kb":94371840,"kb_used":1048736,"kb_used_data":152,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323104,"statfs":{"total":96636764160,"available":95562858496,"internally_reserved":1073741824,"allocated":155648,"data_stored":40772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36499999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38400000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41199999999999998}]}]},{"osd":1,"up_from":9,"seq":38654705670,"num_pgs":63,"num_osds":1,"num_per_pool_osds":1,"kb":94371840,"kb_used":1048732,"kb_used_data":148,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323108,"statfs":{"total":96636764160,"available":95562862592,"internally_reserved":1073741824,"allocated":151552,"data_stored":39936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.22600000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.25900000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28000000000000003}]}]}]}} 2024-04-17T22:29:18.348 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-17T22:29:18.511 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.522 7f5e09ae5700 1 Processor -- start 2024-04-17T22:29:18.511 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.522 7f5e09ae5700 1 -- start start 2024-04-17T22:29:18.511 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.522 7f5e09ae5700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f5e040f5770 0x7f5e0410c110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:18.511 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.522 7f5e09ae5700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5e0410c740 0x7f5e041a0c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:18.511 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.522 7f5e09ae5700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5e0410cb90 0x7f5e041a1230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:18.511 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.522 7f5e09ae5700 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e040f93e0 con 0x7f5e040f5770 2024-04-17T22:29:18.511 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.522 7f5e09ae5700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e041a1860 con 0x7f5e0410cb90 2024-04-17T22:29:18.512 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.522 7f5e09ae5700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5e041a1a90 con 0x7f5e0410c740 2024-04-17T22:29:18.512 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.522 7f5dfbfff700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5e0410c740 0x7f5e041a0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:18.512 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.522 7f5e0285d700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f5e040f5770 0x7f5e0410c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:18.512 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.522 7f5e0305e700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5e0410cb90 0x7f5e041a1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:18.512 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.522 7f5e0285d700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f5e040f5770 0x7f5e0410c110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:40976/0 (socket says 172.21.15.102:40976) 2024-04-17T22:29:18.512 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.522 7f5e0305e700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5e0410cb90 0x7f5e041a1230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:40408/0 (socket says 172.21.15.102:40408) 2024-04-17T22:29:18.512 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.522 7f5dfbfff700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5e0410c740 0x7f5e041a0c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:44358/0 (socket says 172.21.15.102:44358) 2024-04-17T22:29:18.512 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.522 7f5e0285d700 1 -- 172.21.15.102:0/677257847 learned_addr learned my addr 172.21.15.102:0/677257847 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:29:18.512 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.522 7f5e0285d700 1 -- 172.21.15.102:0/677257847 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5e0410c740 msgr2=0x7f5e041a0c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:29:18.512 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.522 7f5e0285d700 1 --2- 172.21.15.102:0/677257847 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5e0410c740 0x7f5e041a0c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:18.512 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.522 7f5e0285d700 1 -- 172.21.15.102:0/677257847 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5e0410cb90 msgr2=0x7f5e041a1230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:29:18.512 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.522 7f5e0285d700 1 --2- 172.21.15.102:0/677257847 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5e0410cb90 0x7f5e041a1230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:18.513 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.522 7f5e0285d700 1 -- 172.21.15.102:0/677257847 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e041a8310 con 0x7f5e040f5770 2024-04-17T22:29:18.513 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.522 7f5dfbfff700 1 --2- 172.21.15.102:0/677257847 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5e0410c740 0x7f5e041a0c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:29:18.513 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.522 7f5e0305e700 1 --2- 172.21.15.102:0/677257847 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5e0410cb90 0x7f5e041a1230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:29:18.513 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.523 7f5e0285d700 1 --2- 172.21.15.102:0/677257847 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5e040f5770 0x7f5e0410c110 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 rx=0x7f5df4002f70 tx=0x7f5df4009a50).ready entity=mon.0 client_cookie=5f03f368fcb47f69 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:29:18.513 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.523 7f5e0205c700 1 -- 172.21.15.102:0/677257847 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f5e041a8310 con 0x7f5e040f5770 2024-04-17T22:29:18.513 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.523 7f5e0205c700 1 -- 172.21.15.102:0/677257847 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5df400f510 con 0x7f5e040f5770 2024-04-17T22:29:18.513 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.523 7f5e0205c700 1 -- 172.21.15.102:0/677257847 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f5df400f6f0 con 0x7f5e040f5770 2024-04-17T22:29:18.513 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.523 7f5e09ae5700 1 -- 172.21.15.102:0/677257847 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5e040f5770 msgr2=0x7f5e0410c110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:18.513 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.523 7f5e09ae5700 1 --2- 172.21.15.102:0/677257847 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5e040f5770 0x7f5e0410c110 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 rx=0x7f5df4002f70 tx=0x7f5df4009a50).stop 2024-04-17T22:29:18.513 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.523 7f5e09ae5700 1 -- 172.21.15.102:0/677257847 shutdown_connections 2024-04-17T22:29:18.513 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.523 7f5e09ae5700 1 --2- 172.21.15.102:0/677257847 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5e0410cb90 0x7f5e041a1230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:18.513 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.523 7f5e09ae5700 1 --2- 172.21.15.102:0/677257847 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5e040f5770 0x7f5e0410c110 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:18.514 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.523 7f5e09ae5700 1 --2- 172.21.15.102:0/677257847 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5e0410c740 0x7f5e041a0c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:18.514 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.523 7f5e09ae5700 1 -- 172.21.15.102:0/677257847 >> 172.21.15.102:0/677257847 conn(0x7f5e041138c0 msgr2=0x7f5e040f4f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:29:18.514 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.523 7f5e09ae5700 1 -- 172.21.15.102:0/677257847 shutdown_connections 2024-04-17T22:29:18.514 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.523 7f5e09ae5700 1 -- 172.21.15.102:0/677257847 wait complete. 2024-04-17T22:29:18.535 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.546 7f5e09ae5700 1 Processor -- start 2024-04-17T22:29:18.535 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.546 7f5e09ae5700 1 -- start start 2024-04-17T22:29:18.535 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.546 7f5e09ae5700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5e040df340 0x7f5e041a80b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:18.535 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.546 7f5e09ae5700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5e041a86e0 0x7f5e040ca570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:18.535 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.546 7f5e09ae5700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5e041a8b30 0x7f5e040caba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:18.535 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.546 7f5e09ae5700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e040e2d30 con 0x7f5e040df340 2024-04-17T22:29:18.535 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.546 7f5e09ae5700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e040f77c0 con 0x7f5e041a8b30 2024-04-17T22:29:18.535 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.546 7f5e09ae5700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5e040cb6e0 con 0x7f5e041a86e0 2024-04-17T22:29:18.535 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.546 7f5e0285d700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5e040df340 0x7f5e041a80b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:18.536 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.546 7f5e0285d700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5e040df340 0x7f5e041a80b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:40992/0 (socket says 172.21.15.102:40992) 2024-04-17T22:29:18.536 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.546 7f5e0285d700 1 -- 172.21.15.102:0/4109259154 learned_addr learned my addr 172.21.15.102:0/4109259154 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:29:18.536 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.546 7f5dfbfff700 1 --2- 172.21.15.102:0/4109259154 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5e041a86e0 0x7f5e040ca570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:18.536 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.546 7f5e0305e700 1 --2- 172.21.15.102:0/4109259154 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5e041a8b30 0x7f5e040caba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:18.536 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.547 7f5e0285d700 1 -- 172.21.15.102:0/4109259154 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5e041a86e0 msgr2=0x7f5e040ca570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:18.536 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.547 7f5e0285d700 1 --2- 172.21.15.102:0/4109259154 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5e041a86e0 0x7f5e040ca570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:18.536 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.547 7f5e0285d700 1 -- 172.21.15.102:0/4109259154 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5e041a8b30 msgr2=0x7f5e040caba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:18.536 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.547 7f5e0285d700 1 --2- 172.21.15.102:0/4109259154 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5e041a8b30 0x7f5e040caba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:18.536 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.547 7f5e0285d700 1 -- 172.21.15.102:0/4109259154 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e040cb1d0 con 0x7f5e040df340 2024-04-17T22:29:18.536 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.547 7f5dfbfff700 1 --2- 172.21.15.102:0/4109259154 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5e041a86e0 0x7f5e040ca570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:29:18.536 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.547 7f5e0285d700 1 --2- 172.21.15.102:0/4109259154 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5e040df340 0x7f5e041a80b0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 rx=0x7f5e040c5bb0 tx=0x7f5df40068c0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:29:18.536 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.547 7f5e00859700 1 -- 172.21.15.102:0/4109259154 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f5e040cb1d0 con 0x7f5e040df340 2024-04-17T22:29:18.537 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.547 7f5e00859700 1 -- 172.21.15.102:0/4109259154 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5df4011610 con 0x7f5e040df340 2024-04-17T22:29:18.537 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.547 7f5e00859700 1 -- 172.21.15.102:0/4109259154 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f5df40119e0 con 0x7f5e040df340 2024-04-17T22:29:18.537 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.547 7f5e09ae5700 1 -- 172.21.15.102:0/4109259154 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e041bb500 con 0x7f5e040df340 2024-04-17T22:29:18.537 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.547 7f5e09ae5700 1 -- 172.21.15.102:0/4109259154 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e041bb8e0 con 0x7f5e040df340 2024-04-17T22:29:18.537 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.547 7f5e00859700 1 -- 172.21.15.102:0/4109259154 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7f5df4011be0 con 0x7f5e040df340 2024-04-17T22:29:18.537 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.548 7f5e00859700 1 -- 172.21.15.102:0/4109259154 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 3194+0+0 (secure 0 0 0) 0x7f5df4020ba0 con 0x7f5e040df340 2024-04-17T22:29:18.537 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.548 7f5e09ae5700 1 -- 172.21.15.102:0/4109259154 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e040f0820 con 0x7f5e040df340 2024-04-17T22:29:18.537 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.551 7f5e00859700 1 -- 172.21.15.102:0/4109259154 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+110455 (secure 0 0 0) 0x7f5df4011400 con 0x7f5e040df340 2024-04-17T22:29:18.639 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.652 7f5e09ae5700 1 -- 172.21.15.102:0/4109259154 --> v1:172.21.15.102:6816/42360 -- command(tid 0: {"prefix": "pg dump", "target": ["mgr", ""], "format": "json"}) v1 -- 0x7f5e0408a300 con 0x7f5de80145a0 2024-04-17T22:29:18.646 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.660 7f5e00859700 1 -- 172.21.15.102:0/4109259154 <== mgr.4104 v1:172.21.15.102:6816/42360 1 ==== command_reply(tid 0: 0 dumped all) v1 ==== 18+0+306120 (unknown 2965378022 0 1331125451) 0x7f5dec00bae0 con 0x7f5de80145a0 2024-04-17T22:29:18.650 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-17T22:29:18.653 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.662 7f5e09ae5700 1 -- 172.21.15.102:0/4109259154 >> v1:172.21.15.102:6816/42360 conn(0x7f5de80145a0 legacy=0x7f5de8016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:18.653 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.662 7f5e09ae5700 1 -- 172.21.15.102:0/4109259154 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5e040df340 msgr2=0x7f5e041a80b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:18.653 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.662 7f5e09ae5700 1 --2- 172.21.15.102:0/4109259154 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5e040df340 0x7f5e041a80b0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 rx=0x7f5e040c5bb0 tx=0x7f5df40068c0).stop 2024-04-17T22:29:18.653 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.662 7f5e09ae5700 1 -- 172.21.15.102:0/4109259154 shutdown_connections 2024-04-17T22:29:18.653 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.662 7f5e09ae5700 1 --2- 172.21.15.102:0/4109259154 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5e041a8b30 0x7f5e040caba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:18.653 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.662 7f5e09ae5700 1 --2- 172.21.15.102:0/4109259154 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5e040df340 0x7f5e041a80b0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:18.653 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.662 7f5e09ae5700 1 --2- 172.21.15.102:0/4109259154 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5e041a86e0 0x7f5e040ca570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:18.653 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.662 7f5e09ae5700 1 -- 172.21.15.102:0/4109259154 >> 172.21.15.102:0/4109259154 conn(0x7f5e040c3080 msgr2=0x7f5e040d0bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:29:18.653 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.662 7f5e09ae5700 1 -- 172.21.15.102:0/4109259154 shutdown_connections 2024-04-17T22:29:18.653 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.662 7f5e09ae5700 1 -- 172.21.15.102:0/4109259154 wait complete. 2024-04-17T22:29:18.653 INFO:teuthology.orchestra.run.smithi102.stderr:dumped all 2024-04-17T22:29:18.660 INFO:teuthology.orchestra.run.smithi102.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2024-04-17 22:29:17.188866","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2286,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":272,"acting":272,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":272,"num_osds":4,"num_per_pool_osds":4,"kb":377487360,"kb_used":4194944,"kb_used_data":608,"kb_used_omap":0,"kb_used_meta":4194304,"kb_avail":373292416,"statfs":{"total":386547056640,"available":382251433984,"internally_reserved":4294967296,"allocated":622592,"data_stored":161692,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4294967296},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001302"},"pg_stats":[{"pgid":"3.1a","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.820762","last_change":"2024-04-17 22:29:07.814049","last_active":"2024-04-17 22:29:07.820762","last_peered":"2024-04-17 22:29:07.820762","last_clean":"2024-04-17 22:29:07.820762","last_became_active":"2024-04-17 22:29:07.811049","last_became_peered":"2024-04-17 22:29:07.811049","last_unstale":"2024-04-17 22:29:07.820762","last_undegraded":"2024-04-17 22:29:07.820762","last_fullsized":"2024-04-17 22:29:07.820762","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.807568","last_change":"2024-04-17 22:29:07.477053","last_active":"2024-04-17 22:29:07.807568","last_peered":"2024-04-17 22:29:07.807568","last_clean":"2024-04-17 22:29:07.807568","last_became_active":"2024-04-17 22:29:07.476568","last_became_peered":"2024-04-17 22:29:07.476568","last_unstale":"2024-04-17 22:29:07.807568","last_undegraded":"2024-04-17 22:29:07.807568","last_fullsized":"2024-04-17 22:29:07.807568","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.828628","last_change":"2024-04-17 22:29:07.818787","last_active":"2024-04-17 22:29:07.828628","last_peered":"2024-04-17 22:29:07.828628","last_clean":"2024-04-17 22:29:07.828628","last_became_active":"2024-04-17 22:29:07.818347","last_became_peered":"2024-04-17 22:29:07.818347","last_unstale":"2024-04-17 22:29:07.828628","last_undegraded":"2024-04-17 22:29:07.828628","last_fullsized":"2024-04-17 22:29:07.828628","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.805356","last_change":"2024-04-17 22:29:07.477933","last_active":"2024-04-17 22:29:07.805356","last_peered":"2024-04-17 22:29:07.805356","last_clean":"2024-04-17 22:29:07.805356","last_became_active":"2024-04-17 22:29:07.472128","last_became_peered":"2024-04-17 22:29:07.472128","last_unstale":"2024-04-17 22:29:07.805356","last_undegraded":"2024-04-17 22:29:07.805356","last_fullsized":"2024-04-17 22:29:07.805356","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.834368","last_change":"2024-04-17 22:29:07.833107","last_active":"2024-04-17 22:29:07.834368","last_peered":"2024-04-17 22:29:07.834368","last_clean":"2024-04-17 22:29:07.834368","last_became_active":"2024-04-17 22:29:07.832822","last_became_peered":"2024-04-17 22:29:07.832822","last_unstale":"2024-04-17 22:29:07.834368","last_undegraded":"2024-04-17 22:29:07.834368","last_fullsized":"2024-04-17 22:29:07.834368","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.802997","last_change":"2024-04-17 22:29:07.471612","last_active":"2024-04-17 22:29:07.802997","last_peered":"2024-04-17 22:29:07.802997","last_clean":"2024-04-17 22:29:07.802997","last_became_active":"2024-04-17 22:29:07.471322","last_became_peered":"2024-04-17 22:29:07.471322","last_unstale":"2024-04-17 22:29:07.802997","last_undegraded":"2024-04-17 22:29:07.802997","last_fullsized":"2024-04-17 22:29:07.802997","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.812548","last_change":"2024-04-17 22:29:07.810081","last_active":"2024-04-17 22:29:07.812548","last_peered":"2024-04-17 22:29:07.812548","last_clean":"2024-04-17 22:29:07.812548","last_became_active":"2024-04-17 22:29:07.809858","last_became_peered":"2024-04-17 22:29:07.809858","last_unstale":"2024-04-17 22:29:07.812548","last_undegraded":"2024-04-17 22:29:07.812548","last_fullsized":"2024-04-17 22:29:07.812548","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.797655","last_change":"2024-04-17 22:29:07.474346","last_active":"2024-04-17 22:29:07.797655","last_peered":"2024-04-17 22:29:07.797655","last_clean":"2024-04-17 22:29:07.797655","last_became_active":"2024-04-17 22:29:07.473783","last_became_peered":"2024-04-17 22:29:07.473783","last_unstale":"2024-04-17 22:29:07.797655","last_undegraded":"2024-04-17 22:29:07.797655","last_fullsized":"2024-04-17 22:29:07.797655","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.825049","last_change":"2024-04-17 22:29:07.811047","last_active":"2024-04-17 22:29:07.825049","last_peered":"2024-04-17 22:29:07.825049","last_clean":"2024-04-17 22:29:07.825049","last_became_active":"2024-04-17 22:29:07.810747","last_became_peered":"2024-04-17 22:29:07.810747","last_unstale":"2024-04-17 22:29:07.825049","last_undegraded":"2024-04-17 22:29:07.825049","last_fullsized":"2024-04-17 22:29:07.825049","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.798589","last_change":"2024-04-17 22:29:07.460228","last_active":"2024-04-17 22:29:07.798589","last_peered":"2024-04-17 22:29:07.798589","last_clean":"2024-04-17 22:29:07.798589","last_became_active":"2024-04-17 22:29:07.459910","last_became_peered":"2024-04-17 22:29:07.459910","last_unstale":"2024-04-17 22:29:07.798589","last_undegraded":"2024-04-17 22:29:07.798589","last_fullsized":"2024-04-17 22:29:07.798589","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.830592","last_change":"2024-04-17 22:29:07.827143","last_active":"2024-04-17 22:29:07.830592","last_peered":"2024-04-17 22:29:07.830592","last_clean":"2024-04-17 22:29:07.830592","last_became_active":"2024-04-17 22:29:07.826842","last_became_peered":"2024-04-17 22:29:07.826842","last_unstale":"2024-04-17 22:29:07.830592","last_undegraded":"2024-04-17 22:29:07.830592","last_fullsized":"2024-04-17 22:29:07.830592","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.807458","last_change":"2024-04-17 22:29:07.477248","last_active":"2024-04-17 22:29:07.807458","last_peered":"2024-04-17 22:29:07.807458","last_clean":"2024-04-17 22:29:07.807458","last_became_active":"2024-04-17 22:29:07.476944","last_became_peered":"2024-04-17 22:29:07.476944","last_unstale":"2024-04-17 22:29:07.807458","last_undegraded":"2024-04-17 22:29:07.807458","last_fullsized":"2024-04-17 22:29:07.807458","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.826061","last_change":"2024-04-17 22:29:07.822961","last_active":"2024-04-17 22:29:07.826061","last_peered":"2024-04-17 22:29:07.826061","last_clean":"2024-04-17 22:29:07.826061","last_became_active":"2024-04-17 22:29:07.822592","last_became_peered":"2024-04-17 22:29:07.822592","last_unstale":"2024-04-17 22:29:07.826061","last_undegraded":"2024-04-17 22:29:07.826061","last_fullsized":"2024-04-17 22:29:07.826061","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.15","version":"15'3","reported_seq":"13","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.826924","last_change":"2024-04-17 22:29:07.485166","last_active":"2024-04-17 22:29:07.826924","last_peered":"2024-04-17 22:29:07.826924","last_clean":"2024-04-17 22:29:07.826924","last_became_active":"2024-04-17 22:29:07.484870","last_became_peered":"2024-04-17 22:29:07.484870","last_unstale":"2024-04-17 22:29:07.826924","last_undegraded":"2024-04-17 22:29:07.826924","last_fullsized":"2024-04-17 22:29:07.826924","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.825665","last_change":"2024-04-17 22:29:07.814509","last_active":"2024-04-17 22:29:07.825665","last_peered":"2024-04-17 22:29:07.825665","last_clean":"2024-04-17 22:29:07.825665","last_became_active":"2024-04-17 22:29:07.814119","last_became_peered":"2024-04-17 22:29:07.814119","last_unstale":"2024-04-17 22:29:07.825665","last_undegraded":"2024-04-17 22:29:07.825665","last_fullsized":"2024-04-17 22:29:07.825665","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.14","version":"15'2","reported_seq":"12","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.823699","last_change":"2024-04-17 22:29:07.473309","last_active":"2024-04-17 22:29:07.823699","last_peered":"2024-04-17 22:29:07.823699","last_clean":"2024-04-17 22:29:07.823699","last_became_active":"2024-04-17 22:29:07.472976","last_became_peered":"2024-04-17 22:29:07.472976","last_unstale":"2024-04-17 22:29:07.823699","last_undegraded":"2024-04-17 22:29:07.823699","last_fullsized":"2024-04-17 22:29:07.823699","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":634,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.832027","last_change":"2024-04-17 22:29:07.827915","last_active":"2024-04-17 22:29:07.832027","last_peered":"2024-04-17 22:29:07.832027","last_clean":"2024-04-17 22:29:07.832027","last_became_active":"2024-04-17 22:29:07.827670","last_became_peered":"2024-04-17 22:29:07.827670","last_unstale":"2024-04-17 22:29:07.832027","last_undegraded":"2024-04-17 22:29:07.832027","last_fullsized":"2024-04-17 22:29:07.832027","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"15'1","reported_seq":"11","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.827199","last_change":"2024-04-17 22:29:07.471458","last_active":"2024-04-17 22:29:07.827199","last_peered":"2024-04-17 22:29:07.827199","last_clean":"2024-04-17 22:29:07.827199","last_became_active":"2024-04-17 22:29:07.471085","last_became_peered":"2024-04-17 22:29:07.471085","last_unstale":"2024-04-17 22:29:07.827199","last_undegraded":"2024-04-17 22:29:07.827199","last_fullsized":"2024-04-17 22:29:07.827199","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.833535","last_change":"2024-04-17 22:29:07.829288","last_active":"2024-04-17 22:29:07.833535","last_peered":"2024-04-17 22:29:07.833535","last_clean":"2024-04-17 22:29:07.833535","last_became_active":"2024-04-17 22:29:07.828994","last_became_peered":"2024-04-17 22:29:07.828994","last_unstale":"2024-04-17 22:29:07.833535","last_undegraded":"2024-04-17 22:29:07.833535","last_fullsized":"2024-04-17 22:29:07.833535","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.805144","last_change":"2024-04-17 22:29:07.480194","last_active":"2024-04-17 22:29:07.805144","last_peered":"2024-04-17 22:29:07.805144","last_clean":"2024-04-17 22:29:07.805144","last_became_active":"2024-04-17 22:29:07.479728","last_became_peered":"2024-04-17 22:29:07.479728","last_unstale":"2024-04-17 22:29:07.805144","last_undegraded":"2024-04-17 22:29:07.805144","last_fullsized":"2024-04-17 22:29:07.805144","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.820915","last_change":"2024-04-17 22:29:07.815790","last_active":"2024-04-17 22:29:07.820915","last_peered":"2024-04-17 22:29:07.820915","last_clean":"2024-04-17 22:29:07.820915","last_became_active":"2024-04-17 22:29:07.815632","last_became_peered":"2024-04-17 22:29:07.815632","last_unstale":"2024-04-17 22:29:07.820915","last_undegraded":"2024-04-17 22:29:07.820915","last_fullsized":"2024-04-17 22:29:07.820915","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.804035","last_change":"2024-04-17 22:29:07.475830","last_active":"2024-04-17 22:29:07.804035","last_peered":"2024-04-17 22:29:07.804035","last_clean":"2024-04-17 22:29:07.804035","last_became_active":"2024-04-17 22:29:07.474411","last_became_peered":"2024-04-17 22:29:07.474411","last_unstale":"2024-04-17 22:29:07.804035","last_undegraded":"2024-04-17 22:29:07.804035","last_fullsized":"2024-04-17 22:29:07.804035","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.828447","last_change":"2024-04-17 22:29:07.826101","last_active":"2024-04-17 22:29:07.828447","last_peered":"2024-04-17 22:29:07.828447","last_clean":"2024-04-17 22:29:07.828447","last_became_active":"2024-04-17 22:29:07.822917","last_became_peered":"2024-04-17 22:29:07.822917","last_unstale":"2024-04-17 22:29:07.828447","last_undegraded":"2024-04-17 22:29:07.828447","last_fullsized":"2024-04-17 22:29:07.828447","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.10","version":"15'4","reported_seq":"14","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.817471","last_change":"2024-04-17 22:29:07.457017","last_active":"2024-04-17 22:29:07.817471","last_peered":"2024-04-17 22:29:07.817471","last_clean":"2024-04-17 22:29:07.817471","last_became_active":"2024-04-17 22:29:07.456705","last_became_peered":"2024-04-17 22:29:07.456705","last_unstale":"2024-04-17 22:29:07.817471","last_undegraded":"2024-04-17 22:29:07.817471","last_fullsized":"2024-04-17 22:29:07.817471","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.825122","last_change":"2024-04-17 22:29:07.820464","last_active":"2024-04-17 22:29:07.825122","last_peered":"2024-04-17 22:29:07.825122","last_clean":"2024-04-17 22:29:07.825122","last_became_active":"2024-04-17 22:29:07.820151","last_became_peered":"2024-04-17 22:29:07.820151","last_unstale":"2024-04-17 22:29:07.825122","last_undegraded":"2024-04-17 22:29:07.825122","last_fullsized":"2024-04-17 22:29:07.825122","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.f","version":"15'1","reported_seq":"11","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.828338","last_change":"2024-04-17 22:29:07.478464","last_active":"2024-04-17 22:29:07.828338","last_peered":"2024-04-17 22:29:07.828338","last_clean":"2024-04-17 22:29:07.828338","last_became_active":"2024-04-17 22:29:07.478144","last_became_peered":"2024-04-17 22:29:07.478144","last_unstale":"2024-04-17 22:29:07.828338","last_undegraded":"2024-04-17 22:29:07.828338","last_fullsized":"2024-04-17 22:29:07.828338","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.828463","last_change":"2024-04-17 22:29:07.815171","last_active":"2024-04-17 22:29:07.828463","last_peered":"2024-04-17 22:29:07.828463","last_clean":"2024-04-17 22:29:07.828463","last_became_active":"2024-04-17 22:29:07.814830","last_became_peered":"2024-04-17 22:29:07.814830","last_unstale":"2024-04-17 22:29:07.828463","last_undegraded":"2024-04-17 22:29:07.828463","last_fullsized":"2024-04-17 22:29:07.828463","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"15'2","reported_seq":"12","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.811237","last_change":"2024-04-17 22:29:07.481865","last_active":"2024-04-17 22:29:07.811237","last_peered":"2024-04-17 22:29:07.811237","last_clean":"2024-04-17 22:29:07.811237","last_became_active":"2024-04-17 22:29:07.481506","last_became_peered":"2024-04-17 22:29:07.481506","last_unstale":"2024-04-17 22:29:07.811237","last_undegraded":"2024-04-17 22:29:07.811237","last_fullsized":"2024-04-17 22:29:07.811237","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.828178","last_change":"2024-04-17 22:29:07.818238","last_active":"2024-04-17 22:29:07.828178","last_peered":"2024-04-17 22:29:07.828178","last_clean":"2024-04-17 22:29:07.828178","last_became_active":"2024-04-17 22:29:07.818060","last_became_peered":"2024-04-17 22:29:07.818060","last_unstale":"2024-04-17 22:29:07.828178","last_undegraded":"2024-04-17 22:29:07.828178","last_fullsized":"2024-04-17 22:29:07.828178","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.796955","last_change":"2024-04-17 22:29:07.484999","last_active":"2024-04-17 22:29:07.796955","last_peered":"2024-04-17 22:29:07.796955","last_clean":"2024-04-17 22:29:07.796955","last_became_active":"2024-04-17 22:29:07.484668","last_became_peered":"2024-04-17 22:29:07.484668","last_unstale":"2024-04-17 22:29:07.796955","last_undegraded":"2024-04-17 22:29:07.796955","last_fullsized":"2024-04-17 22:29:07.796955","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.828594","last_change":"2024-04-17 22:29:07.826426","last_active":"2024-04-17 22:29:07.828594","last_peered":"2024-04-17 22:29:07.828594","last_clean":"2024-04-17 22:29:07.828594","last_became_active":"2024-04-17 22:29:07.826109","last_became_peered":"2024-04-17 22:29:07.826109","last_unstale":"2024-04-17 22:29:07.828594","last_undegraded":"2024-04-17 22:29:07.828594","last_fullsized":"2024-04-17 22:29:07.828594","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.797207","last_change":"2024-04-17 22:29:07.462218","last_active":"2024-04-17 22:29:07.797207","last_peered":"2024-04-17 22:29:07.797207","last_clean":"2024-04-17 22:29:07.797207","last_became_active":"2024-04-17 22:29:07.461842","last_became_peered":"2024-04-17 22:29:07.461842","last_unstale":"2024-04-17 22:29:07.797207","last_undegraded":"2024-04-17 22:29:07.797207","last_fullsized":"2024-04-17 22:29:07.797207","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.820296","last_change":"2024-04-17 22:29:07.817144","last_active":"2024-04-17 22:29:07.820296","last_peered":"2024-04-17 22:29:07.820296","last_clean":"2024-04-17 22:29:07.820296","last_became_active":"2024-04-17 22:29:07.816859","last_became_peered":"2024-04-17 22:29:07.816859","last_unstale":"2024-04-17 22:29:07.820296","last_undegraded":"2024-04-17 22:29:07.820296","last_fullsized":"2024-04-17 22:29:07.820296","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"15'1","reported_seq":"11","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.821604","last_change":"2024-04-17 22:29:07.479816","last_active":"2024-04-17 22:29:07.821604","last_peered":"2024-04-17 22:29:07.821604","last_clean":"2024-04-17 22:29:07.821604","last_became_active":"2024-04-17 22:29:07.479251","last_became_peered":"2024-04-17 22:29:07.479251","last_unstale":"2024-04-17 22:29:07.821604","last_undegraded":"2024-04-17 22:29:07.821604","last_fullsized":"2024-04-17 22:29:07.821604","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.833516","last_change":"2024-04-17 22:29:07.829215","last_active":"2024-04-17 22:29:07.833516","last_peered":"2024-04-17 22:29:07.833516","last_clean":"2024-04-17 22:29:07.833516","last_became_active":"2024-04-17 22:29:07.828803","last_became_peered":"2024-04-17 22:29:07.828803","last_unstale":"2024-04-17 22:29:07.833516","last_undegraded":"2024-04-17 22:29:07.833516","last_fullsized":"2024-04-17 22:29:07.833516","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.796428","last_change":"2024-04-17 22:29:07.473128","last_active":"2024-04-17 22:29:07.796428","last_peered":"2024-04-17 22:29:07.796428","last_clean":"2024-04-17 22:29:07.796428","last_became_active":"2024-04-17 22:29:07.472806","last_became_peered":"2024-04-17 22:29:07.472806","last_unstale":"2024-04-17 22:29:07.796428","last_undegraded":"2024-04-17 22:29:07.796428","last_fullsized":"2024-04-17 22:29:07.796428","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.820555","last_change":"2024-04-17 22:29:07.817630","last_active":"2024-04-17 22:29:07.820555","last_peered":"2024-04-17 22:29:07.820555","last_clean":"2024-04-17 22:29:07.820555","last_became_active":"2024-04-17 22:29:07.817333","last_became_peered":"2024-04-17 22:29:07.817333","last_unstale":"2024-04-17 22:29:07.820555","last_undegraded":"2024-04-17 22:29:07.820555","last_fullsized":"2024-04-17 22:29:07.820555","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.792279","last_change":"2024-04-17 22:29:07.465053","last_active":"2024-04-17 22:29:07.792279","last_peered":"2024-04-17 22:29:07.792279","last_clean":"2024-04-17 22:29:07.792279","last_became_active":"2024-04-17 22:29:07.464766","last_became_peered":"2024-04-17 22:29:07.464766","last_unstale":"2024-04-17 22:29:07.792279","last_undegraded":"2024-04-17 22:29:07.792279","last_fullsized":"2024-04-17 22:29:07.792279","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.827066","last_change":"2024-04-17 22:29:07.823514","last_active":"2024-04-17 22:29:07.827066","last_peered":"2024-04-17 22:29:07.827066","last_clean":"2024-04-17 22:29:07.827066","last_became_active":"2024-04-17 22:29:07.823007","last_became_peered":"2024-04-17 22:29:07.823007","last_unstale":"2024-04-17 22:29:07.827066","last_undegraded":"2024-04-17 22:29:07.827066","last_fullsized":"2024-04-17 22:29:07.827066","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.801156","last_change":"2024-04-17 22:29:07.461568","last_active":"2024-04-17 22:29:07.801156","last_peered":"2024-04-17 22:29:07.801156","last_clean":"2024-04-17 22:29:07.801156","last_became_active":"2024-04-17 22:29:07.461249","last_became_peered":"2024-04-17 22:29:07.461249","last_unstale":"2024-04-17 22:29:07.801156","last_undegraded":"2024-04-17 22:29:07.801156","last_fullsized":"2024-04-17 22:29:07.801156","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.828426","last_change":"2024-04-17 22:29:07.826342","last_active":"2024-04-17 22:29:07.828426","last_peered":"2024-04-17 22:29:07.828426","last_clean":"2024-04-17 22:29:07.828426","last_became_active":"2024-04-17 22:29:07.826029","last_became_peered":"2024-04-17 22:29:07.826029","last_unstale":"2024-04-17 22:29:07.828426","last_undegraded":"2024-04-17 22:29:07.828426","last_fullsized":"2024-04-17 22:29:07.828426","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":"13","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.795463","last_change":"2024-04-17 22:28:59.253977","last_active":"2024-04-17 22:29:07.795463","last_peered":"2024-04-17 22:29:07.795463","last_clean":"2024-04-17 22:29:07.795463","last_became_active":"2024-04-17 22:28:59.253641","last_became_peered":"2024-04-17 22:28:59.253641","last_unstale":"2024-04-17 22:29:07.795463","last_undegraded":"2024-04-17 22:29:07.795463","last_fullsized":"2024-04-17 22:29:07.795463","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:28:58.239187","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:28:58.239187","last_clean_scrub_stamp":"2024-04-17 22:28:58.239187","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.796951","last_change":"2024-04-17 22:29:07.466907","last_active":"2024-04-17 22:29:07.796951","last_peered":"2024-04-17 22:29:07.796951","last_clean":"2024-04-17 22:29:07.796951","last_became_active":"2024-04-17 22:29:07.466583","last_became_peered":"2024-04-17 22:29:07.466583","last_unstale":"2024-04-17 22:29:07.796951","last_undegraded":"2024-04-17 22:29:07.796951","last_fullsized":"2024-04-17 22:29:07.796951","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.829316","last_change":"2024-04-17 22:29:07.824875","last_active":"2024-04-17 22:29:07.829316","last_peered":"2024-04-17 22:29:07.829316","last_clean":"2024-04-17 22:29:07.829316","last_became_active":"2024-04-17 22:29:07.824377","last_became_peered":"2024-04-17 22:29:07.824377","last_unstale":"2024-04-17 22:29:07.829316","last_undegraded":"2024-04-17 22:29:07.829316","last_fullsized":"2024-04-17 22:29:07.829316","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":"13","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.809532","last_change":"2024-04-17 22:28:59.255733","last_active":"2024-04-17 22:29:07.809532","last_peered":"2024-04-17 22:29:07.809532","last_clean":"2024-04-17 22:29:07.809532","last_became_active":"2024-04-17 22:28:59.255411","last_became_peered":"2024-04-17 22:28:59.255411","last_unstale":"2024-04-17 22:29:07.809532","last_undegraded":"2024-04-17 22:29:07.809532","last_fullsized":"2024-04-17 22:29:07.809532","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:28:58.239187","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:28:58.239187","last_clean_scrub_stamp":"2024-04-17 22:28:58.239187","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"15'2","reported_seq":"12","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.832132","last_change":"2024-04-17 22:29:07.478196","last_active":"2024-04-17 22:29:07.832132","last_peered":"2024-04-17 22:29:07.832132","last_clean":"2024-04-17 22:29:07.832132","last_became_active":"2024-04-17 22:29:07.477887","last_became_peered":"2024-04-17 22:29:07.477887","last_unstale":"2024-04-17 22:29:07.832132","last_undegraded":"2024-04-17 22:29:07.832132","last_fullsized":"2024-04-17 22:29:07.832132","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":"13","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.796430","last_change":"2024-04-17 22:28:59.252728","last_active":"2024-04-17 22:29:07.796430","last_peered":"2024-04-17 22:29:07.796430","last_clean":"2024-04-17 22:29:07.796430","last_became_active":"2024-04-17 22:28:59.252345","last_became_peered":"2024-04-17 22:28:59.252345","last_unstale":"2024-04-17 22:29:07.796430","last_undegraded":"2024-04-17 22:29:07.796430","last_fullsized":"2024-04-17 22:29:07.796430","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:28:58.239187","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:28:58.239187","last_clean_scrub_stamp":"2024-04-17 22:28:58.239187","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.797727","last_change":"2024-04-17 22:29:07.474520","last_active":"2024-04-17 22:29:07.797727","last_peered":"2024-04-17 22:29:07.797727","last_clean":"2024-04-17 22:29:07.797727","last_became_active":"2024-04-17 22:29:07.474029","last_became_peered":"2024-04-17 22:29:07.474029","last_unstale":"2024-04-17 22:29:07.797727","last_undegraded":"2024-04-17 22:29:07.797727","last_fullsized":"2024-04-17 22:29:07.797727","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.829302","last_change":"2024-04-17 22:29:07.825091","last_active":"2024-04-17 22:29:07.829302","last_peered":"2024-04-17 22:29:07.829302","last_clean":"2024-04-17 22:29:07.829302","last_became_active":"2024-04-17 22:29:07.824832","last_became_peered":"2024-04-17 22:29:07.824832","last_unstale":"2024-04-17 22:29:07.829302","last_undegraded":"2024-04-17 22:29:07.829302","last_fullsized":"2024-04-17 22:29:07.829302","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":"13","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.807580","last_change":"2024-04-17 22:28:59.253392","last_active":"2024-04-17 22:29:07.807580","last_peered":"2024-04-17 22:29:07.807580","last_clean":"2024-04-17 22:29:07.807580","last_became_active":"2024-04-17 22:28:59.253049","last_became_peered":"2024-04-17 22:28:59.253049","last_unstale":"2024-04-17 22:29:07.807580","last_undegraded":"2024-04-17 22:29:07.807580","last_fullsized":"2024-04-17 22:29:07.807580","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:28:58.239187","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:28:58.239187","last_clean_scrub_stamp":"2024-04-17 22:28:58.239187","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.796512","last_change":"2024-04-17 22:29:07.471913","last_active":"2024-04-17 22:29:07.796512","last_peered":"2024-04-17 22:29:07.796512","last_clean":"2024-04-17 22:29:07.796512","last_became_active":"2024-04-17 22:29:07.471686","last_became_peered":"2024-04-17 22:29:07.471686","last_unstale":"2024-04-17 22:29:07.796512","last_undegraded":"2024-04-17 22:29:07.796512","last_fullsized":"2024-04-17 22:29:07.796512","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.820450","last_change":"2024-04-17 22:29:07.812999","last_active":"2024-04-17 22:29:07.820450","last_peered":"2024-04-17 22:29:07.820450","last_clean":"2024-04-17 22:29:07.820450","last_became_active":"2024-04-17 22:29:07.812701","last_became_peered":"2024-04-17 22:29:07.812701","last_unstale":"2024-04-17 22:29:07.820450","last_undegraded":"2024-04-17 22:29:07.820450","last_fullsized":"2024-04-17 22:29:07.820450","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":"13","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.791848","last_change":"2024-04-17 22:28:59.254907","last_active":"2024-04-17 22:29:07.791848","last_peered":"2024-04-17 22:29:07.791848","last_clean":"2024-04-17 22:29:07.791848","last_became_active":"2024-04-17 22:28:59.254633","last_became_peered":"2024-04-17 22:28:59.254633","last_unstale":"2024-04-17 22:29:07.791848","last_undegraded":"2024-04-17 22:29:07.791848","last_fullsized":"2024-04-17 22:29:07.791848","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:28:58.239187","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:28:58.239187","last_clean_scrub_stamp":"2024-04-17 22:28:58.239187","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.807141","last_change":"2024-04-17 22:29:07.480020","last_active":"2024-04-17 22:29:07.807141","last_peered":"2024-04-17 22:29:07.807141","last_clean":"2024-04-17 22:29:07.807141","last_became_active":"2024-04-17 22:29:07.479492","last_became_peered":"2024-04-17 22:29:07.479492","last_unstale":"2024-04-17 22:29:07.807141","last_undegraded":"2024-04-17 22:29:07.807141","last_fullsized":"2024-04-17 22:29:07.807141","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.826030","last_change":"2024-04-17 22:29:07.821883","last_active":"2024-04-17 22:29:07.826030","last_peered":"2024-04-17 22:29:07.826030","last_clean":"2024-04-17 22:29:07.826030","last_became_active":"2024-04-17 22:29:07.821569","last_became_peered":"2024-04-17 22:29:07.821569","last_unstale":"2024-04-17 22:29:07.826030","last_undegraded":"2024-04-17 22:29:07.826030","last_fullsized":"2024-04-17 22:29:07.826030","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":"13","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.806466","last_change":"2024-04-17 22:28:59.254217","last_active":"2024-04-17 22:29:07.806466","last_peered":"2024-04-17 22:29:07.806466","last_clean":"2024-04-17 22:29:07.806466","last_became_active":"2024-04-17 22:28:59.253877","last_became_peered":"2024-04-17 22:28:59.253877","last_unstale":"2024-04-17 22:29:07.806466","last_undegraded":"2024-04-17 22:29:07.806466","last_fullsized":"2024-04-17 22:29:07.806466","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:28:58.239187","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:28:58.239187","last_clean_scrub_stamp":"2024-04-17 22:28:58.239187","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.805257","last_change":"2024-04-17 22:29:07.480378","last_active":"2024-04-17 22:29:07.805257","last_peered":"2024-04-17 22:29:07.805257","last_clean":"2024-04-17 22:29:07.805257","last_became_active":"2024-04-17 22:29:07.479866","last_became_peered":"2024-04-17 22:29:07.479866","last_unstale":"2024-04-17 22:29:07.805257","last_undegraded":"2024-04-17 22:29:07.805257","last_fullsized":"2024-04-17 22:29:07.805257","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.829882","last_change":"2024-04-17 22:29:07.821315","last_active":"2024-04-17 22:29:07.829882","last_peered":"2024-04-17 22:29:07.829882","last_clean":"2024-04-17 22:29:07.829882","last_became_active":"2024-04-17 22:29:07.820937","last_became_peered":"2024-04-17 22:29:07.820937","last_unstale":"2024-04-17 22:29:07.829882","last_undegraded":"2024-04-17 22:29:07.829882","last_fullsized":"2024-04-17 22:29:07.829882","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":"13","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.813317","last_change":"2024-04-17 22:28:59.254830","last_active":"2024-04-17 22:29:07.813317","last_peered":"2024-04-17 22:29:07.813317","last_clean":"2024-04-17 22:29:07.813317","last_became_active":"2024-04-17 22:28:59.254505","last_became_peered":"2024-04-17 22:28:59.254505","last_unstale":"2024-04-17 22:29:07.813317","last_undegraded":"2024-04-17 22:29:07.813317","last_fullsized":"2024-04-17 22:29:07.813317","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:28:58.239187","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:28:58.239187","last_clean_scrub_stamp":"2024-04-17 22:28:58.239187","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.809624","last_change":"2024-04-17 22:29:07.475017","last_active":"2024-04-17 22:29:07.809624","last_peered":"2024-04-17 22:29:07.809624","last_clean":"2024-04-17 22:29:07.809624","last_became_active":"2024-04-17 22:29:07.473559","last_became_peered":"2024-04-17 22:29:07.473559","last_unstale":"2024-04-17 22:29:07.809624","last_undegraded":"2024-04-17 22:29:07.809624","last_fullsized":"2024-04-17 22:29:07.809624","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.811282","last_change":"2024-04-17 22:29:07.808988","last_active":"2024-04-17 22:29:07.811282","last_peered":"2024-04-17 22:29:07.811282","last_clean":"2024-04-17 22:29:07.811282","last_became_active":"2024-04-17 22:29:07.808663","last_became_peered":"2024-04-17 22:29:07.808663","last_unstale":"2024-04-17 22:29:07.811282","last_undegraded":"2024-04-17 22:29:07.811282","last_fullsized":"2024-04-17 22:29:07.811282","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":"13","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.796932","last_change":"2024-04-17 22:28:59.254055","last_active":"2024-04-17 22:29:07.796932","last_peered":"2024-04-17 22:29:07.796932","last_clean":"2024-04-17 22:29:07.796932","last_became_active":"2024-04-17 22:28:59.253750","last_became_peered":"2024-04-17 22:28:59.253750","last_unstale":"2024-04-17 22:29:07.796932","last_undegraded":"2024-04-17 22:29:07.796932","last_fullsized":"2024-04-17 22:29:07.796932","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:28:58.239187","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:28:58.239187","last_clean_scrub_stamp":"2024-04-17 22:28:58.239187","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.796660","last_change":"2024-04-17 22:29:07.466238","last_active":"2024-04-17 22:29:07.796660","last_peered":"2024-04-17 22:29:07.796660","last_clean":"2024-04-17 22:29:07.796660","last_became_active":"2024-04-17 22:29:07.465806","last_became_peered":"2024-04-17 22:29:07.465806","last_unstale":"2024-04-17 22:29:07.796660","last_undegraded":"2024-04-17 22:29:07.796660","last_fullsized":"2024-04-17 22:29:07.796660","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.820702","last_change":"2024-04-17 22:29:07.809155","last_active":"2024-04-17 22:29:07.820702","last_peered":"2024-04-17 22:29:07.820702","last_clean":"2024-04-17 22:29:07.820702","last_became_active":"2024-04-17 22:29:07.808850","last_became_peered":"2024-04-17 22:29:07.808850","last_unstale":"2024-04-17 22:29:07.820702","last_undegraded":"2024-04-17 22:29:07.820702","last_fullsized":"2024-04-17 22:29:07.820702","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.833569","last_change":"2024-04-17 22:29:07.832794","last_active":"2024-04-17 22:29:07.833569","last_peered":"2024-04-17 22:29:07.833569","last_clean":"2024-04-17 22:29:07.833569","last_became_active":"2024-04-17 22:29:07.832467","last_became_peered":"2024-04-17 22:29:07.832467","last_unstale":"2024-04-17 22:29:07.833569","last_undegraded":"2024-04-17 22:29:07.833569","last_fullsized":"2024-04-17 22:29:07.833569","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.811305","last_change":"2024-04-17 22:29:07.477743","last_active":"2024-04-17 22:29:07.811305","last_peered":"2024-04-17 22:29:07.811305","last_clean":"2024-04-17 22:29:07.811305","last_became_active":"2024-04-17 22:29:07.477503","last_became_peered":"2024-04-17 22:29:07.477503","last_unstale":"2024-04-17 22:29:07.811305","last_undegraded":"2024-04-17 22:29:07.811305","last_fullsized":"2024-04-17 22:29:07.811305","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.830256","last_change":"2024-04-17 22:29:07.828401","last_active":"2024-04-17 22:29:07.830256","last_peered":"2024-04-17 22:29:07.830256","last_clean":"2024-04-17 22:29:07.830256","last_became_active":"2024-04-17 22:29:07.828093","last_became_peered":"2024-04-17 22:29:07.828093","last_unstale":"2024-04-17 22:29:07.830256","last_undegraded":"2024-04-17 22:29:07.830256","last_fullsized":"2024-04-17 22:29:07.830256","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"15'2","reported_seq":"12","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.824506","last_change":"2024-04-17 22:29:07.478361","last_active":"2024-04-17 22:29:07.824506","last_peered":"2024-04-17 22:29:07.824506","last_clean":"2024-04-17 22:29:07.824506","last_became_active":"2024-04-17 22:29:07.477793","last_became_peered":"2024-04-17 22:29:07.477793","last_unstale":"2024-04-17 22:29:07.824506","last_undegraded":"2024-04-17 22:29:07.824506","last_fullsized":"2024-04-17 22:29:07.824506","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.828482","last_change":"2024-04-17 22:29:07.815548","last_active":"2024-04-17 22:29:07.828482","last_peered":"2024-04-17 22:29:07.828482","last_clean":"2024-04-17 22:29:07.828482","last_became_active":"2024-04-17 22:29:07.815263","last_became_peered":"2024-04-17 22:29:07.815263","last_unstale":"2024-04-17 22:29:07.828482","last_undegraded":"2024-04-17 22:29:07.828482","last_fullsized":"2024-04-17 22:29:07.828482","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"15'1","reported_seq":"11","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.831868","last_change":"2024-04-17 22:29:07.482507","last_active":"2024-04-17 22:29:07.831868","last_peered":"2024-04-17 22:29:07.831868","last_clean":"2024-04-17 22:29:07.831868","last_became_active":"2024-04-17 22:29:07.481981","last_became_peered":"2024-04-17 22:29:07.481981","last_unstale":"2024-04-17 22:29:07.831868","last_undegraded":"2024-04-17 22:29:07.831868","last_fullsized":"2024-04-17 22:29:07.831868","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.833109","last_change":"2024-04-17 22:29:07.828459","last_active":"2024-04-17 22:29:07.833109","last_peered":"2024-04-17 22:29:07.833109","last_clean":"2024-04-17 22:29:07.833109","last_became_active":"2024-04-17 22:29:07.828157","last_became_peered":"2024-04-17 22:29:07.828157","last_unstale":"2024-04-17 22:29:07.833109","last_undegraded":"2024-04-17 22:29:07.833109","last_fullsized":"2024-04-17 22:29:07.833109","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"15'1","reported_seq":"11","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.828398","last_change":"2024-04-17 22:29:07.471751","last_active":"2024-04-17 22:29:07.828398","last_peered":"2024-04-17 22:29:07.828398","last_clean":"2024-04-17 22:29:07.828398","last_became_active":"2024-04-17 22:29:07.471385","last_became_peered":"2024-04-17 22:29:07.471385","last_unstale":"2024-04-17 22:29:07.828398","last_undegraded":"2024-04-17 22:29:07.828398","last_fullsized":"2024-04-17 22:29:07.828398","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.821701","last_change":"2024-04-17 22:29:07.812125","last_active":"2024-04-17 22:29:07.821701","last_peered":"2024-04-17 22:29:07.821701","last_clean":"2024-04-17 22:29:07.821701","last_became_active":"2024-04-17 22:29:07.811846","last_became_peered":"2024-04-17 22:29:07.811846","last_unstale":"2024-04-17 22:29:07.821701","last_undegraded":"2024-04-17 22:29:07.821701","last_fullsized":"2024-04-17 22:29:07.821701","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.20","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.798111","last_change":"2024-04-17 22:29:07.477127","last_active":"2024-04-17 22:29:07.798111","last_peered":"2024-04-17 22:29:07.798111","last_clean":"2024-04-17 22:29:07.798111","last_became_active":"2024-04-17 22:29:07.476773","last_became_peered":"2024-04-17 22:29:07.476773","last_unstale":"2024-04-17 22:29:07.798111","last_undegraded":"2024-04-17 22:29:07.798111","last_fullsized":"2024-04-17 22:29:07.798111","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.824583","last_change":"2024-04-17 22:29:07.821132","last_active":"2024-04-17 22:29:07.824583","last_peered":"2024-04-17 22:29:07.824583","last_clean":"2024-04-17 22:29:07.824583","last_became_active":"2024-04-17 22:29:07.820838","last_became_peered":"2024-04-17 22:29:07.820838","last_unstale":"2024-04-17 22:29:07.824583","last_undegraded":"2024-04-17 22:29:07.824583","last_fullsized":"2024-04-17 22:29:07.824583","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.21","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.796071","last_change":"2024-04-17 22:29:07.460110","last_active":"2024-04-17 22:29:07.796071","last_peered":"2024-04-17 22:29:07.796071","last_clean":"2024-04-17 22:29:07.796071","last_became_active":"2024-04-17 22:29:07.459611","last_became_peered":"2024-04-17 22:29:07.459611","last_unstale":"2024-04-17 22:29:07.796071","last_undegraded":"2024-04-17 22:29:07.796071","last_fullsized":"2024-04-17 22:29:07.796071","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.828609","last_change":"2024-04-17 22:29:07.818496","last_active":"2024-04-17 22:29:07.828609","last_peered":"2024-04-17 22:29:07.828609","last_clean":"2024-04-17 22:29:07.828609","last_became_active":"2024-04-17 22:29:07.818184","last_became_peered":"2024-04-17 22:29:07.818184","last_unstale":"2024-04-17 22:29:07.828609","last_undegraded":"2024-04-17 22:29:07.828609","last_fullsized":"2024-04-17 22:29:07.828609","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.22","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.805451","last_change":"2024-04-17 22:29:07.481695","last_active":"2024-04-17 22:29:07.805451","last_peered":"2024-04-17 22:29:07.805451","last_clean":"2024-04-17 22:29:07.805451","last_became_active":"2024-04-17 22:29:07.479982","last_became_peered":"2024-04-17 22:29:07.479982","last_unstale":"2024-04-17 22:29:07.805451","last_undegraded":"2024-04-17 22:29:07.805451","last_fullsized":"2024-04-17 22:29:07.805451","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.835093","last_change":"2024-04-17 22:29:07.833317","last_active":"2024-04-17 22:29:07.835093","last_peered":"2024-04-17 22:29:07.835093","last_clean":"2024-04-17 22:29:07.835093","last_became_active":"2024-04-17 22:29:07.827050","last_became_peered":"2024-04-17 22:29:07.827050","last_unstale":"2024-04-17 22:29:07.835093","last_undegraded":"2024-04-17 22:29:07.835093","last_fullsized":"2024-04-17 22:29:07.835093","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.23","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.807837","last_change":"2024-04-17 22:29:07.477449","last_active":"2024-04-17 22:29:07.807837","last_peered":"2024-04-17 22:29:07.807837","last_clean":"2024-04-17 22:29:07.807837","last_became_active":"2024-04-17 22:29:07.477155","last_became_peered":"2024-04-17 22:29:07.477155","last_unstale":"2024-04-17 22:29:07.807837","last_undegraded":"2024-04-17 22:29:07.807837","last_fullsized":"2024-04-17 22:29:07.807837","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.810313","last_change":"2024-04-17 22:29:07.808974","last_active":"2024-04-17 22:29:07.810313","last_peered":"2024-04-17 22:29:07.810313","last_clean":"2024-04-17 22:29:07.810313","last_became_active":"2024-04-17 22:29:07.808747","last_became_peered":"2024-04-17 22:29:07.808747","last_unstale":"2024-04-17 22:29:07.810313","last_undegraded":"2024-04-17 22:29:07.810313","last_fullsized":"2024-04-17 22:29:07.810313","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.24","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.796551","last_change":"2024-04-17 22:29:07.466052","last_active":"2024-04-17 22:29:07.796551","last_peered":"2024-04-17 22:29:07.796551","last_clean":"2024-04-17 22:29:07.796551","last_became_active":"2024-04-17 22:29:07.465576","last_became_peered":"2024-04-17 22:29:07.465576","last_unstale":"2024-04-17 22:29:07.796551","last_undegraded":"2024-04-17 22:29:07.796551","last_fullsized":"2024-04-17 22:29:07.796551","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.833471","last_change":"2024-04-17 22:29:07.829066","last_active":"2024-04-17 22:29:07.833471","last_peered":"2024-04-17 22:29:07.833471","last_clean":"2024-04-17 22:29:07.833471","last_became_active":"2024-04-17 22:29:07.828735","last_became_peered":"2024-04-17 22:29:07.828735","last_unstale":"2024-04-17 22:29:07.833471","last_undegraded":"2024-04-17 22:29:07.833471","last_fullsized":"2024-04-17 22:29:07.833471","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.25","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.795566","last_change":"2024-04-17 22:29:07.462011","last_active":"2024-04-17 22:29:07.795566","last_peered":"2024-04-17 22:29:07.795566","last_clean":"2024-04-17 22:29:07.795566","last_became_active":"2024-04-17 22:29:07.461693","last_became_peered":"2024-04-17 22:29:07.461693","last_unstale":"2024-04-17 22:29:07.795566","last_undegraded":"2024-04-17 22:29:07.795566","last_fullsized":"2024-04-17 22:29:07.795566","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.828447","last_change":"2024-04-17 22:29:07.826655","last_active":"2024-04-17 22:29:07.828447","last_peered":"2024-04-17 22:29:07.828447","last_clean":"2024-04-17 22:29:07.828447","last_became_active":"2024-04-17 22:29:07.823665","last_became_peered":"2024-04-17 22:29:07.823665","last_unstale":"2024-04-17 22:29:07.828447","last_undegraded":"2024-04-17 22:29:07.828447","last_fullsized":"2024-04-17 22:29:07.828447","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.26","version":"16'1","reported_seq":"11","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:08.802774","last_change":"2024-04-17 22:29:07.480854","last_active":"2024-04-17 22:29:08.802774","last_peered":"2024-04-17 22:29:08.802774","last_clean":"2024-04-17 22:29:08.802774","last_became_active":"2024-04-17 22:29:07.479975","last_became_peered":"2024-04-17 22:29:07.479975","last_unstale":"2024-04-17 22:29:08.802774","last_undegraded":"2024-04-17 22:29:08.802774","last_fullsized":"2024-04-17 22:29:08.802774","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.825070","last_change":"2024-04-17 22:29:07.813744","last_active":"2024-04-17 22:29:07.825070","last_peered":"2024-04-17 22:29:07.825070","last_clean":"2024-04-17 22:29:07.825070","last_became_active":"2024-04-17 22:29:07.810646","last_became_peered":"2024-04-17 22:29:07.810646","last_unstale":"2024-04-17 22:29:07.825070","last_undegraded":"2024-04-17 22:29:07.825070","last_fullsized":"2024-04-17 22:29:07.825070","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.27","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.815316","last_change":"2024-04-17 22:29:07.476428","last_active":"2024-04-17 22:29:07.815316","last_peered":"2024-04-17 22:29:07.815316","last_clean":"2024-04-17 22:29:07.815316","last_became_active":"2024-04-17 22:29:07.476128","last_became_peered":"2024-04-17 22:29:07.476128","last_unstale":"2024-04-17 22:29:07.815316","last_undegraded":"2024-04-17 22:29:07.815316","last_fullsized":"2024-04-17 22:29:07.815316","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.826418","last_change":"2024-04-17 22:29:07.823340","last_active":"2024-04-17 22:29:07.826418","last_peered":"2024-04-17 22:29:07.826418","last_clean":"2024-04-17 22:29:07.826418","last_became_active":"2024-04-17 22:29:07.822764","last_became_peered":"2024-04-17 22:29:07.822764","last_unstale":"2024-04-17 22:29:07.826418","last_undegraded":"2024-04-17 22:29:07.826418","last_fullsized":"2024-04-17 22:29:07.826418","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.28","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.798818","last_change":"2024-04-17 22:29:07.477137","last_active":"2024-04-17 22:29:07.798818","last_peered":"2024-04-17 22:29:07.798818","last_clean":"2024-04-17 22:29:07.798818","last_became_active":"2024-04-17 22:29:07.476796","last_became_peered":"2024-04-17 22:29:07.476796","last_unstale":"2024-04-17 22:29:07.798818","last_undegraded":"2024-04-17 22:29:07.798818","last_fullsized":"2024-04-17 22:29:07.798818","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.826239","last_change":"2024-04-17 22:29:07.823119","last_active":"2024-04-17 22:29:07.826239","last_peered":"2024-04-17 22:29:07.826239","last_clean":"2024-04-17 22:29:07.826239","last_became_active":"2024-04-17 22:29:07.822640","last_became_peered":"2024-04-17 22:29:07.822640","last_unstale":"2024-04-17 22:29:07.826239","last_undegraded":"2024-04-17 22:29:07.826239","last_fullsized":"2024-04-17 22:29:07.826239","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.29","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.792382","last_change":"2024-04-17 22:29:07.473467","last_active":"2024-04-17 22:29:07.792382","last_peered":"2024-04-17 22:29:07.792382","last_clean":"2024-04-17 22:29:07.792382","last_became_active":"2024-04-17 22:29:07.473028","last_became_peered":"2024-04-17 22:29:07.473028","last_unstale":"2024-04-17 22:29:07.792382","last_undegraded":"2024-04-17 22:29:07.792382","last_fullsized":"2024-04-17 22:29:07.792382","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.827332","last_change":"2024-04-17 22:29:07.817097","last_active":"2024-04-17 22:29:07.827332","last_peered":"2024-04-17 22:29:07.827332","last_clean":"2024-04-17 22:29:07.827332","last_became_active":"2024-04-17 22:29:07.816813","last_became_peered":"2024-04-17 22:29:07.816813","last_unstale":"2024-04-17 22:29:07.827332","last_undegraded":"2024-04-17 22:29:07.827332","last_fullsized":"2024-04-17 22:29:07.827332","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2a","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.796331","last_change":"2024-04-17 22:29:07.464330","last_active":"2024-04-17 22:29:07.796331","last_peered":"2024-04-17 22:29:07.796331","last_clean":"2024-04-17 22:29:07.796331","last_became_active":"2024-04-17 22:29:07.464002","last_became_peered":"2024-04-17 22:29:07.464002","last_unstale":"2024-04-17 22:29:07.796331","last_undegraded":"2024-04-17 22:29:07.796331","last_fullsized":"2024-04-17 22:29:07.796331","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.830305","last_change":"2024-04-17 22:29:07.828690","last_active":"2024-04-17 22:29:07.830305","last_peered":"2024-04-17 22:29:07.830305","last_clean":"2024-04-17 22:29:07.830305","last_became_active":"2024-04-17 22:29:07.828407","last_became_peered":"2024-04-17 22:29:07.828407","last_unstale":"2024-04-17 22:29:07.830305","last_undegraded":"2024-04-17 22:29:07.830305","last_fullsized":"2024-04-17 22:29:07.830305","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2b","version":"15'1","reported_seq":"11","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.810554","last_change":"2024-04-17 22:29:07.471300","last_active":"2024-04-17 22:29:07.810554","last_peered":"2024-04-17 22:29:07.810554","last_clean":"2024-04-17 22:29:07.810554","last_became_active":"2024-04-17 22:29:07.470886","last_became_peered":"2024-04-17 22:29:07.470886","last_unstale":"2024-04-17 22:29:07.810554","last_undegraded":"2024-04-17 22:29:07.810554","last_fullsized":"2024-04-17 22:29:07.810554","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.828614","last_change":"2024-04-17 22:29:07.826555","last_active":"2024-04-17 22:29:07.828614","last_peered":"2024-04-17 22:29:07.828614","last_clean":"2024-04-17 22:29:07.828614","last_became_active":"2024-04-17 22:29:07.826262","last_became_peered":"2024-04-17 22:29:07.826262","last_unstale":"2024-04-17 22:29:07.828614","last_undegraded":"2024-04-17 22:29:07.828614","last_fullsized":"2024-04-17 22:29:07.828614","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2c","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.797039","last_change":"2024-04-17 22:29:07.462176","last_active":"2024-04-17 22:29:07.797039","last_peered":"2024-04-17 22:29:07.797039","last_clean":"2024-04-17 22:29:07.797039","last_became_active":"2024-04-17 22:29:07.461594","last_became_peered":"2024-04-17 22:29:07.461594","last_unstale":"2024-04-17 22:29:07.797039","last_undegraded":"2024-04-17 22:29:07.797039","last_fullsized":"2024-04-17 22:29:07.797039","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.828130","last_change":"2024-04-17 22:29:07.816896","last_active":"2024-04-17 22:29:07.828130","last_peered":"2024-04-17 22:29:07.828130","last_clean":"2024-04-17 22:29:07.828130","last_became_active":"2024-04-17 22:29:07.816538","last_became_peered":"2024-04-17 22:29:07.816538","last_unstale":"2024-04-17 22:29:07.828130","last_undegraded":"2024-04-17 22:29:07.828130","last_fullsized":"2024-04-17 22:29:07.828130","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2d","version":"15'1","reported_seq":"11","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.827538","last_change":"2024-04-17 22:29:07.479214","last_active":"2024-04-17 22:29:07.827538","last_peered":"2024-04-17 22:29:07.827538","last_clean":"2024-04-17 22:29:07.827538","last_became_active":"2024-04-17 22:29:07.478790","last_became_peered":"2024-04-17 22:29:07.478790","last_unstale":"2024-04-17 22:29:07.827538","last_undegraded":"2024-04-17 22:29:07.827538","last_fullsized":"2024-04-17 22:29:07.827538","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.833543","last_change":"2024-04-17 22:29:07.832349","last_active":"2024-04-17 22:29:07.833543","last_peered":"2024-04-17 22:29:07.833543","last_clean":"2024-04-17 22:29:07.833543","last_became_active":"2024-04-17 22:29:07.831999","last_became_peered":"2024-04-17 22:29:07.831999","last_unstale":"2024-04-17 22:29:07.833543","last_undegraded":"2024-04-17 22:29:07.833543","last_fullsized":"2024-04-17 22:29:07.833543","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2e","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.797059","last_change":"2024-04-17 22:29:07.483244","last_active":"2024-04-17 22:29:07.797059","last_peered":"2024-04-17 22:29:07.797059","last_clean":"2024-04-17 22:29:07.797059","last_became_active":"2024-04-17 22:29:07.482893","last_became_peered":"2024-04-17 22:29:07.482893","last_unstale":"2024-04-17 22:29:07.797059","last_undegraded":"2024-04-17 22:29:07.797059","last_fullsized":"2024-04-17 22:29:07.797059","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.825028","last_change":"2024-04-17 22:29:07.806512","last_active":"2024-04-17 22:29:07.825028","last_peered":"2024-04-17 22:29:07.825028","last_clean":"2024-04-17 22:29:07.825028","last_became_active":"2024-04-17 22:29:07.806182","last_became_peered":"2024-04-17 22:29:07.806182","last_unstale":"2024-04-17 22:29:07.825028","last_undegraded":"2024-04-17 22:29:07.825028","last_fullsized":"2024-04-17 22:29:07.825028","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2f","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.815418","last_change":"2024-04-17 22:29:07.478655","last_active":"2024-04-17 22:29:07.815418","last_peered":"2024-04-17 22:29:07.815418","last_clean":"2024-04-17 22:29:07.815418","last_became_active":"2024-04-17 22:29:07.478346","last_became_peered":"2024-04-17 22:29:07.478346","last_unstale":"2024-04-17 22:29:07.815418","last_undegraded":"2024-04-17 22:29:07.815418","last_fullsized":"2024-04-17 22:29:07.815418","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.826848","last_change":"2024-04-17 22:29:07.825658","last_active":"2024-04-17 22:29:07.826848","last_peered":"2024-04-17 22:29:07.826848","last_clean":"2024-04-17 22:29:07.826848","last_became_active":"2024-04-17 22:29:07.825349","last_became_peered":"2024-04-17 22:29:07.825349","last_unstale":"2024-04-17 22:29:07.826848","last_undegraded":"2024-04-17 22:29:07.826848","last_fullsized":"2024-04-17 22:29:07.826848","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.30","version":"15'4","reported_seq":"14","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.824396","last_change":"2024-04-17 22:29:07.475470","last_active":"2024-04-17 22:29:07.824396","last_peered":"2024-04-17 22:29:07.824396","last_clean":"2024-04-17 22:29:07.824396","last_became_active":"2024-04-17 22:29:07.474917","last_became_peered":"2024-04-17 22:29:07.474917","last_unstale":"2024-04-17 22:29:07.824396","last_undegraded":"2024-04-17 22:29:07.824396","last_fullsized":"2024-04-17 22:29:07.824396","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.826262","last_change":"2024-04-17 22:29:07.823624","last_active":"2024-04-17 22:29:07.826262","last_peered":"2024-04-17 22:29:07.826262","last_clean":"2024-04-17 22:29:07.826262","last_became_active":"2024-04-17 22:29:07.822815","last_became_peered":"2024-04-17 22:29:07.822815","last_unstale":"2024-04-17 22:29:07.826262","last_undegraded":"2024-04-17 22:29:07.826262","last_fullsized":"2024-04-17 22:29:07.826262","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.31","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.804183","last_change":"2024-04-17 22:29:07.474318","last_active":"2024-04-17 22:29:07.804183","last_peered":"2024-04-17 22:29:07.804183","last_clean":"2024-04-17 22:29:07.804183","last_became_active":"2024-04-17 22:29:07.473754","last_became_peered":"2024-04-17 22:29:07.473754","last_unstale":"2024-04-17 22:29:07.804183","last_undegraded":"2024-04-17 22:29:07.804183","last_fullsized":"2024-04-17 22:29:07.804183","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.833495","last_change":"2024-04-17 22:29:07.829098","last_active":"2024-04-17 22:29:07.833495","last_peered":"2024-04-17 22:29:07.833495","last_clean":"2024-04-17 22:29:07.833495","last_became_active":"2024-04-17 22:29:07.828773","last_became_peered":"2024-04-17 22:29:07.828773","last_unstale":"2024-04-17 22:29:07.833495","last_undegraded":"2024-04-17 22:29:07.833495","last_fullsized":"2024-04-17 22:29:07.833495","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.32","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.795411","last_change":"2024-04-17 22:29:07.470666","last_active":"2024-04-17 22:29:07.795411","last_peered":"2024-04-17 22:29:07.795411","last_clean":"2024-04-17 22:29:07.795411","last_became_active":"2024-04-17 22:29:07.470363","last_became_peered":"2024-04-17 22:29:07.470363","last_unstale":"2024-04-17 22:29:07.795411","last_undegraded":"2024-04-17 22:29:07.795411","last_fullsized":"2024-04-17 22:29:07.795411","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.830343","last_change":"2024-04-17 22:29:07.827580","last_active":"2024-04-17 22:29:07.830343","last_peered":"2024-04-17 22:29:07.830343","last_clean":"2024-04-17 22:29:07.830343","last_became_active":"2024-04-17 22:29:07.827233","last_became_peered":"2024-04-17 22:29:07.827233","last_unstale":"2024-04-17 22:29:07.830343","last_undegraded":"2024-04-17 22:29:07.830343","last_fullsized":"2024-04-17 22:29:07.830343","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.33","version":"15'1","reported_seq":"11","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.811613","last_change":"2024-04-17 22:29:07.460978","last_active":"2024-04-17 22:29:07.811613","last_peered":"2024-04-17 22:29:07.811613","last_clean":"2024-04-17 22:29:07.811613","last_became_active":"2024-04-17 22:29:07.460659","last_became_peered":"2024-04-17 22:29:07.460659","last_unstale":"2024-04-17 22:29:07.811613","last_undegraded":"2024-04-17 22:29:07.811613","last_fullsized":"2024-04-17 22:29:07.811613","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.832128","last_change":"2024-04-17 22:29:07.816335","last_active":"2024-04-17 22:29:07.832128","last_peered":"2024-04-17 22:29:07.832128","last_clean":"2024-04-17 22:29:07.832128","last_became_active":"2024-04-17 22:29:07.816090","last_became_peered":"2024-04-17 22:29:07.816090","last_unstale":"2024-04-17 22:29:07.832128","last_undegraded":"2024-04-17 22:29:07.832128","last_fullsized":"2024-04-17 22:29:07.832128","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.34","version":"15'4","reported_seq":"14","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.835387","last_change":"2024-04-17 22:29:07.475702","last_active":"2024-04-17 22:29:07.835387","last_peered":"2024-04-17 22:29:07.835387","last_clean":"2024-04-17 22:29:07.835387","last_became_active":"2024-04-17 22:29:07.475395","last_became_peered":"2024-04-17 22:29:07.475395","last_unstale":"2024-04-17 22:29:07.835387","last_undegraded":"2024-04-17 22:29:07.835387","last_fullsized":"2024-04-17 22:29:07.835387","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":892,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.828199","last_change":"2024-04-17 22:29:07.818538","last_active":"2024-04-17 22:29:07.828199","last_peered":"2024-04-17 22:29:07.828199","last_clean":"2024-04-17 22:29:07.828199","last_became_active":"2024-04-17 22:29:07.817164","last_became_peered":"2024-04-17 22:29:07.817164","last_unstale":"2024-04-17 22:29:07.828199","last_undegraded":"2024-04-17 22:29:07.828199","last_fullsized":"2024-04-17 22:29:07.828199","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.35","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.797071","last_change":"2024-04-17 22:29:07.467651","last_active":"2024-04-17 22:29:07.797071","last_peered":"2024-04-17 22:29:07.797071","last_clean":"2024-04-17 22:29:07.797071","last_became_active":"2024-04-17 22:29:07.467327","last_became_peered":"2024-04-17 22:29:07.467327","last_unstale":"2024-04-17 22:29:07.797071","last_undegraded":"2024-04-17 22:29:07.797071","last_fullsized":"2024-04-17 22:29:07.797071","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.810303","last_change":"2024-04-17 22:29:07.809693","last_active":"2024-04-17 22:29:07.810303","last_peered":"2024-04-17 22:29:07.810303","last_clean":"2024-04-17 22:29:07.810303","last_became_active":"2024-04-17 22:29:07.809415","last_became_peered":"2024-04-17 22:29:07.809415","last_unstale":"2024-04-17 22:29:07.810303","last_undegraded":"2024-04-17 22:29:07.810303","last_fullsized":"2024-04-17 22:29:07.810303","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.36","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.796841","last_change":"2024-04-17 22:29:07.483402","last_active":"2024-04-17 22:29:07.796841","last_peered":"2024-04-17 22:29:07.796841","last_clean":"2024-04-17 22:29:07.796841","last_became_active":"2024-04-17 22:29:07.482914","last_became_peered":"2024-04-17 22:29:07.482914","last_unstale":"2024-04-17 22:29:07.796841","last_undegraded":"2024-04-17 22:29:07.796841","last_fullsized":"2024-04-17 22:29:07.796841","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.830317","last_change":"2024-04-17 22:29:07.827623","last_active":"2024-04-17 22:29:07.830317","last_peered":"2024-04-17 22:29:07.830317","last_clean":"2024-04-17 22:29:07.830317","last_became_active":"2024-04-17 22:29:07.827337","last_became_peered":"2024-04-17 22:29:07.827337","last_unstale":"2024-04-17 22:29:07.830317","last_undegraded":"2024-04-17 22:29:07.830317","last_fullsized":"2024-04-17 22:29:07.830317","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.37","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.808508","last_change":"2024-04-17 22:29:07.463449","last_active":"2024-04-17 22:29:07.808508","last_peered":"2024-04-17 22:29:07.808508","last_clean":"2024-04-17 22:29:07.808508","last_became_active":"2024-04-17 22:29:07.463110","last_became_peered":"2024-04-17 22:29:07.463110","last_unstale":"2024-04-17 22:29:07.808508","last_undegraded":"2024-04-17 22:29:07.808508","last_fullsized":"2024-04-17 22:29:07.808508","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.827399","last_change":"2024-04-17 22:29:07.815214","last_active":"2024-04-17 22:29:07.827399","last_peered":"2024-04-17 22:29:07.827399","last_clean":"2024-04-17 22:29:07.827399","last_became_active":"2024-04-17 22:29:07.814938","last_became_peered":"2024-04-17 22:29:07.814938","last_unstale":"2024-04-17 22:29:07.827399","last_undegraded":"2024-04-17 22:29:07.827399","last_fullsized":"2024-04-17 22:29:07.827399","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.38","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.801328","last_change":"2024-04-17 22:29:07.474682","last_active":"2024-04-17 22:29:07.801328","last_peered":"2024-04-17 22:29:07.801328","last_clean":"2024-04-17 22:29:07.801328","last_became_active":"2024-04-17 22:29:07.474305","last_became_peered":"2024-04-17 22:29:07.474305","last_unstale":"2024-04-17 22:29:07.801328","last_undegraded":"2024-04-17 22:29:07.801328","last_fullsized":"2024-04-17 22:29:07.801328","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.833512","last_change":"2024-04-17 22:29:07.829769","last_active":"2024-04-17 22:29:07.833512","last_peered":"2024-04-17 22:29:07.833512","last_clean":"2024-04-17 22:29:07.833512","last_became_active":"2024-04-17 22:29:07.829484","last_became_peered":"2024-04-17 22:29:07.829484","last_unstale":"2024-04-17 22:29:07.833512","last_undegraded":"2024-04-17 22:29:07.833512","last_fullsized":"2024-04-17 22:29:07.833512","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.39","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.794112","last_change":"2024-04-17 22:29:07.458386","last_active":"2024-04-17 22:29:07.794112","last_peered":"2024-04-17 22:29:07.794112","last_clean":"2024-04-17 22:29:07.794112","last_became_active":"2024-04-17 22:29:07.456695","last_became_peered":"2024-04-17 22:29:07.456695","last_unstale":"2024-04-17 22:29:07.794112","last_undegraded":"2024-04-17 22:29:07.794112","last_fullsized":"2024-04-17 22:29:07.794112","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.829903","last_change":"2024-04-17 22:29:07.821490","last_active":"2024-04-17 22:29:07.829903","last_peered":"2024-04-17 22:29:07.829903","last_clean":"2024-04-17 22:29:07.829903","last_became_active":"2024-04-17 22:29:07.821103","last_became_peered":"2024-04-17 22:29:07.821103","last_unstale":"2024-04-17 22:29:07.829903","last_undegraded":"2024-04-17 22:29:07.829903","last_fullsized":"2024-04-17 22:29:07.829903","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3a","version":"15'1","reported_seq":"11","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.819620","last_change":"2024-04-17 22:29:07.472610","last_active":"2024-04-17 22:29:07.819620","last_peered":"2024-04-17 22:29:07.819620","last_clean":"2024-04-17 22:29:07.819620","last_became_active":"2024-04-17 22:29:07.472297","last_became_peered":"2024-04-17 22:29:07.472297","last_unstale":"2024-04-17 22:29:07.819620","last_undegraded":"2024-04-17 22:29:07.819620","last_fullsized":"2024-04-17 22:29:07.819620","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.824029","last_change":"2024-04-17 22:29:07.820738","last_active":"2024-04-17 22:29:07.824029","last_peered":"2024-04-17 22:29:07.824029","last_clean":"2024-04-17 22:29:07.824029","last_became_active":"2024-04-17 22:29:07.820354","last_became_peered":"2024-04-17 22:29:07.820354","last_unstale":"2024-04-17 22:29:07.824029","last_undegraded":"2024-04-17 22:29:07.824029","last_fullsized":"2024-04-17 22:29:07.824029","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3b","version":"15'1","reported_seq":"11","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.824121","last_change":"2024-04-17 22:29:07.473963","last_active":"2024-04-17 22:29:07.824121","last_peered":"2024-04-17 22:29:07.824121","last_clean":"2024-04-17 22:29:07.824121","last_became_active":"2024-04-17 22:29:07.472018","last_became_peered":"2024-04-17 22:29:07.472018","last_unstale":"2024-04-17 22:29:07.824121","last_undegraded":"2024-04-17 22:29:07.824121","last_fullsized":"2024-04-17 22:29:07.824121","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.832188","last_change":"2024-04-17 22:29:07.824919","last_active":"2024-04-17 22:29:07.832188","last_peered":"2024-04-17 22:29:07.832188","last_clean":"2024-04-17 22:29:07.832188","last_became_active":"2024-04-17 22:29:07.824600","last_became_peered":"2024-04-17 22:29:07.824600","last_unstale":"2024-04-17 22:29:07.832188","last_undegraded":"2024-04-17 22:29:07.832188","last_fullsized":"2024-04-17 22:29:07.832188","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3c","version":"15'2","reported_seq":"12","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.825427","last_change":"2024-04-17 22:29:07.475485","last_active":"2024-04-17 22:29:07.825427","last_peered":"2024-04-17 22:29:07.825427","last_clean":"2024-04-17 22:29:07.825427","last_became_active":"2024-04-17 22:29:07.475182","last_became_peered":"2024-04-17 22:29:07.475182","last_unstale":"2024-04-17 22:29:07.825427","last_undegraded":"2024-04-17 22:29:07.825427","last_fullsized":"2024-04-17 22:29:07.825427","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.821750","last_change":"2024-04-17 22:29:07.810455","last_active":"2024-04-17 22:29:07.821750","last_peered":"2024-04-17 22:29:07.821750","last_clean":"2024-04-17 22:29:07.821750","last_became_active":"2024-04-17 22:29:07.810143","last_became_peered":"2024-04-17 22:29:07.810143","last_unstale":"2024-04-17 22:29:07.821750","last_undegraded":"2024-04-17 22:29:07.821750","last_fullsized":"2024-04-17 22:29:07.821750","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3d","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.797015","last_change":"2024-04-17 22:29:07.468187","last_active":"2024-04-17 22:29:07.797015","last_peered":"2024-04-17 22:29:07.797015","last_clean":"2024-04-17 22:29:07.797015","last_became_active":"2024-04-17 22:29:07.467849","last_became_peered":"2024-04-17 22:29:07.467849","last_unstale":"2024-04-17 22:29:07.797015","last_undegraded":"2024-04-17 22:29:07.797015","last_fullsized":"2024-04-17 22:29:07.797015","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.830571","last_change":"2024-04-17 22:29:07.822173","last_active":"2024-04-17 22:29:07.830571","last_peered":"2024-04-17 22:29:07.830571","last_clean":"2024-04-17 22:29:07.830571","last_became_active":"2024-04-17 22:29:07.821875","last_became_peered":"2024-04-17 22:29:07.821875","last_unstale":"2024-04-17 22:29:07.830571","last_undegraded":"2024-04-17 22:29:07.830571","last_fullsized":"2024-04-17 22:29:07.830571","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3e","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.807733","last_change":"2024-04-17 22:29:07.479547","last_active":"2024-04-17 22:29:07.807733","last_peered":"2024-04-17 22:29:07.807733","last_clean":"2024-04-17 22:29:07.807733","last_became_active":"2024-04-17 22:29:07.479004","last_became_peered":"2024-04-17 22:29:07.479004","last_unstale":"2024-04-17 22:29:07.807733","last_undegraded":"2024-04-17 22:29:07.807733","last_fullsized":"2024-04-17 22:29:07.807733","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.823835","last_change":"2024-04-17 22:29:07.819432","last_active":"2024-04-17 22:29:07.823835","last_peered":"2024-04-17 22:29:07.823835","last_clean":"2024-04-17 22:29:07.823835","last_became_active":"2024-04-17 22:29:07.819001","last_became_peered":"2024-04-17 22:29:07.819001","last_unstale":"2024-04-17 22:29:07.823835","last_undegraded":"2024-04-17 22:29:07.823835","last_fullsized":"2024-04-17 22:29:07.823835","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3f","version":"15'3","reported_seq":"13","reported_epoch":"15","state":"active+clean","last_fresh":"2024-04-17 22:29:07.829638","last_change":"2024-04-17 22:29:07.466765","last_active":"2024-04-17 22:29:07.829638","last_peered":"2024-04-17 22:29:07.829638","last_clean":"2024-04-17 22:29:07.829638","last_became_active":"2024-04-17 22:29:07.463177","last_became_peered":"2024-04-17 22:29:07.463177","last_unstale":"2024-04-17 22:29:07.829638","last_undegraded":"2024-04-17 22:29:07.829638","last_fullsized":"2024-04-17 22:29:07.829638","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":544,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":4},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":4},{"poolid":2,"num_pg":64,"stat_sum":{"num_bytes":2286,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":128,"acting":128,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705671,"num_pgs":82,"num_osds":1,"num_per_pool_osds":1,"kb":94371840,"kb_used":1048744,"kb_used_data":160,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323096,"statfs":{"total":96636764160,"available":95562850304,"internally_reserved":1073741824,"allocated":163840,"data_stored":41000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29899999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33200000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40699999999999997}]}]},{"osd":2,"up_from":9,"seq":38654705671,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"kb":94371840,"kb_used":1048732,"kb_used_data":148,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323108,"statfs":{"total":96636764160,"available":95562862592,"internally_reserved":1073741824,"allocated":151552,"data_stored":39984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27500000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.22900000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33500000000000002}]}]},{"osd":0,"up_from":9,"seq":38654705671,"num_pgs":78,"num_osds":1,"num_per_pool_osds":1,"kb":94371840,"kb_used":1048736,"kb_used_data":152,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323104,"statfs":{"total":96636764160,"available":95562858496,"internally_reserved":1073741824,"allocated":155648,"data_stored":40772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36499999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38400000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41199999999999998}]}]},{"osd":1,"up_from":9,"seq":38654705670,"num_pgs":63,"num_osds":1,"num_per_pool_osds":1,"kb":94371840,"kb_used":1048732,"kb_used_data":148,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323108,"statfs":{"total":96636764160,"available":95562862592,"internally_reserved":1073741824,"allocated":151552,"data_stored":39936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.22600000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.25900000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28000000000000003}]}]}]}} 2024-04-17T22:29:18.664 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-17T22:29:18.664 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-17T22:29:18.664 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-04-17T22:29:18.665 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-17T22:29:18.827 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.838 7f9d52ff6700 1 Processor -- start 2024-04-17T22:29:18.827 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.838 7f9d52ff6700 1 -- start start 2024-04-17T22:29:18.827 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.838 7f9d52ff6700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f9d4c10c910 0x7f9d4c1a6300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:18.827 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.838 7f9d52ff6700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f9d4c1a6930 0x7f9d4c1a8d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:18.827 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.838 7f9d52ff6700 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d4c1a1c10 con 0x7f9d4c0f5770 2024-04-17T22:29:18.827 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.838 7f9d52ff6700 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d4c0f93e0 con 0x7f9d4c1a6930 2024-04-17T22:29:18.827 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.838 7f9d52ff6700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d4c10cd60 con 0x7f9d4c10c910 2024-04-17T22:29:18.827 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.838 7f9d4bfff700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f9d4c1a6930 0x7f9d4c1a8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:18.827 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.838 7f9d4affd700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f9d4c10c910 0x7f9d4c1a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:18.827 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.838 7f9d4b7fe700 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f9d4c0f5770 0x7f9d4c10c110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:34740/0 (socket says 172.21.15.102:34740) 2024-04-17T22:29:18.827 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.838 7f9d4affd700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f9d4c10c910 0x7f9d4c1a6300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:40424/0 (socket says 172.21.15.102:40424) 2024-04-17T22:29:18.828 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.838 7f9d4bfff700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f9d4c1a6930 0x7f9d4c1a8d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:40994/0 (socket says 172.21.15.102:40994) 2024-04-17T22:29:18.828 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.838 7f9d4b7fe700 1 -- 172.21.15.102:0/754862894 learned_addr learned my addr 172.21.15.102:0/754862894 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-17T22:29:18.828 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.839 7f9d4bfff700 1 -- 172.21.15.102:0/754862894 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f9d4c10c910 msgr2=0x7f9d4c1a6300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:29:18.828 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.839 7f9d4bfff700 1 --2- 172.21.15.102:0/754862894 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f9d4c10c910 0x7f9d4c1a6300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:18.828 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.839 7f9d4bfff700 1 -- 172.21.15.102:0/754862894 >> v1:172.21.15.102:6789/0 conn(0x7f9d4c0f5770 legacy=0x7f9d4c10c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:18.828 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.839 7f9d4bfff700 1 -- 172.21.15.102:0/754862894 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d4c1aa3d0 con 0x7f9d4c1a6930 2024-04-17T22:29:18.828 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.839 7f9d4a7fc700 1 -- 172.21.15.102:0/754862894 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2436013721 0 0) 0x7f9d4c1a1c10 con 0x7f9d4c0f5770 2024-04-17T22:29:18.828 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.839 7f9d4affd700 1 --2- 172.21.15.102:0/754862894 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f9d4c10c910 0x7f9d4c1a6300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:29:18.828 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.839 7f9d4bfff700 1 --2- 172.21.15.102:0/754862894 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9d4c1a6930 0x7f9d4c1a8d90 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 rx=0x7f9d3c002f70 tx=0x7f9d3c00b9f0).ready entity=mon.0 client_cookie=753377db3737b86a server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:29:18.828 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.839 7f9d4a7fc700 1 -- 172.21.15.102:0/754862894 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9d4c1aa3d0 con 0x7f9d4c1a6930 2024-04-17T22:29:18.828 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.839 7f9d4a7fc700 1 -- 172.21.15.102:0/754862894 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d3c00d5c0 con 0x7f9d4c1a6930 2024-04-17T22:29:18.828 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.839 7f9d4a7fc700 1 -- 172.21.15.102:0/754862894 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9d3c00d9e0 con 0x7f9d4c1a6930 2024-04-17T22:29:18.828 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.839 7f9d52ff6700 1 -- 172.21.15.102:0/754862894 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9d4c1a6930 msgr2=0x7f9d4c1a8d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:18.828 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.839 7f9d52ff6700 1 --2- 172.21.15.102:0/754862894 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9d4c1a6930 0x7f9d4c1a8d90 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 rx=0x7f9d3c002f70 tx=0x7f9d3c00b9f0).stop 2024-04-17T22:29:18.829 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.839 7f9d52ff6700 1 -- 172.21.15.102:0/754862894 shutdown_connections 2024-04-17T22:29:18.829 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.839 7f9d52ff6700 1 --2- 172.21.15.102:0/754862894 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9d4c1a6930 0x7f9d4c1a8d90 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:18.829 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.839 7f9d52ff6700 1 --2- 172.21.15.102:0/754862894 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f9d4c10c910 0x7f9d4c1a6300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:18.829 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.839 7f9d52ff6700 1 -- 172.21.15.102:0/754862894 >> 172.21.15.102:0/754862894 conn(0x7f9d4c1138c0 msgr2=0x7f9d4c0f4f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:29:18.829 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.839 7f9d52ff6700 1 -- 172.21.15.102:0/754862894 shutdown_connections 2024-04-17T22:29:18.829 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.840 7f9d52ff6700 1 -- 172.21.15.102:0/754862894 wait complete. 2024-04-17T22:29:18.851 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.862 7f9d52ff6700 1 Processor -- start 2024-04-17T22:29:18.852 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.862 7f9d52ff6700 1 -- start start 2024-04-17T22:29:18.852 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.863 7f9d52ff6700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f9d4c0ca570 0x7f9d4c1ad100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:18.852 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.863 7f9d52ff6700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9d4c1ad730 0x7f9d4c0f61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:18.852 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.863 7f9d52ff6700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f9d4c1adb80 0x7f9d4c0f6810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:18.852 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.863 7f9d52ff6700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d4c0c4260 con 0x7f9d4c1ad730 2024-04-17T22:29:18.852 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.863 7f9d52ff6700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d4c0ca9c0 con 0x7f9d4c1adb80 2024-04-17T22:29:18.852 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.863 7f9d52ff6700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9d4c0f6e40 con 0x7f9d4c0ca570 2024-04-17T22:29:18.852 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.863 7f9d4affd700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9d4c1ad730 0x7f9d4c0f61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:18.852 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.863 7f9d4bfff700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f9d4c1adb80 0x7f9d4c0f6810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:18.852 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.863 7f9d4affd700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9d4c1ad730 0x7f9d4c0f61e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:41008/0 (socket says 172.21.15.102:41008) 2024-04-17T22:29:18.852 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.863 7f9d4affd700 1 -- 172.21.15.102:0/3174710754 learned_addr learned my addr 172.21.15.102:0/3174710754 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:29:18.852 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.863 7f9d4b7fe700 1 --2- 172.21.15.102:0/3174710754 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f9d4c0ca570 0x7f9d4c1ad100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:18.852 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.863 7f9d4bfff700 1 -- 172.21.15.102:0/3174710754 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f9d4c0ca570 msgr2=0x7f9d4c1ad100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:18.852 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.863 7f9d4bfff700 1 --2- 172.21.15.102:0/3174710754 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f9d4c0ca570 0x7f9d4c1ad100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:18.853 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.863 7f9d4bfff700 1 -- 172.21.15.102:0/3174710754 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9d4c1ad730 msgr2=0x7f9d4c0f61e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:18.853 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.863 7f9d4bfff700 1 --2- 172.21.15.102:0/3174710754 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9d4c1ad730 0x7f9d4c0f61e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:18.853 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.863 7f9d4bfff700 1 -- 172.21.15.102:0/3174710754 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d4c0f7070 con 0x7f9d4c1adb80 2024-04-17T22:29:18.853 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.863 7f9d4affd700 1 --2- 172.21.15.102:0/3174710754 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9d4c1ad730 0x7f9d4c0f61e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:29:18.853 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.863 7f9d4bfff700 1 --2- 172.21.15.102:0/3174710754 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f9d4c1adb80 0x7f9d4c0f6810 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7f9d4c0dab10 tx=0x7f9d3c0068c0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:29:18.853 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.863 7f9d48ff9700 1 -- 172.21.15.102:0/3174710754 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9d3c017040 con 0x7f9d4c1adb80 2024-04-17T22:29:18.853 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.864 7f9d48ff9700 1 -- 172.21.15.102:0/3174710754 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d3c011960 con 0x7f9d4c1adb80 2024-04-17T22:29:18.853 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.864 7f9d52ff6700 1 -- 172.21.15.102:0/3174710754 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d4c1bb220 con 0x7f9d4c1adb80 2024-04-17T22:29:18.853 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.864 7f9d48ff9700 1 -- 172.21.15.102:0/3174710754 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9d3c011d30 con 0x7f9d4c1adb80 2024-04-17T22:29:18.853 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.864 7f9d52ff6700 1 -- 172.21.15.102:0/3174710754 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d4c1bb600 con 0x7f9d4c1adb80 2024-04-17T22:29:18.853 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.864 7f9d48ff9700 1 -- 172.21.15.102:0/3174710754 <== mon.1 v2:172.21.15.102:3301/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7f9d3c026360 con 0x7f9d4c1adb80 2024-04-17T22:29:18.853 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.864 7f9d48ff9700 1 -- 172.21.15.102:0/3174710754 <== mon.1 v2:172.21.15.102:3301/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 3194+0+0 (secure 0 0 0) 0x7f9d3c018070 con 0x7f9d4c1adb80 2024-04-17T22:29:18.853 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.865 7f9d52ff6700 1 -- 172.21.15.102:0/3174710754 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d4c0f0820 con 0x7f9d4c1adb80 2024-04-17T22:29:18.854 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:18.868 7f9d48ff9700 1 -- 172.21.15.102:0/3174710754 <== mon.1 v2:172.21.15.102:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+110455 (secure 0 0 0) 0x7f9d3c011750 con 0x7f9d4c1adb80 2024-04-17T22:29:18.990 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.004 7f9d52ff6700 1 -- 172.21.15.102:0/3174710754 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f9d4c0efef0 con 0x7f9d4c1adb80 2024-04-17T22:29:18.991 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.004 7f9d48ff9700 1 -- 172.21.15.102:0/3174710754 <== mon.1 v2:172.21.15.102:3301/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+34 (secure 0 0 0) 0x7f9d3c00d3b0 con 0x7f9d4c1adb80 2024-04-17T22:29:18.991 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-17T22:29:18.992 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.005 7f9d52ff6700 1 -- 172.21.15.102:0/3174710754 >> v1:172.21.15.102:6816/42360 conn(0x7f9d2c0145a0 legacy=0x7f9d2c016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:18.992 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.005 7f9d52ff6700 1 -- 172.21.15.102:0/3174710754 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f9d4c1adb80 msgr2=0x7f9d4c0f6810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:18.992 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.005 7f9d52ff6700 1 --2- 172.21.15.102:0/3174710754 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f9d4c1adb80 0x7f9d4c0f6810 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7f9d4c0dab10 tx=0x7f9d3c0068c0).stop 2024-04-17T22:29:18.993 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.006 7f9d52ff6700 1 -- 172.21.15.102:0/3174710754 shutdown_connections 2024-04-17T22:29:18.993 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.006 7f9d52ff6700 1 --2- 172.21.15.102:0/3174710754 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f9d4c1adb80 0x7f9d4c0f6810 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:18.993 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.006 7f9d52ff6700 1 --2- 172.21.15.102:0/3174710754 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f9d4c0ca570 0x7f9d4c1ad100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:18.993 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.006 7f9d52ff6700 1 --2- 172.21.15.102:0/3174710754 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9d4c1ad730 0x7f9d4c0f61e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:18.993 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.006 7f9d52ff6700 1 -- 172.21.15.102:0/3174710754 >> 172.21.15.102:0/3174710754 conn(0x7f9d4c0e0e70 msgr2=0x7f9d4c0dab90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:29:18.993 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.006 7f9d52ff6700 1 -- 172.21.15.102:0/3174710754 shutdown_connections 2024-04-17T22:29:18.993 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.006 7f9d52ff6700 1 -- 172.21.15.102:0/3174710754 wait complete. 2024-04-17T22:29:19.003 INFO:teuthology.orchestra.run.smithi102.stdout:{"checks":{},"status":"HEALTH_OK"} 2024-04-17T22:29:19.003 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-04-17T22:29:19.003 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T22:29:19.129 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.140 7f2f7d10e700 1 Processor -- start 2024-04-17T22:29:19.129 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.140 7f2f7d10e700 1 -- start start 2024-04-17T22:29:19.129 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.141 7f2f7d10e700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f2f7810b830 0x7f2f781a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:19.129 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.141 7f2f7d10e700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f2f7810bc80 0x7f2f781a7920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:19.129 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.141 7f2f7d10e700 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f78103b60 con 0x7f2f780f5860 2024-04-17T22:29:19.129 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.141 7f2f7d10e700 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f780fa390 con 0x7f2f7810bc80 2024-04-17T22:29:19.130 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.141 7f2f7d10e700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2f781a0b20 con 0x7f2f7810b830 2024-04-17T22:29:19.130 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.141 7f2f755fb700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f2f7810b830 0x7f2f781a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:19.130 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.141 7f2f765fd700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f2f7810bc80 0x7f2f781a7920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:19.130 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.141 7f2f75dfc700 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f2f780f5860 0x7f2f7810b030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:34748/0 (socket says 172.21.15.102:34748) 2024-04-17T22:29:19.130 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.141 7f2f75dfc700 1 -- 172.21.15.102:0/77844657 learned_addr learned my addr 172.21.15.102:0/77844657 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-17T22:29:19.130 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.141 7f2f755fb700 1 -- 172.21.15.102:0/77844657 >> v2:172.21.15.102:3300/0 conn(0x7f2f7810bc80 msgr2=0x7f2f781a7920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:29:19.130 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.141 7f2f755fb700 1 --2- 172.21.15.102:0/77844657 >> v2:172.21.15.102:3300/0 conn(0x7f2f7810bc80 0x7f2f781a7920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:19.130 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.141 7f2f755fb700 1 -- 172.21.15.102:0/77844657 >> v1:172.21.15.102:6789/0 conn(0x7f2f780f5860 legacy=0x7f2f7810b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:19.130 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.141 7f2f755fb700 1 -- 172.21.15.102:0/77844657 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f781a8f60 con 0x7f2f7810b830 2024-04-17T22:29:19.130 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.141 7f2f74dfa700 1 -- 172.21.15.102:0/77844657 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 158397862 0 0) 0x7f2f6c001110 con 0x7f2f780f5860 2024-04-17T22:29:19.130 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.141 7f2f765fd700 1 --2- 172.21.15.102:0/77844657 >> v2:172.21.15.102:3300/0 conn(0x7f2f7810bc80 0x7f2f781a7920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:29:19.130 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.141 7f2f755fb700 1 --2- 172.21.15.102:0/77844657 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f2f7810b830 0x7f2f781a52e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7f2f68002f70 tx=0x7f2f6800b970).ready entity=mon.2 client_cookie=85644909e05f4d9d server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:29:19.130 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.141 7f2f74dfa700 1 -- 172.21.15.102:0/77844657 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f2f781a8f60 con 0x7f2f7810b830 2024-04-17T22:29:19.130 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.141 7f2f74dfa700 1 -- 172.21.15.102:0/77844657 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f6800d5c0 con 0x7f2f7810b830 2024-04-17T22:29:19.131 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.141 7f2f74dfa700 1 -- 172.21.15.102:0/77844657 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f2f6800d990 con 0x7f2f7810b830 2024-04-17T22:29:19.131 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.141 7f2f7d10e700 1 -- 172.21.15.102:0/77844657 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f2f7810b830 msgr2=0x7f2f781a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:19.131 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.141 7f2f7d10e700 1 --2- 172.21.15.102:0/77844657 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f2f7810b830 0x7f2f781a52e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7f2f68002f70 tx=0x7f2f6800b970).stop 2024-04-17T22:29:19.131 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.142 7f2f7d10e700 1 -- 172.21.15.102:0/77844657 shutdown_connections 2024-04-17T22:29:19.131 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.142 7f2f7d10e700 1 --2- 172.21.15.102:0/77844657 >> v2:172.21.15.102:3300/0 conn(0x7f2f7810bc80 0x7f2f781a7920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:19.131 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.142 7f2f7d10e700 1 --2- 172.21.15.102:0/77844657 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f2f7810b830 0x7f2f781a52e0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:19.131 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.142 7f2f7d10e700 1 -- 172.21.15.102:0/77844657 >> 172.21.15.102:0/77844657 conn(0x7f2f781138c0 msgr2=0x7f2f780f4f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:29:19.131 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.142 7f2f7d10e700 1 -- 172.21.15.102:0/77844657 shutdown_connections 2024-04-17T22:29:19.131 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.142 7f2f7d10e700 1 -- 172.21.15.102:0/77844657 wait complete. 2024-04-17T22:29:19.153 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.165 7f2f7d10e700 1 Processor -- start 2024-04-17T22:29:19.153 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.165 7f2f7d10e700 1 -- start start 2024-04-17T22:29:19.154 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.165 7f2f7d10e700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2f780dd010 0x7f2f781adc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:19.154 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.165 7f2f7d10e700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f2f781ae260 0x7f2f781b46e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:19.154 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.165 7f2f7d10e700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2f781ae6b0 0x7f2f781b4d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:19.154 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.165 7f2f7d10e700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f780e2ba0 con 0x7f2f780dd010 2024-04-17T22:29:19.154 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.165 7f2f7d10e700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f780dc510 con 0x7f2f781ae6b0 2024-04-17T22:29:19.154 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.165 7f2f7d10e700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2f780dd460 con 0x7f2f781ae260 2024-04-17T22:29:19.154 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.165 7f2f75dfc700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2f780dd010 0x7f2f781adc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:19.154 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.165 7f2f75dfc700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2f780dd010 0x7f2f781adc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:41032/0 (socket says 172.21.15.102:41032) 2024-04-17T22:29:19.154 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.165 7f2f755fb700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f2f781ae260 0x7f2f781b46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:19.154 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.165 7f2f75dfc700 1 -- 172.21.15.102:0/2863022952 learned_addr learned my addr 172.21.15.102:0/2863022952 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:29:19.154 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.165 7f2f765fd700 1 --2- 172.21.15.102:0/2863022952 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2f781ae6b0 0x7f2f781b4d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:19.154 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.165 7f2f75dfc700 1 -- 172.21.15.102:0/2863022952 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f2f781ae260 msgr2=0x7f2f781b46e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:19.155 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.165 7f2f75dfc700 1 --2- 172.21.15.102:0/2863022952 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f2f781ae260 0x7f2f781b46e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:19.155 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.165 7f2f75dfc700 1 -- 172.21.15.102:0/2863022952 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2f781ae6b0 msgr2=0x7f2f781b4d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:19.155 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.165 7f2f75dfc700 1 --2- 172.21.15.102:0/2863022952 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2f781ae6b0 0x7f2f781b4d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:19.155 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.165 7f2f75dfc700 1 -- 172.21.15.102:0/2863022952 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f781b5340 con 0x7f2f780dd010 2024-04-17T22:29:19.155 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.165 7f2f765fd700 1 --2- 172.21.15.102:0/2863022952 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2f781ae6b0 0x7f2f781b4d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:29:19.155 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.165 7f2f755fb700 1 --2- 172.21.15.102:0/2863022952 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f2f781ae260 0x7f2f781b46e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:29:19.155 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.166 7f2f75dfc700 1 --2- 172.21.15.102:0/2863022952 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2f780dd010 0x7f2f781adc30 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 rx=0x7f2f6c000bc0 tx=0x7f2f6c00c830).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:29:19.155 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.166 7f2f66ffd700 1 -- 172.21.15.102:0/2863022952 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f2f781b5340 con 0x7f2f780dd010 2024-04-17T22:29:19.155 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.166 7f2f66ffd700 1 -- 172.21.15.102:0/2863022952 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f6c0072e0 con 0x7f2f780dd010 2024-04-17T22:29:19.155 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.166 7f2f7d10e700 1 -- 172.21.15.102:0/2863022952 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f781b5660 con 0x7f2f780dd010 2024-04-17T22:29:19.155 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.166 7f2f66ffd700 1 -- 172.21.15.102:0/2863022952 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f2f6c005810 con 0x7f2f780dd010 2024-04-17T22:29:19.155 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.166 7f2f7d10e700 1 -- 172.21.15.102:0/2863022952 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f781bc030 con 0x7f2f780dd010 2024-04-17T22:29:19.155 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.166 7f2f66ffd700 1 -- 172.21.15.102:0/2863022952 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7f2f6c005a10 con 0x7f2f780dd010 2024-04-17T22:29:19.155 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.166 7f2f66ffd700 1 -- 172.21.15.102:0/2863022952 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 3194+0+0 (secure 0 0 0) 0x7f2f6c014730 con 0x7f2f780dd010 2024-04-17T22:29:19.156 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.167 7f2f7d10e700 1 -- 172.21.15.102:0/2863022952 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f780f0820 con 0x7f2f780dd010 2024-04-17T22:29:19.156 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.170 7f2f66ffd700 1 -- 172.21.15.102:0/2863022952 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+110455 (secure 0 0 0) 0x7f2f6c0070d0 con 0x7f2f780dd010 2024-04-17T22:29:19.263 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.277 7f2f7d10e700 1 -- 172.21.15.102:0/2863022952 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2f781aee00 con 0x7f2f780dd010 2024-04-17T22:29:19.264 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.277 7f2f66ffd700 1 -- 172.21.15.102:0/2863022952 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+8761 (secure 0 0 0) 0x7f2f6c000c30 con 0x7f2f780dd010 2024-04-17T22:29:19.264 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-17T22:29:19.264 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":16,"fsid":"cb63e789-091d-49b1-b120-55f9db526871","created":"2024-04-17 22:28:48.209635","modified":"2024-04-17 22:29:08.789044","last_up_change":"2024-04-17 22:28:54.236215","last_in_change":"2024-04-17 22:28:54.236215","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":4,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-04-17 22:28:57.478545","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-17 22:29:06.144708","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-17 22:29:06.748924","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3351ef85-ade4-4b90-a8f4-213f4e00ba01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6812","nonce":42955}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6813","nonce":42955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6815","nonce":42955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6814","nonce":42955}]},"public_addr":"172.21.15.102:6812/42955","cluster_addr":"172.21.15.102:6813/42955","heartbeat_back_addr":"172.21.15.102:6815/42955","heartbeat_front_addr":"172.21.15.102:6814/42955","state":["exists","up"]},{"osd":1,"uuid":"04f67eac-f36b-48f5-982d-4cfc776596d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6808","nonce":42958}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6809","nonce":42958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6811","nonce":42958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6810","nonce":42958}]},"public_addr":"172.21.15.102:6808/42958","cluster_addr":"172.21.15.102:6809/42958","heartbeat_back_addr":"172.21.15.102:6811/42958","heartbeat_front_addr":"172.21.15.102:6810/42958","state":["exists","up"]},{"osd":2,"uuid":"d524204b-70b2-40a8-8f73-f9e7d44318af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6804","nonce":42954}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6805","nonce":42954}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6807","nonce":42954}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6806","nonce":42954}]},"public_addr":"172.21.15.102:6804/42954","cluster_addr":"172.21.15.102:6805/42954","heartbeat_back_addr":"172.21.15.102:6807/42954","heartbeat_front_addr":"172.21.15.102:6806/42954","state":["exists","up"]},{"osd":3,"uuid":"4fc9d20c-1ad0-484f-aafa-2c9a7574245b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6800","nonce":42953}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6801","nonce":42953}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6803","nonce":42953}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6802","nonce":42953}]},"public_addr":"172.21.15.102:6800/42953","cluster_addr":"172.21.15.102:6801/42953","heartbeat_back_addr":"172.21.15.102:6803/42953","heartbeat_front_addr":"172.21.15.102:6802/42953","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2024-04-17T22:29:19.265 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.279 7f2f7d10e700 1 -- 172.21.15.102:0/2863022952 >> v1:172.21.15.102:6816/42360 conn(0x7f2f540145a0 legacy=0x7f2f54016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:19.266 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.279 7f2f7d10e700 1 -- 172.21.15.102:0/2863022952 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2f780dd010 msgr2=0x7f2f781adc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:19.266 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.279 7f2f7d10e700 1 --2- 172.21.15.102:0/2863022952 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2f780dd010 0x7f2f781adc30 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 rx=0x7f2f6c000bc0 tx=0x7f2f6c00c830).stop 2024-04-17T22:29:19.266 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.279 7f2f7d10e700 1 -- 172.21.15.102:0/2863022952 shutdown_connections 2024-04-17T22:29:19.266 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.279 7f2f7d10e700 1 --2- 172.21.15.102:0/2863022952 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2f781ae6b0 0x7f2f781b4d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:19.266 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.279 7f2f7d10e700 1 --2- 172.21.15.102:0/2863022952 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2f780dd010 0x7f2f781adc30 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:19.266 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.279 7f2f7d10e700 1 --2- 172.21.15.102:0/2863022952 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f2f781ae260 0x7f2f781b46e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:19.266 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.279 7f2f7d10e700 1 -- 172.21.15.102:0/2863022952 >> 172.21.15.102:0/2863022952 conn(0x7f2f780e0a70 msgr2=0x7f2f780ce8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:29:19.266 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.279 7f2f7d10e700 1 -- 172.21.15.102:0/2863022952 shutdown_connections 2024-04-17T22:29:19.266 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.279 7f2f7d10e700 1 -- 172.21.15.102:0/2863022952 wait complete. 2024-04-17T22:29:19.276 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2024-04-17 22:28:57.478545', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '12', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-17 22:29:06.144708', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-17 22:29:06.748924', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-17T22:29:19.276 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2024-04-17T22:29:19.407 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.418 7f471ab76700 1 Processor -- start 2024-04-17T22:29:19.407 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.418 7f471ab76700 1 -- start start 2024-04-17T22:29:19.407 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.418 7f471ab76700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f471410e240 0x7f471410e690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:19.407 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.418 7f471ab76700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f47140f1f30 0x7f47141ab4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:19.407 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.418 7f471ab76700 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47140f2380 con 0x7f471410ecc0 2024-04-17T22:29:19.407 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.418 7f471ab76700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47140f9f10 con 0x7f471410e240 2024-04-17T22:29:19.407 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.418 7f471ab76700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f47140f9030 con 0x7f47140f1f30 2024-04-17T22:29:19.408 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.418 7f47137fe700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f471410e240 0x7f471410e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:19.408 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.418 7f4713fff700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f47140f1f30 0x7f47141ab4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:19.408 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.418 7f4712ffd700 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f471410ecc0 0x7f47140f1730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:34762/0 (socket says 172.21.15.102:34762) 2024-04-17T22:29:19.408 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.418 7f4712ffd700 1 -- 172.21.15.102:0/3136985234 learned_addr learned my addr 172.21.15.102:0/3136985234 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-17T22:29:19.408 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.419 7f4713fff700 1 -- 172.21.15.102:0/3136985234 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f471410e240 msgr2=0x7f471410e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:29:19.408 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.419 7f4713fff700 1 --2- 172.21.15.102:0/3136985234 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f471410e240 0x7f471410e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:19.408 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.419 7f4713fff700 1 -- 172.21.15.102:0/3136985234 >> v1:172.21.15.102:6789/0 conn(0x7f471410ecc0 legacy=0x7f47140f1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:19.408 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.419 7f47127fc700 1 -- 172.21.15.102:0/3136985234 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 844485385 0 0) 0x7f4704001110 con 0x7f471410ecc0 2024-04-17T22:29:19.408 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.419 7f4713fff700 1 -- 172.21.15.102:0/3136985234 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47140f25a0 con 0x7f47140f1f30 2024-04-17T22:29:19.408 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.419 7f47137fe700 1 --2- 172.21.15.102:0/3136985234 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f471410e240 0x7f471410e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:29:19.408 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.419 7f4713fff700 1 --2- 172.21.15.102:0/3136985234 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f47140f1f30 0x7f47141ab4e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7f47140f4470 tx=0x7f470c009cf0).ready entity=mon.2 client_cookie=f68e17807b1d5baf server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:29:19.409 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.419 7f47127fc700 1 -- 172.21.15.102:0/3136985234 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f47140f25a0 con 0x7f47140f1f30 2024-04-17T22:29:19.409 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.419 7f47127fc700 1 -- 172.21.15.102:0/3136985234 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f470c00b840 con 0x7f47140f1f30 2024-04-17T22:29:19.409 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.419 7f47127fc700 1 -- 172.21.15.102:0/3136985234 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f470c00ba20 con 0x7f47140f1f30 2024-04-17T22:29:19.409 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.419 7f471ab76700 1 -- 172.21.15.102:0/3136985234 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f47140f1f30 msgr2=0x7f47141ab4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:19.409 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.419 7f471ab76700 1 --2- 172.21.15.102:0/3136985234 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f47140f1f30 0x7f47141ab4e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7f47140f4470 tx=0x7f470c009cf0).stop 2024-04-17T22:29:19.409 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.419 7f471ab76700 1 -- 172.21.15.102:0/3136985234 shutdown_connections 2024-04-17T22:29:19.409 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.419 7f471ab76700 1 --2- 172.21.15.102:0/3136985234 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f47140f1f30 0x7f47141ab4e0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:19.409 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.419 7f471ab76700 1 --2- 172.21.15.102:0/3136985234 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f471410e240 0x7f471410e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:19.409 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.419 7f471ab76700 1 -- 172.21.15.102:0/3136985234 >> 172.21.15.102:0/3136985234 conn(0x7f47141138a0 msgr2=0x7f47140f4f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:29:19.409 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.419 7f471ab76700 1 -- 172.21.15.102:0/3136985234 shutdown_connections 2024-04-17T22:29:19.409 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.419 7f471ab76700 1 -- 172.21.15.102:0/3136985234 wait complete. 2024-04-17T22:29:19.429 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.442 7f471ab76700 1 Processor -- start 2024-04-17T22:29:19.429 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.442 7f471ab76700 1 -- start start 2024-04-17T22:29:19.431 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.443 7f471ab76700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f47140ded90 0x7f47140df1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:19.431 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.443 7f471ab76700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f47140df810 0x7f47141b47f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:19.432 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.443 7f471ab76700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f47141b4e20 0x7f47141bbd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:19.432 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.443 7f471ab76700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47140fa1b0 con 0x7f47140df810 2024-04-17T22:29:19.432 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.443 7f471ab76700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47140ca850 con 0x7f47141b4e20 2024-04-17T22:29:19.432 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.443 7f471ab76700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f47140d02f0 con 0x7f47140ded90 2024-04-17T22:29:19.432 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.443 7f47137fe700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f47140ded90 0x7f47140df1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:19.432 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.443 7f4712ffd700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f47140df810 0x7f47141b47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:19.432 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.443 7f47137fe700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f47140ded90 0x7f47140df1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:44408/0 (socket says 172.21.15.102:44408) 2024-04-17T22:29:19.432 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.443 7f47137fe700 1 -- 172.21.15.102:0/3082090903 learned_addr learned my addr 172.21.15.102:0/3082090903 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:29:19.432 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.443 7f4713fff700 1 --2- 172.21.15.102:0/3082090903 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f47141b4e20 0x7f47141bbd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:19.432 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.443 7f47137fe700 1 -- 172.21.15.102:0/3082090903 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f47141b4e20 msgr2=0x7f47141bbd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:19.432 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.443 7f47137fe700 1 --2- 172.21.15.102:0/3082090903 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f47141b4e20 0x7f47141bbd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:19.433 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.443 7f47137fe700 1 -- 172.21.15.102:0/3082090903 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f47140df810 msgr2=0x7f47141b47f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:19.433 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.443 7f47137fe700 1 --2- 172.21.15.102:0/3082090903 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f47140df810 0x7f47141b47f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:19.433 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.443 7f47137fe700 1 -- 172.21.15.102:0/3082090903 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47140ca230 con 0x7f47140ded90 2024-04-17T22:29:19.433 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.443 7f4713fff700 1 --2- 172.21.15.102:0/3082090903 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f47141b4e20 0x7f47141bbd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:29:19.433 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.443 7f4712ffd700 1 --2- 172.21.15.102:0/3082090903 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f47140df810 0x7f47141b47f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:29:19.433 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.443 7f47137fe700 1 --2- 172.21.15.102:0/3082090903 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f47140ded90 0x7f47140df1e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7f47140d2770 tx=0x7f470800bf70).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:29:19.433 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.443 7f4710ff9700 1 -- 172.21.15.102:0/3082090903 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f47140ca230 con 0x7f47140ded90 2024-04-17T22:29:19.433 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.443 7f4710ff9700 1 -- 172.21.15.102:0/3082090903 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f470800ee60 con 0x7f47140ded90 2024-04-17T22:29:19.433 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.443 7f471ab76700 1 -- 172.21.15.102:0/3082090903 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47140d2460 con 0x7f47140ded90 2024-04-17T22:29:19.433 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.443 7f4710ff9700 1 -- 172.21.15.102:0/3082090903 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f470800f040 con 0x7f47140ded90 2024-04-17T22:29:19.433 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.444 7f471ab76700 1 -- 172.21.15.102:0/3082090903 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47141b5270 con 0x7f47140ded90 2024-04-17T22:29:19.433 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.444 7f4710ff9700 1 -- 172.21.15.102:0/3082090903 <== mon.2 v2:172.21.15.102:3302/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7f4708011070 con 0x7f47140ded90 2024-04-17T22:29:19.433 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.444 7f4710ff9700 1 -- 172.21.15.102:0/3082090903 <== mon.2 v2:172.21.15.102:3302/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 3194+0+0 (secure 0 0 0) 0x7f4708010040 con 0x7f47140ded90 2024-04-17T22:29:19.434 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.444 7f471ab76700 1 -- 172.21.15.102:0/3082090903 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47140f0800 con 0x7f47140ded90 2024-04-17T22:29:19.434 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.448 7f4710ff9700 1 -- 172.21.15.102:0/3082090903 <== mon.2 v2:172.21.15.102:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+110455 (secure 0 0 0) 0x7f470800c620 con 0x7f47140ded90 2024-04-17T22:29:19.539 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.552 7f471ab76700 1 -- 172.21.15.102:0/3082090903 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f47140efed0 con 0x7f47140ded90 2024-04-17T22:29:19.539 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.553 7f4710ff9700 1 -- 172.21.15.102:0/3082090903 <== mon.2 v2:172.21.15.102:3302/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v16) v1 ==== 92+0+10 (secure 0 0 0) 0x7f4708000c30 con 0x7f47140ded90 2024-04-17T22:29:19.539 INFO:teuthology.orchestra.run.smithi102.stdout:pg_num: 8 2024-04-17T22:29:19.541 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.554 7f471ab76700 1 -- 172.21.15.102:0/3082090903 >> v1:172.21.15.102:6816/42360 conn(0x7f46f40145a0 legacy=0x7f46f4016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:19.541 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.554 7f471ab76700 1 -- 172.21.15.102:0/3082090903 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f47140ded90 msgr2=0x7f47140df1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:19.541 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.554 7f471ab76700 1 --2- 172.21.15.102:0/3082090903 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f47140ded90 0x7f47140df1e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7f47140d2770 tx=0x7f470800bf70).stop 2024-04-17T22:29:19.541 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.554 7f471ab76700 1 -- 172.21.15.102:0/3082090903 shutdown_connections 2024-04-17T22:29:19.541 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.554 7f471ab76700 1 --2- 172.21.15.102:0/3082090903 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f47141b4e20 0x7f47141bbd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:19.541 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.554 7f471ab76700 1 --2- 172.21.15.102:0/3082090903 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f47140ded90 0x7f47140df1e0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:19.541 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.554 7f471ab76700 1 --2- 172.21.15.102:0/3082090903 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f47140df810 0x7f47141b47f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:19.541 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.554 7f471ab76700 1 -- 172.21.15.102:0/3082090903 >> 172.21.15.102:0/3082090903 conn(0x7f47141a7c50 msgr2=0x7f47140d7520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:29:19.541 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.554 7f471ab76700 1 -- 172.21.15.102:0/3082090903 shutdown_connections 2024-04-17T22:29:19.541 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.554 7f471ab76700 1 -- 172.21.15.102:0/3082090903 wait complete. 2024-04-17T22:29:19.551 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-17T22:29:19.680 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.691 7f5e66f1e700 1 Processor -- start 2024-04-17T22:29:19.680 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.691 7f5e66f1e700 1 -- start start 2024-04-17T22:29:19.680 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.691 7f5e66f1e700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5e6010e240 0x7f5e6010e690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:19.681 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.691 7f5e66f1e700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5e600f1f30 0x7f5e601ab4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:19.681 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.692 7f5e66f1e700 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e600f2380 con 0x7f5e6010ecc0 2024-04-17T22:29:19.681 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.692 7f5e66f1e700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e600f9f10 con 0x7f5e600f1f30 2024-04-17T22:29:19.681 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.692 7f5e66f1e700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5e600f9030 con 0x7f5e6010e240 2024-04-17T22:29:19.681 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.692 7f5e5ffff700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5e600f1f30 0x7f5e601ab4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:19.681 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.692 7f5e5f7fe700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5e6010e240 0x7f5e6010e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:19.681 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.692 7f5e5ffff700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5e600f1f30 0x7f5e601ab4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:40464/0 (socket says 172.21.15.102:40464) 2024-04-17T22:29:19.681 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.692 7f5e5f7fe700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5e6010e240 0x7f5e6010e690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:44416/0 (socket says 172.21.15.102:44416) 2024-04-17T22:29:19.681 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.692 7f5e5ffff700 1 -- 172.21.15.102:0/404294141 learned_addr learned my addr 172.21.15.102:0/404294141 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:29:19.681 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.692 7f5e5effd700 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f5e6010ecc0 0x7f5e600f1730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:34768/0 (socket says 172.21.15.102:34768) 2024-04-17T22:29:19.681 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.692 7f5e5e7fc700 1 -- 172.21.15.102:0/404294141 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 592107483 0 0) 0x7f5e50001110 con 0x7f5e6010ecc0 2024-04-17T22:29:19.681 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.692 7f5e5e7fc700 1 -- 172.21.15.102:0/404294141 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5e40003270 con 0x7f5e6010ecc0 2024-04-17T22:29:19.681 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.692 7f5e5ffff700 1 -- 172.21.15.102:0/404294141 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5e6010e240 msgr2=0x7f5e6010e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:29:19.681 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.692 7f5e5ffff700 1 --2- 172.21.15.102:0/404294141 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5e6010e240 0x7f5e6010e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:19.682 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.692 7f5e5ffff700 1 -- 172.21.15.102:0/404294141 >> v1:172.21.15.102:6789/0 conn(0x7f5e6010ecc0 legacy=0x7f5e600f1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:19.682 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.692 7f5e5ffff700 1 -- 172.21.15.102:0/404294141 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e600f25a0 con 0x7f5e600f1f30 2024-04-17T22:29:19.682 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.692 7f5e5f7fe700 1 --2- 172.21.15.102:0/404294141 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5e6010e240 0x7f5e6010e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:29:19.682 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.692 7f5e5ffff700 1 --2- 172.21.15.102:0/404294141 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5e600f1f30 0x7f5e601ab4e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7f5e600f46a0 tx=0x7f5e58009e40).ready entity=mon.1 client_cookie=e67d2dd0aea4ae54 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:29:19.682 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.692 7f5e5e7fc700 1 -- 172.21.15.102:0/404294141 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f5e600f25a0 con 0x7f5e600f1f30 2024-04-17T22:29:19.682 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.692 7f5e5e7fc700 1 -- 172.21.15.102:0/404294141 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e5800b870 con 0x7f5e600f1f30 2024-04-17T22:29:19.682 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.692 7f5e5e7fc700 1 -- 172.21.15.102:0/404294141 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f5e5800ba50 con 0x7f5e600f1f30 2024-04-17T22:29:19.682 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.692 7f5e66f1e700 1 -- 172.21.15.102:0/404294141 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5e600f1f30 msgr2=0x7f5e601ab4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:19.682 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.692 7f5e66f1e700 1 --2- 172.21.15.102:0/404294141 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5e600f1f30 0x7f5e601ab4e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7f5e600f46a0 tx=0x7f5e58009e40).stop 2024-04-17T22:29:19.682 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.693 7f5e66f1e700 1 -- 172.21.15.102:0/404294141 shutdown_connections 2024-04-17T22:29:19.682 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.693 7f5e66f1e700 1 --2- 172.21.15.102:0/404294141 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5e600f1f30 0x7f5e601ab4e0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:19.682 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.693 7f5e66f1e700 1 --2- 172.21.15.102:0/404294141 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5e6010e240 0x7f5e6010e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:19.682 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.693 7f5e66f1e700 1 -- 172.21.15.102:0/404294141 >> 172.21.15.102:0/404294141 conn(0x7f5e601138a0 msgr2=0x7f5e600f4f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:29:19.682 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.693 7f5e66f1e700 1 -- 172.21.15.102:0/404294141 shutdown_connections 2024-04-17T22:29:19.683 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.693 7f5e66f1e700 1 -- 172.21.15.102:0/404294141 wait complete. 2024-04-17T22:29:19.704 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.715 7f5e66f1e700 1 Processor -- start 2024-04-17T22:29:19.704 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.716 7f5e66f1e700 1 -- start start 2024-04-17T22:29:19.704 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.716 7f5e66f1e700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5e601b0ee0 0x7f5e601b1330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:19.704 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.716 7f5e66f1e700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5e601b1960 0x7f5e601b4420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:19.704 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.716 7f5e66f1e700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5e601b4a50 0x7f5e601b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:19.704 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.716 7f5e66f1e700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e600fa1b0 con 0x7f5e601b1960 2024-04-17T22:29:19.704 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.716 7f5e66f1e700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e600db350 con 0x7f5e601b0ee0 2024-04-17T22:29:19.704 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.716 7f5e66f1e700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5e600d5250 con 0x7f5e601b4a50 2024-04-17T22:29:19.704 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.716 7f5e5f7fe700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5e601b0ee0 0x7f5e601b1330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:19.704 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.716 7f5e5ffff700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5e601b4a50 0x7f5e601b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:19.704 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.716 7f5e5effd700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5e601b1960 0x7f5e601b4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:19.705 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.716 7f5e5f7fe700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5e601b0ee0 0x7f5e601b1330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:40480/0 (socket says 172.21.15.102:40480) 2024-04-17T22:29:19.705 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.716 7f5e5f7fe700 1 -- 172.21.15.102:0/248712909 learned_addr learned my addr 172.21.15.102:0/248712909 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:29:19.705 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.716 7f5e5f7fe700 1 -- 172.21.15.102:0/248712909 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5e601b4a50 msgr2=0x7f5e601b4ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:19.705 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.716 7f5e5f7fe700 1 --2- 172.21.15.102:0/248712909 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5e601b4a50 0x7f5e601b4ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:19.705 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.716 7f5e5f7fe700 1 -- 172.21.15.102:0/248712909 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5e601b1960 msgr2=0x7f5e601b4420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:19.705 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.716 7f5e5f7fe700 1 --2- 172.21.15.102:0/248712909 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5e601b1960 0x7f5e601b4420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:19.705 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.716 7f5e5f7fe700 1 -- 172.21.15.102:0/248712909 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e600d5530 con 0x7f5e601b0ee0 2024-04-17T22:29:19.705 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.716 7f5e5effd700 1 --2- 172.21.15.102:0/248712909 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5e601b1960 0x7f5e601b4420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:29:19.705 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.717 7f5e5f7fe700 1 --2- 172.21.15.102:0/248712909 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5e601b0ee0 0x7f5e601b1330 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7f5e4800c000 tx=0x7f5e4800bf70).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:29:19.705 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.717 7f5e5cff9700 1 -- 172.21.15.102:0/248712909 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f5e600d5530 con 0x7f5e601b0ee0 2024-04-17T22:29:19.705 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.717 7f5e5cff9700 1 -- 172.21.15.102:0/248712909 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e4800c9a0 con 0x7f5e601b0ee0 2024-04-17T22:29:19.705 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.717 7f5e66f1e700 1 -- 172.21.15.102:0/248712909 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e601b1db0 con 0x7f5e601b0ee0 2024-04-17T22:29:19.705 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.717 7f5e5cff9700 1 -- 172.21.15.102:0/248712909 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f5e4800f040 con 0x7f5e601b0ee0 2024-04-17T22:29:19.705 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.717 7f5e66f1e700 1 -- 172.21.15.102:0/248712909 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e601bba80 con 0x7f5e601b0ee0 2024-04-17T22:29:19.705 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.717 7f5e5cff9700 1 -- 172.21.15.102:0/248712909 <== mon.1 v2:172.21.15.102:3301/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7f5e48011070 con 0x7f5e601b0ee0 2024-04-17T22:29:19.706 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.717 7f5e5cff9700 1 -- 172.21.15.102:0/248712909 <== mon.1 v2:172.21.15.102:3301/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 3194+0+0 (secure 0 0 0) 0x7f5e48010040 con 0x7f5e601b0ee0 2024-04-17T22:29:19.706 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.718 7f5e66f1e700 1 -- 172.21.15.102:0/248712909 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e600f0800 con 0x7f5e601b0ee0 2024-04-17T22:29:19.707 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.721 7f5e5cff9700 1 -- 172.21.15.102:0/248712909 <== mon.1 v2:172.21.15.102:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+110455 (secure 0 0 0) 0x7f5e4800c5a0 con 0x7f5e601b0ee0 2024-04-17T22:29:19.812 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.826 7f5e66f1e700 1 -- 172.21.15.102:0/248712909 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f5e600efed0 con 0x7f5e601b0ee0 2024-04-17T22:29:19.813 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.826 7f5e5cff9700 1 -- 172.21.15.102:0/248712909 <== mon.1 v2:172.21.15.102:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v16) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5e48000c30 con 0x7f5e601b0ee0 2024-04-17T22:29:19.813 INFO:teuthology.orchestra.run.smithi102.stdout:pg_num: 64 2024-04-17T22:29:19.814 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.827 7f5e66f1e700 1 -- 172.21.15.102:0/248712909 >> v1:172.21.15.102:6816/42360 conn(0x7f5e440145a0 legacy=0x7f5e44016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:19.815 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.827 7f5e66f1e700 1 -- 172.21.15.102:0/248712909 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5e601b0ee0 msgr2=0x7f5e601b1330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:19.815 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.828 7f5e66f1e700 1 --2- 172.21.15.102:0/248712909 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5e601b0ee0 0x7f5e601b1330 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7f5e4800c000 tx=0x7f5e4800bf70).stop 2024-04-17T22:29:19.815 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.828 7f5e66f1e700 1 -- 172.21.15.102:0/248712909 shutdown_connections 2024-04-17T22:29:19.815 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.828 7f5e66f1e700 1 --2- 172.21.15.102:0/248712909 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5e601b4a50 0x7f5e601b4ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:19.815 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.828 7f5e66f1e700 1 --2- 172.21.15.102:0/248712909 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5e601b0ee0 0x7f5e601b1330 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:19.815 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.828 7f5e66f1e700 1 --2- 172.21.15.102:0/248712909 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5e601b1960 0x7f5e601b4420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:19.815 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.828 7f5e66f1e700 1 -- 172.21.15.102:0/248712909 >> 172.21.15.102:0/248712909 conn(0x7f5e600c6e80 msgr2=0x7f5e601a8750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:29:19.815 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.828 7f5e66f1e700 1 -- 172.21.15.102:0/248712909 shutdown_connections 2024-04-17T22:29:19.815 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.828 7f5e66f1e700 1 -- 172.21.15.102:0/248712909 wait complete. 2024-04-17T22:29:19.825 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-17T22:29:19.952 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.965 7fb0b0dda700 1 Processor -- start 2024-04-17T22:29:19.952 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.965 7fb0b0dda700 1 -- start start 2024-04-17T22:29:19.953 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.966 7fb0b0dda700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb0ac19fde0 0x7fb0ac1a0230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:19.953 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.966 7fb0b0dda700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb0ac1a0860 0x7fb0ac1030e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:19.953 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.966 7fb0b0dda700 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0ac0ad030 con 0x7fb0ac103710 2024-04-17T22:29:19.953 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.966 7fb0b0dda700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0ac1043d0 con 0x7fb0ac1a0860 2024-04-17T22:29:19.953 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.966 7fb0b0dda700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb0ac0f9c20 con 0x7fb0ac19fde0 2024-04-17T22:29:19.953 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.966 7fb0aa365700 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7fb0ac103710 0x7fb0ac1aa640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:34782/0 (socket says 172.21.15.102:34782) 2024-04-17T22:29:19.953 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.966 7fb0a9b64700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb0ac19fde0 0x7fb0ac1a0230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:19.953 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.966 7fb0aa365700 1 -- 172.21.15.102:0/1730375115 learned_addr learned my addr 172.21.15.102:0/1730375115 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-17T22:29:19.953 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.966 7fb0a9363700 1 --2- 172.21.15.102:0/1730375115 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb0ac1a0860 0x7fb0ac1030e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:19.955 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.966 7fb0a9b64700 1 -- 172.21.15.102:0/1730375115 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb0ac1a0860 msgr2=0x7fb0ac1030e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:29:19.955 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.966 7fb0a9b64700 1 --2- 172.21.15.102:0/1730375115 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb0ac1a0860 0x7fb0ac1030e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:19.955 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.966 7fb0a9b64700 1 -- 172.21.15.102:0/1730375115 >> v1:172.21.15.102:6789/0 conn(0x7fb0ac103710 legacy=0x7fb0ac1aa640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:19.955 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.966 7fb0a8b62700 1 -- 172.21.15.102:0/1730375115 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 295232191 0 0) 0x7fb0a4001490 con 0x7fb0ac103710 2024-04-17T22:29:19.955 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.966 7fb0a9b64700 1 -- 172.21.15.102:0/1730375115 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0ac0f9870 con 0x7fb0ac19fde0 2024-04-17T22:29:19.955 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.966 7fb0a9363700 1 --2- 172.21.15.102:0/1730375115 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb0ac1a0860 0x7fb0ac1030e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:29:19.955 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.966 7fb0a9b64700 1 --2- 172.21.15.102:0/1730375115 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb0ac19fde0 0x7fb0ac1a0230 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7fb0a0002f70 tx=0x7fb0a000b9a0).ready entity=mon.2 client_cookie=1e5a4a7f7a392824 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:29:19.955 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.967 7fb0a8b62700 1 -- 172.21.15.102:0/1730375115 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb0ac0f9870 con 0x7fb0ac19fde0 2024-04-17T22:29:19.955 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.967 7fb0a8b62700 1 -- 172.21.15.102:0/1730375115 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0a000d5c0 con 0x7fb0ac19fde0 2024-04-17T22:29:19.955 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.967 7fb0a8b62700 1 -- 172.21.15.102:0/1730375115 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb0a000d990 con 0x7fb0ac19fde0 2024-04-17T22:29:19.955 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.967 7fb0b0dda700 1 -- 172.21.15.102:0/1730375115 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb0ac19fde0 msgr2=0x7fb0ac1a0230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:19.955 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.967 7fb0b0dda700 1 --2- 172.21.15.102:0/1730375115 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb0ac19fde0 0x7fb0ac1a0230 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7fb0a0002f70 tx=0x7fb0a000b9a0).stop 2024-04-17T22:29:19.955 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.967 7fb0b0dda700 1 -- 172.21.15.102:0/1730375115 shutdown_connections 2024-04-17T22:29:19.956 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.967 7fb0b0dda700 1 --2- 172.21.15.102:0/1730375115 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb0ac19fde0 0x7fb0ac1a0230 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:19.956 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.967 7fb0b0dda700 1 --2- 172.21.15.102:0/1730375115 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb0ac1a0860 0x7fb0ac1030e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:19.956 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.967 7fb0b0dda700 1 -- 172.21.15.102:0/1730375115 >> 172.21.15.102:0/1730375115 conn(0x7fb0ac113000 msgr2=0x7fb0ac113450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:29:19.956 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.967 7fb0b0dda700 1 -- 172.21.15.102:0/1730375115 shutdown_connections 2024-04-17T22:29:19.956 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.967 7fb0b0dda700 1 -- 172.21.15.102:0/1730375115 wait complete. 2024-04-17T22:29:19.978 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.990 7fb0b0dda700 1 Processor -- start 2024-04-17T22:29:19.978 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.990 7fb0b0dda700 1 -- start start 2024-04-17T22:29:19.978 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.990 7fb0b0dda700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb0ac0d5fa0 0x7fb0ac0d63f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:19.978 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.990 7fb0b0dda700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb0ac0d6a20 0x7fb0ac1b3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:19.978 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.990 7fb0b0dda700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb0ac1b4510 0x7fb0ac1b4960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:19.978 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.990 7fb0b0dda700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0ac0c4500 con 0x7fb0ac0d5fa0 2024-04-17T22:29:19.978 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.990 7fb0b0dda700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0ac0cd780 con 0x7fb0ac1b4510 2024-04-17T22:29:19.978 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.990 7fb0b0dda700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb0ac0cdad0 con 0x7fb0ac0d6a20 2024-04-17T22:29:19.979 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.990 7fb0aa365700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb0ac1b4510 0x7fb0ac1b4960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:19.979 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.990 7fb0a9b64700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb0ac0d5fa0 0x7fb0ac0d63f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:19.979 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.990 7fb0a9363700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb0ac0d6a20 0x7fb0ac1b3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:19.979 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.990 7fb0aa365700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb0ac1b4510 0x7fb0ac1b4960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:40510/0 (socket says 172.21.15.102:40510) 2024-04-17T22:29:19.979 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.990 7fb0a9b64700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb0ac0d5fa0 0x7fb0ac0d63f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:41070/0 (socket says 172.21.15.102:41070) 2024-04-17T22:29:19.979 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.990 7fb0aa365700 1 -- 172.21.15.102:0/3670840186 learned_addr learned my addr 172.21.15.102:0/3670840186 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:29:19.979 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.990 7fb0aa365700 1 -- 172.21.15.102:0/3670840186 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb0ac0d6a20 msgr2=0x7fb0ac1b3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:19.979 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.990 7fb0aa365700 1 --2- 172.21.15.102:0/3670840186 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb0ac0d6a20 0x7fb0ac1b3ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:19.979 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.990 7fb0aa365700 1 -- 172.21.15.102:0/3670840186 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb0ac0d5fa0 msgr2=0x7fb0ac0d63f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:19.979 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.990 7fb0aa365700 1 --2- 172.21.15.102:0/3670840186 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb0ac0d5fa0 0x7fb0ac0d63f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:19.979 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.990 7fb0aa365700 1 -- 172.21.15.102:0/3670840186 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0ac0ce2e0 con 0x7fb0ac1b4510 2024-04-17T22:29:19.979 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.991 7fb0aa365700 1 --2- 172.21.15.102:0/3670840186 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb0ac1b4510 0x7fb0ac1b4960 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7fb0ac0c4fc0 tx=0x7fb0a400cc40).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:29:19.979 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.991 7fb09affd700 1 -- 172.21.15.102:0/3670840186 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb0ac0ce2e0 con 0x7fb0ac1b4510 2024-04-17T22:29:19.979 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.991 7fb09affd700 1 -- 172.21.15.102:0/3670840186 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0a40059a0 con 0x7fb0ac1b4510 2024-04-17T22:29:19.980 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.991 7fb0b0dda700 1 -- 172.21.15.102:0/3670840186 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0ac1bb5d0 con 0x7fb0ac1b4510 2024-04-17T22:29:19.980 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.991 7fb09affd700 1 -- 172.21.15.102:0/3670840186 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb0a4005d70 con 0x7fb0ac1b4510 2024-04-17T22:29:19.980 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.991 7fb0b0dda700 1 -- 172.21.15.102:0/3670840186 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0ac1bb7d0 con 0x7fb0ac1b4510 2024-04-17T22:29:19.980 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.991 7fb09affd700 1 -- 172.21.15.102:0/3670840186 <== mon.1 v2:172.21.15.102:3301/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7fb0a401a020 con 0x7fb0ac1b4510 2024-04-17T22:29:19.980 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.991 7fb09affd700 1 -- 172.21.15.102:0/3670840186 <== mon.1 v2:172.21.15.102:3301/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 3194+0+0 (secure 0 0 0) 0x7fb0a4009d90 con 0x7fb0ac1b4510 2024-04-17T22:29:19.980 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.992 7fb0b0dda700 1 -- 172.21.15.102:0/3670840186 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0ac0f09f0 con 0x7fb0ac1b4510 2024-04-17T22:29:19.981 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:19.995 7fb09affd700 1 -- 172.21.15.102:0/3670840186 <== mon.1 v2:172.21.15.102:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+110455 (secure 0 0 0) 0x7fb0a400cd80 con 0x7fb0ac1b4510 2024-04-17T22:29:20.086 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.099 7fb0b0dda700 1 -- 172.21.15.102:0/3670840186 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb0ac1ae620 con 0x7fb0ac1b4510 2024-04-17T22:29:20.086 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.100 7fb09affd700 1 -- 172.21.15.102:0/3670840186 <== mon.1 v2:172.21.15.102:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v16) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb0ac113d90 con 0x7fb0ac1b4510 2024-04-17T22:29:20.087 INFO:teuthology.orchestra.run.smithi102.stdout:pg_num: 64 2024-04-17T22:29:20.088 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.101 7fb0b0dda700 1 -- 172.21.15.102:0/3670840186 >> v1:172.21.15.102:6816/42360 conn(0x7fb0880145a0 legacy=0x7fb088016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:20.088 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.101 7fb0b0dda700 1 -- 172.21.15.102:0/3670840186 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb0ac1b4510 msgr2=0x7fb0ac1b4960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:20.088 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.101 7fb0b0dda700 1 --2- 172.21.15.102:0/3670840186 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb0ac1b4510 0x7fb0ac1b4960 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7fb0ac0c4fc0 tx=0x7fb0a400cc40).stop 2024-04-17T22:29:20.088 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.101 7fb0b0dda700 1 -- 172.21.15.102:0/3670840186 shutdown_connections 2024-04-17T22:29:20.088 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.101 7fb0b0dda700 1 --2- 172.21.15.102:0/3670840186 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb0ac1b4510 0x7fb0ac1b4960 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:20.088 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.101 7fb0b0dda700 1 --2- 172.21.15.102:0/3670840186 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb0ac0d5fa0 0x7fb0ac0d63f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:20.088 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.101 7fb0b0dda700 1 --2- 172.21.15.102:0/3670840186 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb0ac0d6a20 0x7fb0ac1b3ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:20.088 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.101 7fb0b0dda700 1 -- 172.21.15.102:0/3670840186 >> 172.21.15.102:0/3670840186 conn(0x7fb0ac0e10e0 msgr2=0x7fb0ac0c2fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:29:20.088 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.101 7fb0b0dda700 1 -- 172.21.15.102:0/3670840186 shutdown_connections 2024-04-17T22:29:20.088 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.101 7fb0b0dda700 1 -- 172.21.15.102:0/3670840186 wait complete. 2024-04-17T22:29:20.099 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T22:29:20.225 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.236 7f3ed008e700 1 Processor -- start 2024-04-17T22:29:20.225 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.236 7f3ed008e700 1 -- start start 2024-04-17T22:29:20.225 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.237 7f3ed008e700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f3ec80f5770 0x7f3ec810c110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:20.225 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.237 7f3ed008e700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f3ec810c740 0x7f3ec81a0c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:20.225 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.237 7f3ed008e700 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ec81a1a30 con 0x7f3ec810cb90 2024-04-17T22:29:20.225 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.237 7f3ed008e700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ec80f93e0 con 0x7f3ec80f5770 2024-04-17T22:29:20.225 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.237 7f3ed008e700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3ec81a1c50 con 0x7f3ec810c740 2024-04-17T22:29:20.226 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.237 7f3ecce89700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f3ec80f5770 0x7f3ec810c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:20.226 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.237 7f3ecd68a700 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f3ec810cb90 0x7f3ec81a1230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:34788/0 (socket says 172.21.15.102:34788) 2024-04-17T22:29:20.226 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.237 7f3ecce89700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f3ec80f5770 0x7f3ec810c110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:40516/0 (socket says 172.21.15.102:40516) 2024-04-17T22:29:20.226 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.237 7f3ecd68a700 1 -- 172.21.15.102:0/1611220060 learned_addr learned my addr 172.21.15.102:0/1611220060 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-17T22:29:20.226 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.237 7f3ec7fff700 1 --2- 172.21.15.102:0/1611220060 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f3ec810c740 0x7f3ec81a0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:20.226 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.237 7f3ec77fe700 1 -- 172.21.15.102:0/1611220060 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 728257655 0 0) 0x7f3ec0001490 con 0x7f3ec810cb90 2024-04-17T22:29:20.226 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.237 7f3ecce89700 1 -- 172.21.15.102:0/1611220060 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f3ec810c740 msgr2=0x7f3ec81a0c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:29:20.226 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.237 7f3ecce89700 1 --2- 172.21.15.102:0/1611220060 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f3ec810c740 0x7f3ec81a0c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:20.226 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.237 7f3ecce89700 1 -- 172.21.15.102:0/1611220060 >> v1:172.21.15.102:6789/0 conn(0x7f3ec810cb90 legacy=0x7f3ec81a1230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:20.226 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.237 7f3ecce89700 1 -- 172.21.15.102:0/1611220060 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ec81aa330 con 0x7f3ec80f5770 2024-04-17T22:29:20.226 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.237 7f3ecce89700 1 --2- 172.21.15.102:0/1611220060 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f3ec80f5770 0x7f3ec810c110 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7f3ec80f9590 tx=0x7f3ebc00ba50).ready entity=mon.1 client_cookie=17b4de1b67f0a06b server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:29:20.226 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.237 7f3ec77fe700 1 -- 172.21.15.102:0/1611220060 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f3ec81aa330 con 0x7f3ec80f5770 2024-04-17T22:29:20.227 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.237 7f3ec77fe700 1 -- 172.21.15.102:0/1611220060 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ebc00d570 con 0x7f3ec80f5770 2024-04-17T22:29:20.227 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.237 7f3ec77fe700 1 -- 172.21.15.102:0/1611220060 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f3ebc00d940 con 0x7f3ec80f5770 2024-04-17T22:29:20.227 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.237 7f3ed008e700 1 -- 172.21.15.102:0/1611220060 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f3ec80f5770 msgr2=0x7f3ec810c110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:20.227 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.237 7f3ed008e700 1 --2- 172.21.15.102:0/1611220060 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f3ec80f5770 0x7f3ec810c110 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7f3ec80f9590 tx=0x7f3ebc00ba50).stop 2024-04-17T22:29:20.227 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.238 7f3ed008e700 1 -- 172.21.15.102:0/1611220060 shutdown_connections 2024-04-17T22:29:20.227 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.238 7f3ed008e700 1 --2- 172.21.15.102:0/1611220060 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f3ec80f5770 0x7f3ec810c110 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:20.227 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.238 7f3ed008e700 1 --2- 172.21.15.102:0/1611220060 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f3ec810c740 0x7f3ec81a0c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:20.227 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.238 7f3ed008e700 1 -- 172.21.15.102:0/1611220060 >> 172.21.15.102:0/1611220060 conn(0x7f3ec81138c0 msgr2=0x7f3ec80f4f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:29:20.227 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.238 7f3ed008e700 1 -- 172.21.15.102:0/1611220060 shutdown_connections 2024-04-17T22:29:20.227 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.238 7f3ed008e700 1 -- 172.21.15.102:0/1611220060 wait complete. 2024-04-17T22:29:20.248 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.262 7f3ed008e700 1 Processor -- start 2024-04-17T22:29:20.248 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.262 7f3ed008e700 1 -- start start 2024-04-17T22:29:20.250 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.262 7f3ed008e700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f3ec80d5e00 0x7f3ec80d6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:20.250 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.262 7f3ed008e700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f3ec80d6880 0x7f3ec81b1ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:20.250 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.262 7f3ed008e700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3ec81b21d0 0x7f3ec81b2620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:20.250 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.262 7f3ed008e700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ec80e2ce0 con 0x7f3ec81b21d0 2024-04-17T22:29:20.251 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.262 7f3ed008e700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ec81b2c50 con 0x7f3ec80d5e00 2024-04-17T22:29:20.251 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.262 7f3ed008e700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3ec80cd140 con 0x7f3ec80d6880 2024-04-17T22:29:20.251 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.263 7f3ecd68a700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3ec81b21d0 0x7f3ec81b2620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:20.251 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.263 7f3ecce89700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f3ec80d5e00 0x7f3ec80d6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:20.251 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.263 7f3ec7fff700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f3ec80d6880 0x7f3ec81b1ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:20.251 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.263 7f3ecce89700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f3ec80d5e00 0x7f3ec80d6250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:40530/0 (socket says 172.21.15.102:40530) 2024-04-17T22:29:20.251 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.263 7f3ecce89700 1 -- 172.21.15.102:0/2163262388 learned_addr learned my addr 172.21.15.102:0/2163262388 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:29:20.251 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.263 7f3ecd68a700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3ec81b21d0 0x7f3ec81b2620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:41080/0 (socket says 172.21.15.102:41080) 2024-04-17T22:29:20.251 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.263 7f3ec7fff700 1 -- 172.21.15.102:0/2163262388 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f3ec80d5e00 msgr2=0x7f3ec80d6250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:20.251 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.263 7f3ec7fff700 1 --2- 172.21.15.102:0/2163262388 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f3ec80d5e00 0x7f3ec80d6250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:20.251 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.263 7f3ec7fff700 1 -- 172.21.15.102:0/2163262388 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3ec81b21d0 msgr2=0x7f3ec81b2620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:20.251 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.263 7f3ec7fff700 1 --2- 172.21.15.102:0/2163262388 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3ec81b21d0 0x7f3ec81b2620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:20.251 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.263 7f3ec7fff700 1 -- 172.21.15.102:0/2163262388 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ec80cce40 con 0x7f3ec80d6880 2024-04-17T22:29:20.251 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.263 7f3ecd68a700 1 --2- 172.21.15.102:0/2163262388 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3ec81b21d0 0x7f3ec81b2620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:29:20.252 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.263 7f3ec7fff700 1 --2- 172.21.15.102:0/2163262388 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f3ec80d6880 0x7f3ec81b1ba0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7f3ec80d71e0 tx=0x7f3eb400ebc0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:29:20.252 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.263 7f3ec5ffb700 1 -- 172.21.15.102:0/2163262388 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f3ec80cce40 con 0x7f3ec80d6880 2024-04-17T22:29:20.252 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.263 7f3ec5ffb700 1 -- 172.21.15.102:0/2163262388 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3eb4010780 con 0x7f3ec80d6880 2024-04-17T22:29:20.252 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.263 7f3ed008e700 1 -- 172.21.15.102:0/2163262388 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ec80cda00 con 0x7f3ec80d6880 2024-04-17T22:29:20.252 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.263 7f3ec5ffb700 1 -- 172.21.15.102:0/2163262388 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f3eb400f040 con 0x7f3ec80d6880 2024-04-17T22:29:20.252 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.263 7f3ed008e700 1 -- 172.21.15.102:0/2163262388 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ec81b9500 con 0x7f3ec80d6880 2024-04-17T22:29:20.252 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.264 7f3ec5ffb700 1 -- 172.21.15.102:0/2163262388 <== mon.2 v2:172.21.15.102:3302/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7f3eb4011070 con 0x7f3ec80d6880 2024-04-17T22:29:20.252 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.264 7f3ec5ffb700 1 -- 172.21.15.102:0/2163262388 <== mon.2 v2:172.21.15.102:3302/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 3194+0+0 (secure 0 0 0) 0x7f3eb4010040 con 0x7f3ec80d6880 2024-04-17T22:29:20.252 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.264 7f3ed008e700 1 -- 172.21.15.102:0/2163262388 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ec80f0820 con 0x7f3ec80d6880 2024-04-17T22:29:20.254 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.267 7f3ec5ffb700 1 -- 172.21.15.102:0/2163262388 <== mon.2 v2:172.21.15.102:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+110455 (secure 0 0 0) 0x7f3eb4010380 con 0x7f3ec80d6880 2024-04-17T22:29:20.387 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.401 7f3ed008e700 1 -- 172.21.15.102:0/2163262388 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3ec80efef0 con 0x7f3ec80d6880 2024-04-17T22:29:20.388 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.401 7f3ec5ffb700 1 -- 172.21.15.102:0/2163262388 <== mon.2 v2:172.21.15.102:3302/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) v1 ==== 93+0+1704 (secure 0 0 0) 0x7f3eb4000c30 con 0x7f3ec80d6880 2024-04-17T22:29:20.388 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-17T22:29:20.388 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":5,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[],"filesystems":[{"mdsmap":{"epoch":5,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17 22:29:07.787730","modified":"2024-04-17 22:29:09.795249","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"-1 (unspecified)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4177},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4177":{"gid":4177,"name":"a","rank":0,"incarnation":3,"state":"up:active","state_seq":4,"addr":"172.21.15.102:6817/1944542598","addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6817","nonce":1944542598}]},"export_targets":[],"features":4611087854035861503,"flags":0}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-04-17T22:29:20.389 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.402 7f3ed008e700 1 -- 172.21.15.102:0/2163262388 >> v1:172.21.15.102:6816/42360 conn(0x7f3eb00145a0 legacy=0x7f3eb0016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:20.389 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.402 7f3ed008e700 1 -- 172.21.15.102:0/2163262388 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f3ec80d6880 msgr2=0x7f3ec81b1ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:20.389 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.402 7f3ed008e700 1 --2- 172.21.15.102:0/2163262388 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f3ec80d6880 0x7f3ec81b1ba0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7f3ec80d71e0 tx=0x7f3eb400ebc0).stop 2024-04-17T22:29:20.390 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.403 7f3ed008e700 1 -- 172.21.15.102:0/2163262388 shutdown_connections 2024-04-17T22:29:20.390 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.403 7f3ed008e700 1 --2- 172.21.15.102:0/2163262388 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3ec81b21d0 0x7f3ec81b2620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:20.390 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.403 7f3ed008e700 1 --2- 172.21.15.102:0/2163262388 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f3ec80d5e00 0x7f3ec80d6250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:20.390 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.403 7f3ed008e700 1 --2- 172.21.15.102:0/2163262388 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f3ec80d6880 0x7f3ec81b1ba0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:20.390 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.403 7f3ed008e700 1 -- 172.21.15.102:0/2163262388 >> 172.21.15.102:0/2163262388 conn(0x7f3ec80e0f60 msgr2=0x7f3ec80dac80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:29:20.390 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.403 7f3ed008e700 1 -- 172.21.15.102:0/2163262388 shutdown_connections 2024-04-17T22:29:20.390 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.403 7f3ed008e700 1 -- 172.21.15.102:0/2163262388 wait complete. 2024-04-17T22:29:20.390 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 5 2024-04-17T22:29:20.400 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T22:29:20.526 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.538 7f9fc6163700 1 Processor -- start 2024-04-17T22:29:20.527 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.538 7f9fc6163700 1 -- start start 2024-04-17T22:29:20.527 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.538 7f9fc6163700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f9fc00f5770 0x7f9fc010c110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:20.527 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.538 7f9fc6163700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f9fc010cb90 0x7f9fc01a1400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:20.527 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.538 7f9fc6163700 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9fc01a1a30 con 0x7f9fc010c740 2024-04-17T22:29:20.527 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.538 7f9fc6163700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fc00f93e0 con 0x7f9fc010cb90 2024-04-17T22:29:20.527 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.538 7f9fc6163700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9fc01a1c50 con 0x7f9fc00f5770 2024-04-17T22:29:20.527 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.539 7f9fbedc6700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f9fc00f5770 0x7f9fc010c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:20.527 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.539 7f9fbf5c7700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f9fc010cb90 0x7f9fc01a1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:20.527 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.539 7f9fbe5c5700 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f9fc010c740 0x7f9fc01a0c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:34802/0 (socket says 172.21.15.102:34802) 2024-04-17T22:29:20.527 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.539 7f9fbe5c5700 1 -- 172.21.15.102:0/2540073315 learned_addr learned my addr 172.21.15.102:0/2540073315 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-17T22:29:20.527 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.539 7f9fbddc4700 1 -- 172.21.15.102:0/2540073315 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2362007844 0 0) 0x7f9fb0001110 con 0x7f9fc010c740 2024-04-17T22:29:20.527 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.539 7f9fbf5c7700 1 -- 172.21.15.102:0/2540073315 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f9fc00f5770 msgr2=0x7f9fc010c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:29:20.528 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.539 7f9fbf5c7700 1 --2- 172.21.15.102:0/2540073315 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f9fc00f5770 0x7f9fc010c110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:20.528 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.539 7f9fbf5c7700 1 -- 172.21.15.102:0/2540073315 >> v1:172.21.15.102:6789/0 conn(0x7f9fc010c740 legacy=0x7f9fc01a0c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:20.528 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.539 7f9fbf5c7700 1 -- 172.21.15.102:0/2540073315 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fc01aa330 con 0x7f9fc010cb90 2024-04-17T22:29:20.528 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.539 7f9fbedc6700 1 --2- 172.21.15.102:0/2540073315 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f9fc00f5770 0x7f9fc010c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:29:20.528 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.539 7f9fbf5c7700 1 --2- 172.21.15.102:0/2540073315 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f9fc010cb90 0x7f9fc01a1400 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7f9fc00f4490 tx=0x7f9fb8009d70).ready entity=mon.1 client_cookie=22251068561c9c62 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:29:20.528 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.539 7f9fbddc4700 1 -- 172.21.15.102:0/2540073315 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9fc01aa330 con 0x7f9fc010cb90 2024-04-17T22:29:20.528 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.539 7f9fbddc4700 1 -- 172.21.15.102:0/2540073315 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fb800b7c0 con 0x7f9fc010cb90 2024-04-17T22:29:20.528 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.539 7f9fbddc4700 1 -- 172.21.15.102:0/2540073315 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9fb800b9a0 con 0x7f9fc010cb90 2024-04-17T22:29:20.528 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.539 7f9fc6163700 1 -- 172.21.15.102:0/2540073315 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f9fc010cb90 msgr2=0x7f9fc01a1400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:20.528 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.539 7f9fc6163700 1 --2- 172.21.15.102:0/2540073315 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f9fc010cb90 0x7f9fc01a1400 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7f9fc00f4490 tx=0x7f9fb8009d70).stop 2024-04-17T22:29:20.528 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.540 7f9fc6163700 1 -- 172.21.15.102:0/2540073315 shutdown_connections 2024-04-17T22:29:20.528 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.540 7f9fc6163700 1 --2- 172.21.15.102:0/2540073315 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f9fc010cb90 0x7f9fc01a1400 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:20.528 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.540 7f9fc6163700 1 --2- 172.21.15.102:0/2540073315 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f9fc00f5770 0x7f9fc010c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:20.528 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.540 7f9fc6163700 1 -- 172.21.15.102:0/2540073315 >> 172.21.15.102:0/2540073315 conn(0x7f9fc01138c0 msgr2=0x7f9fc00f4f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:29:20.528 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.540 7f9fc6163700 1 -- 172.21.15.102:0/2540073315 shutdown_connections 2024-04-17T22:29:20.529 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.540 7f9fc6163700 1 -- 172.21.15.102:0/2540073315 wait complete. 2024-04-17T22:29:20.551 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.562 7f9fc6163700 1 Processor -- start 2024-04-17T22:29:20.551 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.563 7f9fc6163700 1 -- start start 2024-04-17T22:29:20.551 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.563 7f9fc6163700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f9fc00cdd30 0x7f9fc00d5f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:20.551 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.563 7f9fc6163700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9fc00d6570 0x7f9fc01b1e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:20.551 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.563 7f9fc6163700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f9fc00d69c0 0x7f9fc01b24a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:20.551 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.563 7f9fc6163700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fc00f99f0 con 0x7f9fc00d6570 2024-04-17T22:29:20.551 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.563 7f9fc6163700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fc00cf810 con 0x7f9fc00cdd30 2024-04-17T22:29:20.551 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.563 7f9fc6163700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9fc01b2ad0 con 0x7f9fc00d69c0 2024-04-17T22:29:20.551 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.563 7f9fbedc6700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f9fc00cdd30 0x7f9fc00d5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:20.551 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.563 7f9fbe5c5700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9fc00d6570 0x7f9fc01b1e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:20.551 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.563 7f9fbf5c7700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f9fc00d69c0 0x7f9fc01b24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:20.551 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.563 7f9fbedc6700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f9fc00cdd30 0x7f9fc00d5f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:40548/0 (socket says 172.21.15.102:40548) 2024-04-17T22:29:20.552 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.563 7f9fbf5c7700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f9fc00d69c0 0x7f9fc01b24a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:44492/0 (socket says 172.21.15.102:44492) 2024-04-17T22:29:20.552 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.563 7f9fbe5c5700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9fc00d6570 0x7f9fc01b1e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:41084/0 (socket says 172.21.15.102:41084) 2024-04-17T22:29:20.552 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.563 7f9fbedc6700 1 -- 172.21.15.102:0/527210022 learned_addr learned my addr 172.21.15.102:0/527210022 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:29:20.552 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.563 7f9fbedc6700 1 -- 172.21.15.102:0/527210022 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f9fc00d69c0 msgr2=0x7f9fc01b24a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:20.552 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.563 7f9fbedc6700 1 --2- 172.21.15.102:0/527210022 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f9fc00d69c0 0x7f9fc01b24a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:20.552 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.563 7f9fbedc6700 1 -- 172.21.15.102:0/527210022 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9fc00d6570 msgr2=0x7f9fc01b1e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:20.552 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.563 7f9fbedc6700 1 --2- 172.21.15.102:0/527210022 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9fc00d6570 0x7f9fc01b1e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:20.552 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.563 7f9fbedc6700 1 -- 172.21.15.102:0/527210022 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fc01b2cd0 con 0x7f9fc00cdd30 2024-04-17T22:29:20.552 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.563 7f9fbf5c7700 1 --2- 172.21.15.102:0/527210022 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f9fc00d69c0 0x7f9fc01b24a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:29:20.552 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.563 7f9fbedc6700 1 --2- 172.21.15.102:0/527210022 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f9fc00cdd30 0x7f9fc00d5f40 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7f9fc00c5c40 tx=0x7f9fb400bf70).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:29:20.552 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.564 7f9faffff700 1 -- 172.21.15.102:0/527210022 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9fc01b2cd0 con 0x7f9fc00cdd30 2024-04-17T22:29:20.552 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.564 7f9faffff700 1 -- 172.21.15.102:0/527210022 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fb400c9a0 con 0x7f9fc00cdd30 2024-04-17T22:29:20.552 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.564 7f9fc6163700 1 -- 172.21.15.102:0/527210022 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fc00f61e0 con 0x7f9fc00cdd30 2024-04-17T22:29:20.552 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.564 7f9faffff700 1 -- 172.21.15.102:0/527210022 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9fb400f040 con 0x7f9fc00cdd30 2024-04-17T22:29:20.552 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.564 7f9fc6163700 1 -- 172.21.15.102:0/527210022 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fc00f6760 con 0x7f9fc00cdd30 2024-04-17T22:29:20.553 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.564 7f9faffff700 1 -- 172.21.15.102:0/527210022 <== mon.1 v2:172.21.15.102:3301/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7f9fb4011070 con 0x7f9fc00cdd30 2024-04-17T22:29:20.553 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.564 7f9faffff700 1 -- 172.21.15.102:0/527210022 <== mon.1 v2:172.21.15.102:3301/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 3194+0+0 (secure 0 0 0) 0x7f9fb4010040 con 0x7f9fc00cdd30 2024-04-17T22:29:20.553 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.565 7f9fc6163700 1 -- 172.21.15.102:0/527210022 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9fc00f0820 con 0x7f9fc00cdd30 2024-04-17T22:29:20.554 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.568 7f9faffff700 1 -- 172.21.15.102:0/527210022 <== mon.1 v2:172.21.15.102:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+110455 (secure 0 0 0) 0x7f9fb400c5a0 con 0x7f9fc00cdd30 2024-04-17T22:29:20.658 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.672 7f9fc6163700 1 -- 172.21.15.102:0/527210022 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9fc01ac590 con 0x7f9fc00cdd30 2024-04-17T22:29:20.659 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.673 7f9faffff700 1 -- 172.21.15.102:0/527210022 <== mon.1 v2:172.21.15.102:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+8761 (secure 0 0 0) 0x7f9fb4000c30 con 0x7f9fc00cdd30 2024-04-17T22:29:20.659 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-17T22:29:20.659 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":16,"fsid":"cb63e789-091d-49b1-b120-55f9db526871","created":"2024-04-17 22:28:48.209635","modified":"2024-04-17 22:29:08.789044","last_up_change":"2024-04-17 22:28:54.236215","last_in_change":"2024-04-17 22:28:54.236215","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":4,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-04-17 22:28:57.478545","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-17 22:29:06.144708","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-17 22:29:06.748924","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3351ef85-ade4-4b90-a8f4-213f4e00ba01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6812","nonce":42955}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6813","nonce":42955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6815","nonce":42955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6814","nonce":42955}]},"public_addr":"172.21.15.102:6812/42955","cluster_addr":"172.21.15.102:6813/42955","heartbeat_back_addr":"172.21.15.102:6815/42955","heartbeat_front_addr":"172.21.15.102:6814/42955","state":["exists","up"]},{"osd":1,"uuid":"04f67eac-f36b-48f5-982d-4cfc776596d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6808","nonce":42958}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6809","nonce":42958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6811","nonce":42958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6810","nonce":42958}]},"public_addr":"172.21.15.102:6808/42958","cluster_addr":"172.21.15.102:6809/42958","heartbeat_back_addr":"172.21.15.102:6811/42958","heartbeat_front_addr":"172.21.15.102:6810/42958","state":["exists","up"]},{"osd":2,"uuid":"d524204b-70b2-40a8-8f73-f9e7d44318af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6804","nonce":42954}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6805","nonce":42954}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6807","nonce":42954}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6806","nonce":42954}]},"public_addr":"172.21.15.102:6804/42954","cluster_addr":"172.21.15.102:6805/42954","heartbeat_back_addr":"172.21.15.102:6807/42954","heartbeat_front_addr":"172.21.15.102:6806/42954","state":["exists","up"]},{"osd":3,"uuid":"4fc9d20c-1ad0-484f-aafa-2c9a7574245b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6800","nonce":42953}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6801","nonce":42953}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6803","nonce":42953}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6802","nonce":42953}]},"public_addr":"172.21.15.102:6800/42953","cluster_addr":"172.21.15.102:6801/42953","heartbeat_back_addr":"172.21.15.102:6803/42953","heartbeat_front_addr":"172.21.15.102:6802/42953","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2024-04-17T22:29:20.660 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.674 7f9fc6163700 1 -- 172.21.15.102:0/527210022 >> v1:172.21.15.102:6816/42360 conn(0x7f9f9c0145a0 legacy=0x7f9f9c016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:20.661 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.674 7f9fc6163700 1 -- 172.21.15.102:0/527210022 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f9fc00cdd30 msgr2=0x7f9fc00d5f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:20.661 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.674 7f9fc6163700 1 --2- 172.21.15.102:0/527210022 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f9fc00cdd30 0x7f9fc00d5f40 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7f9fc00c5c40 tx=0x7f9fb400bf70).stop 2024-04-17T22:29:20.661 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.674 7f9fc6163700 1 -- 172.21.15.102:0/527210022 shutdown_connections 2024-04-17T22:29:20.661 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.674 7f9fc6163700 1 --2- 172.21.15.102:0/527210022 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f9fc00d69c0 0x7f9fc01b24a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:20.661 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.674 7f9fc6163700 1 --2- 172.21.15.102:0/527210022 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f9fc00cdd30 0x7f9fc00d5f40 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:20.661 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.674 7f9fc6163700 1 --2- 172.21.15.102:0/527210022 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9fc00d6570 0x7f9fc01b1e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:20.661 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.674 7f9fc6163700 1 -- 172.21.15.102:0/527210022 >> 172.21.15.102:0/527210022 conn(0x7f9fc00c3810 msgr2=0x7f9fc00d76c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:29:20.661 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.674 7f9fc6163700 1 -- 172.21.15.102:0/527210022 shutdown_connections 2024-04-17T22:29:20.661 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.674 7f9fc6163700 1 -- 172.21.15.102:0/527210022 wait complete. 2024-04-17T22:29:20.671 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2024-04-17 22:28:57.478545', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '12', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-17 22:29:06.144708', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-17 22:29:06.748924', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-17T22:29:20.672 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2024-04-17T22:29:20.800 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.812 7fba5b593700 1 Processor -- start 2024-04-17T22:29:20.800 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.812 7fba5b593700 1 -- start start 2024-04-17T22:29:20.801 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.812 7fba5b593700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fba5410e240 0x7fba5410e690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:20.801 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.812 7fba5b593700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fba540f1f30 0x7fba541ab4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:20.801 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.812 7fba5b593700 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba540f2380 con 0x7fba5410ecc0 2024-04-17T22:29:20.801 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.812 7fba5b593700 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba540f9f10 con 0x7fba5410e240 2024-04-17T22:29:20.801 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.812 7fba5b593700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fba540f9030 con 0x7fba540f1f30 2024-04-17T22:29:20.801 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.812 7fba58b8f700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fba540f1f30 0x7fba541ab4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:20.801 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.812 7fba53fff700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fba5410e240 0x7fba5410e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:20.801 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.812 7fba53fff700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fba5410e240 0x7fba5410e690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:41090/0 (socket says 172.21.15.102:41090) 2024-04-17T22:29:20.801 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.812 7fba58b8f700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fba540f1f30 0x7fba541ab4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:44508/0 (socket says 172.21.15.102:44508) 2024-04-17T22:29:20.801 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.812 7fba53fff700 1 -- 172.21.15.102:0/2043639852 learned_addr learned my addr 172.21.15.102:0/2043639852 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:29:20.801 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.813 7fba53fff700 1 -- 172.21.15.102:0/2043639852 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fba540f1f30 msgr2=0x7fba541ab4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:29:20.801 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.813 7fba53fff700 1 --2- 172.21.15.102:0/2043639852 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fba540f1f30 0x7fba541ab4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:20.801 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.813 7fba53fff700 1 -- 172.21.15.102:0/2043639852 >> v1:172.21.15.102:6789/0 conn(0x7fba5410ecc0 legacy=0x7fba540f1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:20.801 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.813 7fba53fff700 1 -- 172.21.15.102:0/2043639852 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba540f25a0 con 0x7fba5410e240 2024-04-17T22:29:20.801 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.813 7fba52ffd700 1 -- 172.21.15.102:0/2043639852 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3045184142 0 0) 0x7fba44001110 con 0x7fba5410ecc0 2024-04-17T22:29:20.802 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.813 7fba58b8f700 1 --2- 172.21.15.102:0/2043639852 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fba540f1f30 0x7fba541ab4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:29:20.802 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.813 7fba53fff700 1 --2- 172.21.15.102:0/2043639852 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fba5410e240 0x7fba5410e690 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 rx=0x7fba540f46a0 tx=0x7fba48009a80).ready entity=mon.0 client_cookie=e1a107781f624a43 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:29:20.802 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.813 7fba52ffd700 1 -- 172.21.15.102:0/2043639852 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fba540f25a0 con 0x7fba5410e240 2024-04-17T22:29:20.802 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.813 7fba52ffd700 1 -- 172.21.15.102:0/2043639852 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba4800f510 con 0x7fba5410e240 2024-04-17T22:29:20.802 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.813 7fba52ffd700 1 -- 172.21.15.102:0/2043639852 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fba4800f6f0 con 0x7fba5410e240 2024-04-17T22:29:20.802 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.813 7fba5b593700 1 -- 172.21.15.102:0/2043639852 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fba5410e240 msgr2=0x7fba5410e690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:20.802 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.813 7fba5b593700 1 --2- 172.21.15.102:0/2043639852 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fba5410e240 0x7fba5410e690 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 rx=0x7fba540f46a0 tx=0x7fba48009a80).stop 2024-04-17T22:29:20.802 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.814 7fba5b593700 1 -- 172.21.15.102:0/2043639852 shutdown_connections 2024-04-17T22:29:20.802 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.814 7fba5b593700 1 --2- 172.21.15.102:0/2043639852 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fba540f1f30 0x7fba541ab4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:20.802 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.814 7fba5b593700 1 --2- 172.21.15.102:0/2043639852 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fba5410e240 0x7fba5410e690 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:20.802 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.814 7fba5b593700 1 -- 172.21.15.102:0/2043639852 >> 172.21.15.102:0/2043639852 conn(0x7fba541138a0 msgr2=0x7fba540f4f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:29:20.802 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.814 7fba5b593700 1 -- 172.21.15.102:0/2043639852 shutdown_connections 2024-04-17T22:29:20.802 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.814 7fba5b593700 1 -- 172.21.15.102:0/2043639852 wait complete. 2024-04-17T22:29:20.822 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.836 7fba5b593700 1 Processor -- start 2024-04-17T22:29:20.822 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.836 7fba5b593700 1 -- start start 2024-04-17T22:29:20.825 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.836 7fba5b593700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fba541ad9e0 0x7fba541ade30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:20.825 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.836 7fba5b593700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fba541ae460 0x7fba541b43e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:20.825 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.836 7fba5b593700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fba541b4a10 0x7fba541b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:20.825 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.836 7fba5b593700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba540fa1b0 con 0x7fba541b4a10 2024-04-17T22:29:20.825 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.836 7fba5b593700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba540d7cf0 con 0x7fba541ae460 2024-04-17T22:29:20.825 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.836 7fba5b593700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fba540d7ac0 con 0x7fba541ad9e0 2024-04-17T22:29:20.825 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.836 7fba53fff700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fba541ad9e0 0x7fba541ade30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:20.825 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.837 7fba53fff700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fba541ad9e0 0x7fba541ade30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:44514/0 (socket says 172.21.15.102:44514) 2024-04-17T22:29:20.825 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.837 7fba537fe700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fba541ae460 0x7fba541b43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:20.825 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.837 7fba58b8f700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fba541b4a10 0x7fba541b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:20.825 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.837 7fba53fff700 1 -- 172.21.15.102:0/665417954 learned_addr learned my addr 172.21.15.102:0/665417954 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:29:20.825 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.837 7fba58b8f700 1 -- 172.21.15.102:0/665417954 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fba541ad9e0 msgr2=0x7fba541ade30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:20.825 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.837 7fba58b8f700 1 --2- 172.21.15.102:0/665417954 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fba541ad9e0 0x7fba541ade30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:20.826 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.837 7fba58b8f700 1 -- 172.21.15.102:0/665417954 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fba541ae460 msgr2=0x7fba541b43e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:20.826 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.837 7fba58b8f700 1 --2- 172.21.15.102:0/665417954 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fba541ae460 0x7fba541b43e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:20.826 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.837 7fba58b8f700 1 -- 172.21.15.102:0/665417954 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba540d87f0 con 0x7fba541b4a10 2024-04-17T22:29:20.826 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.837 7fba537fe700 1 --2- 172.21.15.102:0/665417954 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fba541ae460 0x7fba541b43e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:29:20.826 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.837 7fba53fff700 1 --2- 172.21.15.102:0/665417954 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fba541ad9e0 0x7fba541ade30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:29:20.826 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.837 7fba58b8f700 1 --2- 172.21.15.102:0/665417954 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fba541b4a10 0x7fba541b4e60 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 rx=0x7fba540d4370 tx=0x7fba4c00eda0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:29:20.826 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.837 7fba517fa700 1 -- 172.21.15.102:0/665417954 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fba540d87f0 con 0x7fba541b4a10 2024-04-17T22:29:20.826 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.837 7fba517fa700 1 -- 172.21.15.102:0/665417954 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba4c00c880 con 0x7fba541b4a10 2024-04-17T22:29:20.826 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.837 7fba5b593700 1 -- 172.21.15.102:0/665417954 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba541bbb50 con 0x7fba541b4a10 2024-04-17T22:29:20.826 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.837 7fba517fa700 1 -- 172.21.15.102:0/665417954 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fba4c00f040 con 0x7fba541b4a10 2024-04-17T22:29:20.826 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.837 7fba5b593700 1 -- 172.21.15.102:0/665417954 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba541bbf30 con 0x7fba541b4a10 2024-04-17T22:29:20.826 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.838 7fba517fa700 1 -- 172.21.15.102:0/665417954 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7fba4c022060 con 0x7fba541b4a10 2024-04-17T22:29:20.826 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.838 7fba517fa700 1 -- 172.21.15.102:0/665417954 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 3194+0+0 (secure 0 0 0) 0x7fba4c010040 con 0x7fba541b4a10 2024-04-17T22:29:20.826 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.838 7fba5b593700 1 -- 172.21.15.102:0/665417954 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba540f0800 con 0x7fba541b4a10 2024-04-17T22:29:20.828 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.841 7fba517fa700 1 -- 172.21.15.102:0/665417954 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+110455 (secure 0 0 0) 0x7fba4c00c670 con 0x7fba541b4a10 2024-04-17T22:29:20.932 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.945 7fba5b593700 1 -- 172.21.15.102:0/665417954 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fba540abc60 con 0x7fba541b4a10 2024-04-17T22:29:20.932 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.946 7fba517fa700 1 -- 172.21.15.102:0/665417954 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v16) v1 ==== 92+0+10 (secure 0 0 0) 0x7fba54114420 con 0x7fba541b4a10 2024-04-17T22:29:20.932 INFO:teuthology.orchestra.run.smithi102.stdout:pg_num: 8 2024-04-17T22:29:20.934 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.947 7fba5b593700 1 -- 172.21.15.102:0/665417954 >> v1:172.21.15.102:6816/42360 conn(0x7fba340145a0 legacy=0x7fba34016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:20.934 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.947 7fba5b593700 1 -- 172.21.15.102:0/665417954 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fba541b4a10 msgr2=0x7fba541b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:20.934 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.947 7fba5b593700 1 --2- 172.21.15.102:0/665417954 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fba541b4a10 0x7fba541b4e60 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 rx=0x7fba540d4370 tx=0x7fba4c00eda0).stop 2024-04-17T22:29:20.934 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.947 7fba5b593700 1 -- 172.21.15.102:0/665417954 shutdown_connections 2024-04-17T22:29:20.934 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.947 7fba5b593700 1 --2- 172.21.15.102:0/665417954 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fba541b4a10 0x7fba541b4e60 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:20.934 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.947 7fba5b593700 1 --2- 172.21.15.102:0/665417954 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fba541ad9e0 0x7fba541ade30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:20.934 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.947 7fba5b593700 1 --2- 172.21.15.102:0/665417954 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fba541ae460 0x7fba541b43e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:20.934 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.947 7fba5b593700 1 -- 172.21.15.102:0/665417954 >> 172.21.15.102:0/665417954 conn(0x7fba540c6af0 msgr2=0x7fba541a84e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:29:20.934 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.947 7fba5b593700 1 -- 172.21.15.102:0/665417954 shutdown_connections 2024-04-17T22:29:20.934 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:20.947 7fba5b593700 1 -- 172.21.15.102:0/665417954 wait complete. 2024-04-17T22:29:20.944 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-17T22:29:21.073 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.085 7f985575e700 1 Processor -- start 2024-04-17T22:29:21.073 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.085 7f985575e700 1 -- start start 2024-04-17T22:29:21.073 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.085 7f985575e700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f98500f5be0 0x7f985010e840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:21.073 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.085 7f985575e700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f985010ee70 0x7f98500f1920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:21.073 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.085 7f985575e700 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9850103670 con 0x7f98500f1f50 2024-04-17T22:29:21.074 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.085 7f985575e700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f985010f2c0 con 0x7f98500f5be0 2024-04-17T22:29:21.074 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.085 7f985575e700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9850110e50 con 0x7f985010ee70 2024-04-17T22:29:21.074 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.085 7f984dceb700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f985010ee70 0x7f98500f1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:21.074 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.085 7f984e4ec700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f98500f5be0 0x7f985010e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:21.074 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.085 7f984eced700 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f98500f1f50 0x7f98501aa3d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:34806/0 (socket says 172.21.15.102:34806) 2024-04-17T22:29:21.074 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.085 7f984dceb700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f985010ee70 0x7f98500f1920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:44530/0 (socket says 172.21.15.102:44530) 2024-04-17T22:29:21.074 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.085 7f984eced700 1 -- 172.21.15.102:0/1310095499 learned_addr learned my addr 172.21.15.102:0/1310095499 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-17T22:29:21.074 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.086 7f984d4ea700 1 -- 172.21.15.102:0/1310095499 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1848153183 0 0) 0x7f9848001490 con 0x7f98500f1f50 2024-04-17T22:29:21.074 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.086 7f984e4ec700 1 -- 172.21.15.102:0/1310095499 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f985010ee70 msgr2=0x7f98500f1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:29:21.074 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.086 7f984e4ec700 1 --2- 172.21.15.102:0/1310095499 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f985010ee70 0x7f98500f1920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:21.074 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.086 7f984e4ec700 1 -- 172.21.15.102:0/1310095499 >> v1:172.21.15.102:6789/0 conn(0x7f98500f1f50 legacy=0x7f98501aa3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:21.074 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.086 7f984e4ec700 1 -- 172.21.15.102:0/1310095499 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98500f23a0 con 0x7f98500f5be0 2024-04-17T22:29:21.074 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.086 7f984dceb700 1 --2- 172.21.15.102:0/1310095499 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f985010ee70 0x7f98500f1920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:29:21.074 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.086 7f984e4ec700 1 --2- 172.21.15.102:0/1310095499 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f98500f5be0 0x7f985010e840 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7f9844002f70 tx=0x7f984400ba20).ready entity=mon.1 client_cookie=b33ce7ea7a4ace54 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:29:21.074 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.086 7f984d4ea700 1 -- 172.21.15.102:0/1310095499 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f98500f23a0 con 0x7f98500f5be0 2024-04-17T22:29:21.075 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.086 7f984d4ea700 1 -- 172.21.15.102:0/1310095499 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f984400d570 con 0x7f98500f5be0 2024-04-17T22:29:21.075 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.086 7f984d4ea700 1 -- 172.21.15.102:0/1310095499 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f984400d940 con 0x7f98500f5be0 2024-04-17T22:29:21.075 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.086 7f985575e700 1 -- 172.21.15.102:0/1310095499 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f98500f5be0 msgr2=0x7f985010e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:21.075 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.086 7f985575e700 1 --2- 172.21.15.102:0/1310095499 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f98500f5be0 0x7f985010e840 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7f9844002f70 tx=0x7f984400ba20).stop 2024-04-17T22:29:21.075 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.087 7f985575e700 1 -- 172.21.15.102:0/1310095499 shutdown_connections 2024-04-17T22:29:21.075 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.087 7f985575e700 1 --2- 172.21.15.102:0/1310095499 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f98500f5be0 0x7f985010e840 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:21.075 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.087 7f985575e700 1 --2- 172.21.15.102:0/1310095499 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f985010ee70 0x7f98500f1920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:21.075 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.087 7f985575e700 1 -- 172.21.15.102:0/1310095499 >> 172.21.15.102:0/1310095499 conn(0x7f9850113a90 msgr2=0x7f98500f5140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:29:21.075 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.087 7f985575e700 1 -- 172.21.15.102:0/1310095499 shutdown_connections 2024-04-17T22:29:21.075 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.087 7f985575e700 1 -- 172.21.15.102:0/1310095499 wait complete. 2024-04-17T22:29:21.097 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.109 7f985575e700 1 Processor -- start 2024-04-17T22:29:21.098 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.109 7f985575e700 1 -- start start 2024-04-17T22:29:21.098 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.110 7f985575e700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f98501ad8c0 0x7f98501add10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:21.098 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.110 7f985575e700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f98501ae340 0x7f98501b4460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:21.098 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.110 7f985575e700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f98501b4a90 0x7f98501b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:21.098 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.110 7f985575e700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98500e0540 con 0x7f98501ae340 2024-04-17T22:29:21.098 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.110 7f985575e700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98500c4cd0 con 0x7f98501ad8c0 2024-04-17T22:29:21.098 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.110 7f985575e700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f98500e0c20 con 0x7f98501b4a90 2024-04-17T22:29:21.098 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.110 7f984e4ec700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f98501ad8c0 0x7f98501add10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:21.098 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.110 7f984eced700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f98501b4a90 0x7f98501b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:21.098 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.110 7f984dceb700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f98501ae340 0x7f98501b4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:21.098 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.110 7f984e4ec700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f98501ad8c0 0x7f98501add10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:40568/0 (socket says 172.21.15.102:40568) 2024-04-17T22:29:21.098 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.110 7f984dceb700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f98501ae340 0x7f98501b4460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:41116/0 (socket says 172.21.15.102:41116) 2024-04-17T22:29:21.098 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.110 7f984eced700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f98501b4a90 0x7f98501b4ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:44532/0 (socket says 172.21.15.102:44532) 2024-04-17T22:29:21.099 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.110 7f984e4ec700 1 -- 172.21.15.102:0/903392533 learned_addr learned my addr 172.21.15.102:0/903392533 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:29:21.099 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.110 7f984e4ec700 1 -- 172.21.15.102:0/903392533 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f98501b4a90 msgr2=0x7f98501b4ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:21.099 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.110 7f984e4ec700 1 --2- 172.21.15.102:0/903392533 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f98501b4a90 0x7f98501b4ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:21.099 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.110 7f984e4ec700 1 -- 172.21.15.102:0/903392533 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f98501ae340 msgr2=0x7f98501b4460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:21.099 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.110 7f984e4ec700 1 --2- 172.21.15.102:0/903392533 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f98501ae340 0x7f98501b4460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:21.099 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.110 7f984e4ec700 1 -- 172.21.15.102:0/903392533 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98501bbb50 con 0x7f98501ad8c0 2024-04-17T22:29:21.099 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.110 7f984eced700 1 --2- 172.21.15.102:0/903392533 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f98501b4a90 0x7f98501b4ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:29:21.099 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.110 7f984dceb700 1 --2- 172.21.15.102:0/903392533 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f98501ae340 0x7f98501b4460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:29:21.099 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.110 7f984e4ec700 1 --2- 172.21.15.102:0/903392533 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f98501ad8c0 0x7f98501add10 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7f98500c2aa0 tx=0x7f98440068c0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:29:21.099 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.110 7f983f7fe700 1 -- 172.21.15.102:0/903392533 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f98501bbb50 con 0x7f98501ad8c0 2024-04-17T22:29:21.099 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.110 7f983f7fe700 1 -- 172.21.15.102:0/903392533 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9844011770 con 0x7f98501ad8c0 2024-04-17T22:29:21.099 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.110 7f985575e700 1 -- 172.21.15.102:0/903392533 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98501bbd50 con 0x7f98501ad8c0 2024-04-17T22:29:21.099 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.110 7f983f7fe700 1 -- 172.21.15.102:0/903392533 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9844017040 con 0x7f98501ad8c0 2024-04-17T22:29:21.099 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.111 7f985575e700 1 -- 172.21.15.102:0/903392533 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98501bc130 con 0x7f98501ad8c0 2024-04-17T22:29:21.100 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.111 7f983f7fe700 1 -- 172.21.15.102:0/903392533 <== mon.1 v2:172.21.15.102:3301/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7f9844011b40 con 0x7f98501ad8c0 2024-04-17T22:29:21.100 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.111 7f983f7fe700 1 -- 172.21.15.102:0/903392533 <== mon.1 v2:172.21.15.102:3301/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 3194+0+0 (secure 0 0 0) 0x7f9844018070 con 0x7f98501ad8c0 2024-04-17T22:29:21.100 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.112 7f985575e700 1 -- 172.21.15.102:0/903392533 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98500ad030 con 0x7f98501ad8c0 2024-04-17T22:29:21.101 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.115 7f983f7fe700 1 -- 172.21.15.102:0/903392533 <== mon.1 v2:172.21.15.102:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+110455 (secure 0 0 0) 0x7f9844011560 con 0x7f98501ad8c0 2024-04-17T22:29:21.206 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.219 7f985575e700 1 -- 172.21.15.102:0/903392533 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f98500ac930 con 0x7f98501ad8c0 2024-04-17T22:29:21.206 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.220 7f983f7fe700 1 -- 172.21.15.102:0/903392533 <== mon.1 v2:172.21.15.102:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v16) v1 ==== 104+0+11 (secure 0 0 0) 0x7f984400d360 con 0x7f98501ad8c0 2024-04-17T22:29:21.207 INFO:teuthology.orchestra.run.smithi102.stdout:pg_num: 64 2024-04-17T22:29:21.207 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.221 7f985575e700 1 -- 172.21.15.102:0/903392533 >> v1:172.21.15.102:6816/42360 conn(0x7f982c0145a0 legacy=0x7f982c016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:21.208 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.221 7f985575e700 1 -- 172.21.15.102:0/903392533 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f98501ad8c0 msgr2=0x7f98501add10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:21.208 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.221 7f985575e700 1 --2- 172.21.15.102:0/903392533 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f98501ad8c0 0x7f98501add10 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7f98500c2aa0 tx=0x7f98440068c0).stop 2024-04-17T22:29:21.208 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.221 7f985575e700 1 -- 172.21.15.102:0/903392533 shutdown_connections 2024-04-17T22:29:21.208 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.221 7f985575e700 1 --2- 172.21.15.102:0/903392533 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f98501b4a90 0x7f98501b4ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:21.208 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.221 7f985575e700 1 --2- 172.21.15.102:0/903392533 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f98501ad8c0 0x7f98501add10 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:21.208 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.221 7f985575e700 1 --2- 172.21.15.102:0/903392533 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f98501ae340 0x7f98501b4460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:21.208 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.221 7f985575e700 1 -- 172.21.15.102:0/903392533 >> 172.21.15.102:0/903392533 conn(0x7f98500c6f70 msgr2=0x7f98501a8380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:29:21.208 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.221 7f985575e700 1 -- 172.21.15.102:0/903392533 shutdown_connections 2024-04-17T22:29:21.208 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.221 7f985575e700 1 -- 172.21.15.102:0/903392533 wait complete. 2024-04-17T22:29:21.219 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-17T22:29:21.350 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.361 7f8985fd8700 1 Processor -- start 2024-04-17T22:29:21.350 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.361 7f8985fd8700 1 -- start start 2024-04-17T22:29:21.350 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.361 7f8985fd8700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f89800f97b0 0x7f89800fb9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:21.350 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.361 7f8985fd8700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f898019f3b0 0x7f898019f800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:21.350 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.361 7f8985fd8700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f898019fe30 0x7f89801a6890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:21.351 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.362 7f8985fd8700 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89800f2c70 con 0x7f898019f3b0 2024-04-17T22:29:21.351 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.362 7f8985fd8700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89801a0280 con 0x7f898019fe30 2024-04-17T22:29:21.351 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.362 7f8985fd8700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8980104fe0 con 0x7f89800f97b0 2024-04-17T22:29:21.351 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.362 7f897edc6700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f89800f97b0 0x7f89800fb9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:21.351 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.362 7f897f5c7700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f898019fe30 0x7f89801a6890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:21.351 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.362 7f897edc6700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f89800f97b0 0x7f89800fb9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:44536/0 (socket says 172.21.15.102:44536) 2024-04-17T22:29:21.351 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.362 7f897edc6700 1 -- 172.21.15.102:0/2119462904 learned_addr learned my addr 172.21.15.102:0/2119462904 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:29:21.351 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.362 7f897f5c7700 1 --2- 172.21.15.102:0/2119462904 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f898019fe30 0x7f89801a6890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:40584/0 (socket says 172.21.15.102:40584) 2024-04-17T22:29:21.351 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.362 7f8977fff700 1 --2- 172.21.15.102:0/2119462904 >> v2:172.21.15.102:3300/0 conn(0x7f898019f3b0 0x7f898019f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:21.351 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.362 7f897edc6700 1 -- 172.21.15.102:0/2119462904 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f898019fe30 msgr2=0x7f89801a6890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:29:21.351 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.362 7f897edc6700 1 --2- 172.21.15.102:0/2119462904 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f898019fe30 0x7f89801a6890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:21.351 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.362 7f897edc6700 1 -- 172.21.15.102:0/2119462904 >> v2:172.21.15.102:3300/0 conn(0x7f898019f3b0 msgr2=0x7f898019f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:29:21.351 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.362 7f897edc6700 1 --2- 172.21.15.102:0/2119462904 >> v2:172.21.15.102:3300/0 conn(0x7f898019f3b0 0x7f898019f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:21.351 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.362 7f897edc6700 1 -- 172.21.15.102:0/2119462904 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89800fc020 con 0x7f89800f97b0 2024-04-17T22:29:21.351 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.362 7f897f5c7700 1 --2- 172.21.15.102:0/2119462904 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f898019fe30 0x7f89801a6890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:29:21.352 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.362 7f897edc6700 1 --2- 172.21.15.102:0/2119462904 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f89800f97b0 0x7f89800fb9f0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7f8970002f70 tx=0x7f897000bc00).ready entity=mon.2 client_cookie=1c71c9c0f322739e server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:29:21.352 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.362 7f897e5c5700 1 -- 172.21.15.102:0/2119462904 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f89800fc020 con 0x7f89800f97b0 2024-04-17T22:29:21.352 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.362 7f897e5c5700 1 -- 172.21.15.102:0/2119462904 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f897000dc80 con 0x7f89800f97b0 2024-04-17T22:29:21.352 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.362 7f897e5c5700 1 -- 172.21.15.102:0/2119462904 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8970010660 con 0x7f89800f97b0 2024-04-17T22:29:21.352 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.362 7f8985fd8700 1 -- 172.21.15.102:0/2119462904 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f89800f97b0 msgr2=0x7f89800fb9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:21.352 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.362 7f8985fd8700 1 --2- 172.21.15.102:0/2119462904 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f89800f97b0 0x7f89800fb9f0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7f8970002f70 tx=0x7f897000bc00).stop 2024-04-17T22:29:21.352 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.363 7f8985fd8700 1 -- 172.21.15.102:0/2119462904 shutdown_connections 2024-04-17T22:29:21.352 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.363 7f8985fd8700 1 --2- 172.21.15.102:0/2119462904 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f898019fe30 0x7f89801a6890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:21.352 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.363 7f8985fd8700 1 --2- 172.21.15.102:0/2119462904 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f89800f97b0 0x7f89800fb9f0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:21.352 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.363 7f8985fd8700 1 --2- 172.21.15.102:0/2119462904 >> v2:172.21.15.102:3300/0 conn(0x7f898019f3b0 0x7f898019f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:21.352 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.363 7f8985fd8700 1 -- 172.21.15.102:0/2119462904 >> 172.21.15.102:0/2119462904 conn(0x7f8980113000 msgr2=0x7f8980113450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:29:21.352 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.363 7f8985fd8700 1 -- 172.21.15.102:0/2119462904 shutdown_connections 2024-04-17T22:29:21.353 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.363 7f8985fd8700 1 -- 172.21.15.102:0/2119462904 wait complete. 2024-04-17T22:29:21.374 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.386 7f8985fd8700 1 Processor -- start 2024-04-17T22:29:21.374 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.386 7f8985fd8700 1 -- start start 2024-04-17T22:29:21.374 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.386 7f8985fd8700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f89800ca820 0x7f89800cac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:21.374 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.386 7f8985fd8700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f89801afe50 0x7f89801b02a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:21.374 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.386 7f8985fd8700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f89801b08d0 0x7f89801b7330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:21.374 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.386 7f8985fd8700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89800f00f0 con 0x7f89801afe50 2024-04-17T22:29:21.374 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.386 7f8985fd8700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89800c4260 con 0x7f89801b08d0 2024-04-17T22:29:21.375 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.386 7f8985fd8700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f89800e02b0 con 0x7f89800ca820 2024-04-17T22:29:21.375 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.386 7f897edc6700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f89800ca820 0x7f89800cac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:21.375 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.386 7f897f5c7700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f89801b08d0 0x7f89801b7330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:21.375 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.386 7f8977fff700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f89801afe50 0x7f89801b02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:21.375 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.386 7f897f5c7700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f89801b08d0 0x7f89801b7330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:40586/0 (socket says 172.21.15.102:40586) 2024-04-17T22:29:21.375 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.386 7f8977fff700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f89801afe50 0x7f89801b02a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:41120/0 (socket says 172.21.15.102:41120) 2024-04-17T22:29:21.375 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.386 7f897edc6700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f89800ca820 0x7f89800cac70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:44540/0 (socket says 172.21.15.102:44540) 2024-04-17T22:29:21.375 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.386 7f897f5c7700 1 -- 172.21.15.102:0/4082983938 learned_addr learned my addr 172.21.15.102:0/4082983938 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:29:21.375 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.387 7f897edc6700 1 -- 172.21.15.102:0/4082983938 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f89801b08d0 msgr2=0x7f89801b7330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:21.375 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.387 7f897edc6700 1 --2- 172.21.15.102:0/4082983938 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f89801b08d0 0x7f89801b7330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:21.375 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.387 7f897edc6700 1 -- 172.21.15.102:0/4082983938 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f89801afe50 msgr2=0x7f89801b02a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:21.375 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.387 7f897edc6700 1 --2- 172.21.15.102:0/4082983938 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f89801afe50 0x7f89801b02a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:21.375 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.387 7f897edc6700 1 -- 172.21.15.102:0/4082983938 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89800d4ce0 con 0x7f89800ca820 2024-04-17T22:29:21.375 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.387 7f8977fff700 1 --2- 172.21.15.102:0/4082983938 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f89801afe50 0x7f89801b02a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:29:21.375 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.387 7f897edc6700 1 --2- 172.21.15.102:0/4082983938 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f89800ca820 0x7f89800cac70 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7f89800c4820 tx=0x7f8970006cd0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:29:21.375 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.387 7f897cdc2700 1 -- 172.21.15.102:0/4082983938 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f89800d4ce0 con 0x7f89800ca820 2024-04-17T22:29:21.376 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.387 7f897cdc2700 1 -- 172.21.15.102:0/4082983938 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8970010da0 con 0x7f89800ca820 2024-04-17T22:29:21.376 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.387 7f8985fd8700 1 -- 172.21.15.102:0/4082983938 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89801b0d20 con 0x7f89800ca820 2024-04-17T22:29:21.376 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.387 7f897cdc2700 1 -- 172.21.15.102:0/4082983938 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8970024640 con 0x7f89800ca820 2024-04-17T22:29:21.376 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.387 7f8985fd8700 1 -- 172.21.15.102:0/4082983938 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89801b7960 con 0x7f89800ca820 2024-04-17T22:29:21.376 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.387 7f897cdc2700 1 -- 172.21.15.102:0/4082983938 <== mon.2 v2:172.21.15.102:3302/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7f8970024840 con 0x7f89800ca820 2024-04-17T22:29:21.376 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.387 7f897cdc2700 1 -- 172.21.15.102:0/4082983938 <== mon.2 v2:172.21.15.102:3302/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 3194+0+0 (secure 0 0 0) 0x7f897001d130 con 0x7f89800ca820 2024-04-17T22:29:21.376 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.388 7f8985fd8700 1 -- 172.21.15.102:0/4082983938 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89800f09f0 con 0x7f89800ca820 2024-04-17T22:29:21.377 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.391 7f897cdc2700 1 -- 172.21.15.102:0/4082983938 <== mon.2 v2:172.21.15.102:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+110455 (secure 0 0 0) 0x7f897000d880 con 0x7f89800ca820 2024-04-17T22:29:21.482 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.496 7f8985fd8700 1 -- 172.21.15.102:0/4082983938 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f89800da980 con 0x7f89800ca820 2024-04-17T22:29:21.483 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.496 7f897cdc2700 1 -- 172.21.15.102:0/4082983938 <== mon.2 v2:172.21.15.102:3302/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v16) v1 ==== 100+0+11 (secure 0 0 0) 0x7f8970010b90 con 0x7f89800ca820 2024-04-17T22:29:21.483 INFO:teuthology.orchestra.run.smithi102.stdout:pg_num: 64 2024-04-17T22:29:21.484 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.498 7f8985fd8700 1 -- 172.21.15.102:0/4082983938 >> v1:172.21.15.102:6816/42360 conn(0x7f89640145a0 legacy=0x7f8964016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:21.485 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.498 7f8985fd8700 1 -- 172.21.15.102:0/4082983938 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f89800ca820 msgr2=0x7f89800cac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:21.485 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.498 7f8985fd8700 1 --2- 172.21.15.102:0/4082983938 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f89800ca820 0x7f89800cac70 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7f89800c4820 tx=0x7f8970006cd0).stop 2024-04-17T22:29:21.485 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.498 7f8985fd8700 1 -- 172.21.15.102:0/4082983938 shutdown_connections 2024-04-17T22:29:21.485 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.498 7f8985fd8700 1 --2- 172.21.15.102:0/4082983938 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f89801b08d0 0x7f89801b7330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:21.485 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.498 7f8985fd8700 1 --2- 172.21.15.102:0/4082983938 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f89800ca820 0x7f89800cac70 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:21.485 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.498 7f8985fd8700 1 --2- 172.21.15.102:0/4082983938 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f89801afe50 0x7f89801b02a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:21.485 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.498 7f8985fd8700 1 -- 172.21.15.102:0/4082983938 >> 172.21.15.102:0/4082983938 conn(0x7f89800c5cb0 msgr2=0x7f89800d6a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:29:21.485 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.498 7f8985fd8700 1 -- 172.21.15.102:0/4082983938 shutdown_connections 2024-04-17T22:29:21.485 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.498 7f8985fd8700 1 -- 172.21.15.102:0/4082983938 wait complete. 2024-04-17T22:29:21.495 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T22:29:21.619 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.633 7f590d254700 1 Processor -- start 2024-04-17T22:29:21.620 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.633 7f590d254700 1 -- start start 2024-04-17T22:29:21.622 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.634 7f590d254700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f590810e1d0 0x7f590810e620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:21.622 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.634 7f590d254700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f590810ec50 0x7f59080f16c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:21.622 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.634 7f590d254700 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59080f2140 con 0x7f59080f1cf0 2024-04-17T22:29:21.622 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.634 7f590d254700 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59080f9ea0 con 0x7f590810ec50 2024-04-17T22:29:21.622 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.634 7f590d254700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59080f8fc0 con 0x7f590810e1d0 2024-04-17T22:29:21.622 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.634 7f5906037700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f590810e1d0 0x7f590810e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:21.622 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.634 7f5906838700 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f59080f1cf0 0x7f59081aa460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:34818/0 (socket says 172.21.15.102:34818) 2024-04-17T22:29:21.622 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.634 7f5906838700 1 -- 172.21.15.102:0/1008041307 learned_addr learned my addr 172.21.15.102:0/1008041307 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-17T22:29:21.622 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.634 7f5905836700 1 --2- 172.21.15.102:0/1008041307 >> v2:172.21.15.102:3300/0 conn(0x7f590810ec50 0x7f59080f16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:21.622 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.634 7f5906037700 1 -- 172.21.15.102:0/1008041307 >> v2:172.21.15.102:3300/0 conn(0x7f590810ec50 msgr2=0x7f59080f16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:29:21.622 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.634 7f5906037700 1 --2- 172.21.15.102:0/1008041307 >> v2:172.21.15.102:3300/0 conn(0x7f590810ec50 0x7f59080f16c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:21.623 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.634 7f5906037700 1 -- 172.21.15.102:0/1008041307 >> v1:172.21.15.102:6789/0 conn(0x7f59080f1cf0 legacy=0x7f59081aa460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:21.623 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.634 7f5905035700 1 -- 172.21.15.102:0/1008041307 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2258424163 0 0) 0x7f5900001490 con 0x7f59080f1cf0 2024-04-17T22:29:21.623 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.634 7f5906037700 1 -- 172.21.15.102:0/1008041307 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59080f2360 con 0x7f590810e1d0 2024-04-17T22:29:21.623 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.634 7f5905836700 1 --2- 172.21.15.102:0/1008041307 >> v2:172.21.15.102:3300/0 conn(0x7f590810ec50 0x7f59080f16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:29:21.623 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.634 7f5906037700 1 --2- 172.21.15.102:0/1008041307 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f590810e1d0 0x7f590810e620 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7f58fc002f70 tx=0x7f58fc00b970).ready entity=mon.1 client_cookie=eab154c415a68f87 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:29:21.623 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.634 7f5905035700 1 -- 172.21.15.102:0/1008041307 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f59080f2360 con 0x7f590810e1d0 2024-04-17T22:29:21.623 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.634 7f5905035700 1 -- 172.21.15.102:0/1008041307 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58fc00d610 con 0x7f590810e1d0 2024-04-17T22:29:21.623 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.634 7f5905035700 1 -- 172.21.15.102:0/1008041307 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f58fc00d9e0 con 0x7f590810e1d0 2024-04-17T22:29:21.623 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.634 7f590d254700 1 -- 172.21.15.102:0/1008041307 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f590810e1d0 msgr2=0x7f590810e620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:21.623 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.634 7f590d254700 1 --2- 172.21.15.102:0/1008041307 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f590810e1d0 0x7f590810e620 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7f58fc002f70 tx=0x7f58fc00b970).stop 2024-04-17T22:29:21.623 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.635 7f590d254700 1 -- 172.21.15.102:0/1008041307 shutdown_connections 2024-04-17T22:29:21.623 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.635 7f590d254700 1 --2- 172.21.15.102:0/1008041307 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f590810e1d0 0x7f590810e620 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:21.623 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.635 7f590d254700 1 --2- 172.21.15.102:0/1008041307 >> v2:172.21.15.102:3300/0 conn(0x7f590810ec50 0x7f59080f16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:21.623 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.635 7f590d254700 1 -- 172.21.15.102:0/1008041307 >> 172.21.15.102:0/1008041307 conn(0x7f5908113830 msgr2=0x7f59080f4ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:29:21.624 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.635 7f590d254700 1 -- 172.21.15.102:0/1008041307 shutdown_connections 2024-04-17T22:29:21.624 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.635 7f590d254700 1 -- 172.21.15.102:0/1008041307 wait complete. 2024-04-17T22:29:21.646 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.657 7f590d254700 1 Processor -- start 2024-04-17T22:29:21.646 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.657 7f590d254700 1 -- start start 2024-04-17T22:29:21.646 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.658 7f590d254700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f59080decb0 0x7f59081ada10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:21.646 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.658 7f590d254700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f59081ae040 0x7f59081b4470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:21.646 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.658 7f590d254700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f59081ae490 0x7f59081b4aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:21.646 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.658 7f590d254700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59080fa140 con 0x7f59080decb0 2024-04-17T22:29:21.646 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.658 7f590d254700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59080df100 con 0x7f59081ae490 2024-04-17T22:29:21.646 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.658 7f590d254700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f59081b50d0 con 0x7f59081ae040 2024-04-17T22:29:21.646 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.658 7f5906037700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f59080decb0 0x7f59081ada10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:21.647 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.658 7f5906838700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f59081ae490 0x7f59081b4aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:21.647 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.658 7f5905836700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f59081ae040 0x7f59081b4470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:21.647 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.658 7f5906037700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f59080decb0 0x7f59081ada10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:41134/0 (socket says 172.21.15.102:41134) 2024-04-17T22:29:21.647 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.658 7f5906037700 1 -- 172.21.15.102:0/4013649161 learned_addr learned my addr 172.21.15.102:0/4013649161 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:29:21.647 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.658 7f5906037700 1 -- 172.21.15.102:0/4013649161 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f59081ae040 msgr2=0x7f59081b4470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:21.647 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.658 7f5906037700 1 --2- 172.21.15.102:0/4013649161 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f59081ae040 0x7f59081b4470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:21.647 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.658 7f5906037700 1 -- 172.21.15.102:0/4013649161 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f59081ae490 msgr2=0x7f59081b4aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:21.647 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.658 7f5906037700 1 --2- 172.21.15.102:0/4013649161 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f59081ae490 0x7f59081b4aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:21.647 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.658 7f5906037700 1 -- 172.21.15.102:0/4013649161 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59081b5300 con 0x7f59080decb0 2024-04-17T22:29:21.647 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.658 7f5905836700 1 --2- 172.21.15.102:0/4013649161 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f59081ae040 0x7f59081b4470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:29:21.647 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.658 7f5906838700 1 --2- 172.21.15.102:0/4013649161 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f59081ae490 0x7f59081b4aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:29:21.647 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.658 7f5906037700 1 --2- 172.21.15.102:0/4013649161 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f59080decb0 0x7f59081ada10 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 rx=0x7f58fc002f70 tx=0x7f58fc00a040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:29:21.647 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.658 7f58f77fe700 1 -- 172.21.15.102:0/4013649161 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f59081b5300 con 0x7f59080decb0 2024-04-17T22:29:21.647 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.658 7f58f77fe700 1 -- 172.21.15.102:0/4013649161 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58fc013a90 con 0x7f59080decb0 2024-04-17T22:29:21.648 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.658 7f590d254700 1 -- 172.21.15.102:0/4013649161 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59081bbb60 con 0x7f59080decb0 2024-04-17T22:29:21.648 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.658 7f58f77fe700 1 -- 172.21.15.102:0/4013649161 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f58fc01f450 con 0x7f59080decb0 2024-04-17T22:29:21.648 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.659 7f590d254700 1 -- 172.21.15.102:0/4013649161 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59081b5500 con 0x7f59080decb0 2024-04-17T22:29:21.648 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.659 7f58f77fe700 1 -- 172.21.15.102:0/4013649161 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7f58fc0283f0 con 0x7f59080decb0 2024-04-17T22:29:21.648 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.659 7f58f77fe700 1 -- 172.21.15.102:0/4013649161 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 3194+0+0 (secure 0 0 0) 0x7f58fc008c70 con 0x7f59080decb0 2024-04-17T22:29:21.648 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.660 7f590d254700 1 -- 172.21.15.102:0/4013649161 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59080f0790 con 0x7f59080decb0 2024-04-17T22:29:21.649 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.663 7f58f77fe700 1 -- 172.21.15.102:0/4013649161 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+110455 (secure 0 0 0) 0x7f58fc013880 con 0x7f59080decb0 2024-04-17T22:29:21.783 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.796 7f590d254700 1 -- 172.21.15.102:0/4013649161 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f59080abea0 con 0x7f59080decb0 2024-04-17T22:29:21.783 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.797 7f58f77fe700 1 -- 172.21.15.102:0/4013649161 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) v1 ==== 93+0+1704 (secure 0 0 0) 0x7f58fc00d400 con 0x7f59080decb0 2024-04-17T22:29:21.784 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-17T22:29:21.784 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":5,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[],"filesystems":[{"mdsmap":{"epoch":5,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17 22:29:07.787730","modified":"2024-04-17 22:29:09.795249","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"-1 (unspecified)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4177},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4177":{"gid":4177,"name":"a","rank":0,"incarnation":3,"state":"up:active","state_seq":4,"addr":"172.21.15.102:6817/1944542598","addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6817","nonce":1944542598}]},"export_targets":[],"features":4611087854035861503,"flags":0}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-04-17T22:29:21.785 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.798 7f590d254700 1 -- 172.21.15.102:0/4013649161 >> v1:172.21.15.102:6816/42360 conn(0x7f58e40145a0 legacy=0x7f58e4016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:21.785 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.798 7f590d254700 1 -- 172.21.15.102:0/4013649161 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f59080decb0 msgr2=0x7f59081ada10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:21.785 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.798 7f590d254700 1 --2- 172.21.15.102:0/4013649161 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f59080decb0 0x7f59081ada10 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 rx=0x7f58fc002f70 tx=0x7f58fc00a040).stop 2024-04-17T22:29:21.785 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.798 7f590d254700 1 -- 172.21.15.102:0/4013649161 shutdown_connections 2024-04-17T22:29:21.785 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.798 7f590d254700 1 --2- 172.21.15.102:0/4013649161 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f59081ae490 0x7f59081b4aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:21.785 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.798 7f590d254700 1 --2- 172.21.15.102:0/4013649161 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f59080decb0 0x7f59081ada10 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:21.785 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.798 7f590d254700 1 --2- 172.21.15.102:0/4013649161 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f59081ae040 0x7f59081b4470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:21.785 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.798 7f590d254700 1 -- 172.21.15.102:0/4013649161 >> 172.21.15.102:0/4013649161 conn(0x7f59080c9260 msgr2=0x7f59080d7330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:29:21.785 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.798 7f590d254700 1 -- 172.21.15.102:0/4013649161 shutdown_connections 2024-04-17T22:29:21.785 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.798 7f590d254700 1 -- 172.21.15.102:0/4013649161 wait complete. 2024-04-17T22:29:21.786 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 5 2024-04-17T22:29:21.795 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T22:29:21.921 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.933 7f91e3985700 1 Processor -- start 2024-04-17T22:29:21.921 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.933 7f91e3985700 1 -- start start 2024-04-17T22:29:21.922 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.933 7f91e3985700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f91dc0f56e0 0x7f91dc10c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:21.922 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.933 7f91e3985700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f91dc10cb00 0x7f91dc1a1370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:21.922 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.933 7f91e3985700 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91dc1a19a0 con 0x7f91dc10c6b0 2024-04-17T22:29:21.922 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.933 7f91e3985700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91dc0f9350 con 0x7f91dc10cb00 2024-04-17T22:29:21.922 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.933 7f91e3985700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f91dc1a1bc0 con 0x7f91dc0f56e0 2024-04-17T22:29:21.922 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.933 7f91dbfff700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f91dc0f56e0 0x7f91dc10c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:21.922 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.933 7f91e0f81700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f91dc10cb00 0x7f91dc1a1370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:21.922 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.933 7f91d3fff700 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f91dc10c6b0 0x7f91dc1a0b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:34834/0 (socket says 172.21.15.102:34834) 2024-04-17T22:29:21.922 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.933 7f91d3fff700 1 -- 172.21.15.102:0/1654417217 learned_addr learned my addr 172.21.15.102:0/1654417217 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-17T22:29:21.922 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.933 7f91db7fe700 1 -- 172.21.15.102:0/1654417217 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 713518734 0 0) 0x7f91c4001110 con 0x7f91dc10c6b0 2024-04-17T22:29:21.922 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.934 7f91e0f81700 1 -- 172.21.15.102:0/1654417217 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f91dc0f56e0 msgr2=0x7f91dc10c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:29:21.922 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.934 7f91e0f81700 1 --2- 172.21.15.102:0/1654417217 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f91dc0f56e0 0x7f91dc10c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:21.922 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.934 7f91e0f81700 1 -- 172.21.15.102:0/1654417217 >> v1:172.21.15.102:6789/0 conn(0x7f91dc10c6b0 legacy=0x7f91dc1a0b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:21.922 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.934 7f91e0f81700 1 -- 172.21.15.102:0/1654417217 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91dc1aa2a0 con 0x7f91dc10cb00 2024-04-17T22:29:21.923 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.934 7f91dbfff700 1 --2- 172.21.15.102:0/1654417217 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f91dc0f56e0 0x7f91dc10c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:29:21.923 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.934 7f91e0f81700 1 --2- 172.21.15.102:0/1654417217 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f91dc10cb00 0x7f91dc1a1370 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7f91dc0f4400 tx=0x7f91d4009d70).ready entity=mon.1 client_cookie=adcbe65093ce843b server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:29:21.923 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.934 7f91db7fe700 1 -- 172.21.15.102:0/1654417217 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f91dc1aa2a0 con 0x7f91dc10cb00 2024-04-17T22:29:21.923 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.934 7f91db7fe700 1 -- 172.21.15.102:0/1654417217 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91d400b7c0 con 0x7f91dc10cb00 2024-04-17T22:29:21.923 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.934 7f91db7fe700 1 -- 172.21.15.102:0/1654417217 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f91d400b9a0 con 0x7f91dc10cb00 2024-04-17T22:29:21.923 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.934 7f91e3985700 1 -- 172.21.15.102:0/1654417217 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f91dc10cb00 msgr2=0x7f91dc1a1370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:21.923 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.934 7f91e3985700 1 --2- 172.21.15.102:0/1654417217 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f91dc10cb00 0x7f91dc1a1370 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7f91dc0f4400 tx=0x7f91d4009d70).stop 2024-04-17T22:29:21.923 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.934 7f91e3985700 1 -- 172.21.15.102:0/1654417217 shutdown_connections 2024-04-17T22:29:21.923 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.934 7f91e3985700 1 --2- 172.21.15.102:0/1654417217 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f91dc10cb00 0x7f91dc1a1370 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:21.923 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.934 7f91e3985700 1 --2- 172.21.15.102:0/1654417217 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f91dc0f56e0 0x7f91dc10c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:21.923 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.934 7f91e3985700 1 -- 172.21.15.102:0/1654417217 >> 172.21.15.102:0/1654417217 conn(0x7f91dc113830 msgr2=0x7f91dc0f4ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:29:21.923 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.935 7f91e3985700 1 -- 172.21.15.102:0/1654417217 shutdown_connections 2024-04-17T22:29:21.923 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.935 7f91e3985700 1 -- 172.21.15.102:0/1654417217 wait complete. 2024-04-17T22:29:21.946 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.957 7f91e3985700 1 Processor -- start 2024-04-17T22:29:21.946 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.957 7f91e3985700 1 -- start start 2024-04-17T22:29:21.946 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.958 7f91e3985700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f91dc1ad530 0x7f91dc1ad980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:21.946 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.958 7f91e3985700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f91dc1adfb0 0x7f91dc1b3de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:21.946 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.958 7f91e3985700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f91dc1b4410 0x7f91dc1b4860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:21.946 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.958 7f91e3985700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91dc0878f0 con 0x7f91dc1b4410 2024-04-17T22:29:21.946 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.958 7f91e3985700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91dc0f72c0 con 0x7f91dc1adfb0 2024-04-17T22:29:21.946 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.958 7f91e3985700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f91dc0f8170 con 0x7f91dc1ad530 2024-04-17T22:29:21.946 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.958 7f91dbfff700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f91dc1ad530 0x7f91dc1ad980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:21.946 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.958 7f91dbfff700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f91dc1ad530 0x7f91dc1ad980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:44584/0 (socket says 172.21.15.102:44584) 2024-04-17T22:29:21.946 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.958 7f91d3fff700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f91dc1adfb0 0x7f91dc1b3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:21.946 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.958 7f91dbfff700 1 -- 172.21.15.102:0/1637806903 learned_addr learned my addr 172.21.15.102:0/1637806903 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:29:21.947 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.958 7f91e0f81700 1 --2- 172.21.15.102:0/1637806903 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f91dc1b4410 0x7f91dc1b4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:21.947 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.958 7f91dbfff700 1 -- 172.21.15.102:0/1637806903 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f91dc1adfb0 msgr2=0x7f91dc1b3de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:21.947 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.958 7f91dbfff700 1 --2- 172.21.15.102:0/1637806903 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f91dc1adfb0 0x7f91dc1b3de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:21.947 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.958 7f91dbfff700 1 -- 172.21.15.102:0/1637806903 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f91dc1b4410 msgr2=0x7f91dc1b4860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:21.947 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.958 7f91dbfff700 1 --2- 172.21.15.102:0/1637806903 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f91dc1b4410 0x7f91dc1b4860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:21.947 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.958 7f91dbfff700 1 -- 172.21.15.102:0/1637806903 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91dc0c4470 con 0x7f91dc1ad530 2024-04-17T22:29:21.947 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.958 7f91e0f81700 1 --2- 172.21.15.102:0/1637806903 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f91dc1b4410 0x7f91dc1b4860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:29:21.947 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.958 7f91dbfff700 1 --2- 172.21.15.102:0/1637806903 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f91dc1ad530 0x7f91dc1ad980 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7f91dc0c4cb0 tx=0x7f91cc00bf70).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:29:21.947 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.958 7f91d9ffb700 1 -- 172.21.15.102:0/1637806903 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f91cc00f040 con 0x7f91dc1ad530 2024-04-17T22:29:21.947 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.958 7f91d9ffb700 1 -- 172.21.15.102:0/1637806903 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91cc00c870 con 0x7f91dc1ad530 2024-04-17T22:29:21.947 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.958 7f91e3985700 1 -- 172.21.15.102:0/1637806903 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91dc1a6b70 con 0x7f91dc1ad530 2024-04-17T22:29:21.947 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.958 7f91d9ffb700 1 -- 172.21.15.102:0/1637806903 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f91cc00cc40 con 0x7f91dc1ad530 2024-04-17T22:29:21.947 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.959 7f91e3985700 1 -- 172.21.15.102:0/1637806903 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91dc1bb5e0 con 0x7f91dc1ad530 2024-04-17T22:29:21.947 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.959 7f91d9ffb700 1 -- 172.21.15.102:0/1637806903 <== mon.2 v2:172.21.15.102:3302/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7f91cc011070 con 0x7f91dc1ad530 2024-04-17T22:29:21.948 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.959 7f91d9ffb700 1 -- 172.21.15.102:0/1637806903 <== mon.2 v2:172.21.15.102:3302/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 3194+0+0 (secure 0 0 0) 0x7f91cc010040 con 0x7f91dc1ad530 2024-04-17T22:29:21.948 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.959 7f91e3985700 1 -- 172.21.15.102:0/1637806903 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91dc0f0790 con 0x7f91dc1ad530 2024-04-17T22:29:21.949 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:21.962 7f91d9ffb700 1 -- 172.21.15.102:0/1637806903 <== mon.2 v2:172.21.15.102:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+110455 (secure 0 0 0) 0x7f91cc00c470 con 0x7f91dc1ad530 2024-04-17T22:29:22.054 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.068 7f91e3985700 1 -- 172.21.15.102:0/1637806903 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f91dc0abea0 con 0x7f91dc1ad530 2024-04-17T22:29:22.055 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.068 7f91d9ffb700 1 -- 172.21.15.102:0/1637806903 <== mon.2 v2:172.21.15.102:3302/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+8761 (secure 0 0 0) 0x7f91cc000c30 con 0x7f91dc1ad530 2024-04-17T22:29:22.055 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-17T22:29:22.055 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":16,"fsid":"cb63e789-091d-49b1-b120-55f9db526871","created":"2024-04-17 22:28:48.209635","modified":"2024-04-17 22:29:08.789044","last_up_change":"2024-04-17 22:28:54.236215","last_in_change":"2024-04-17 22:28:54.236215","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":4,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-04-17 22:28:57.478545","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-17 22:29:06.144708","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-17 22:29:06.748924","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3351ef85-ade4-4b90-a8f4-213f4e00ba01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6812","nonce":42955}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6813","nonce":42955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6815","nonce":42955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6814","nonce":42955}]},"public_addr":"172.21.15.102:6812/42955","cluster_addr":"172.21.15.102:6813/42955","heartbeat_back_addr":"172.21.15.102:6815/42955","heartbeat_front_addr":"172.21.15.102:6814/42955","state":["exists","up"]},{"osd":1,"uuid":"04f67eac-f36b-48f5-982d-4cfc776596d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6808","nonce":42958}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6809","nonce":42958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6811","nonce":42958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6810","nonce":42958}]},"public_addr":"172.21.15.102:6808/42958","cluster_addr":"172.21.15.102:6809/42958","heartbeat_back_addr":"172.21.15.102:6811/42958","heartbeat_front_addr":"172.21.15.102:6810/42958","state":["exists","up"]},{"osd":2,"uuid":"d524204b-70b2-40a8-8f73-f9e7d44318af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6804","nonce":42954}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6805","nonce":42954}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6807","nonce":42954}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6806","nonce":42954}]},"public_addr":"172.21.15.102:6804/42954","cluster_addr":"172.21.15.102:6805/42954","heartbeat_back_addr":"172.21.15.102:6807/42954","heartbeat_front_addr":"172.21.15.102:6806/42954","state":["exists","up"]},{"osd":3,"uuid":"4fc9d20c-1ad0-484f-aafa-2c9a7574245b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6800","nonce":42953}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6801","nonce":42953}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6803","nonce":42953}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6802","nonce":42953}]},"public_addr":"172.21.15.102:6800/42953","cluster_addr":"172.21.15.102:6801/42953","heartbeat_back_addr":"172.21.15.102:6803/42953","heartbeat_front_addr":"172.21.15.102:6802/42953","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2024-04-17T22:29:22.056 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.069 7f91e3985700 1 -- 172.21.15.102:0/1637806903 >> v1:172.21.15.102:6816/42360 conn(0x7f91c00145a0 legacy=0x7f91c0016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:22.056 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.069 7f91e3985700 1 -- 172.21.15.102:0/1637806903 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f91dc1ad530 msgr2=0x7f91dc1ad980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:22.057 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.069 7f91e3985700 1 --2- 172.21.15.102:0/1637806903 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f91dc1ad530 0x7f91dc1ad980 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7f91dc0c4cb0 tx=0x7f91cc00bf70).stop 2024-04-17T22:29:22.057 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.069 7f91e3985700 1 -- 172.21.15.102:0/1637806903 shutdown_connections 2024-04-17T22:29:22.057 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.069 7f91e3985700 1 --2- 172.21.15.102:0/1637806903 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f91dc1b4410 0x7f91dc1b4860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:22.057 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.069 7f91e3985700 1 --2- 172.21.15.102:0/1637806903 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f91dc1ad530 0x7f91dc1ad980 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:22.057 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.069 7f91e3985700 1 --2- 172.21.15.102:0/1637806903 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f91dc1adfb0 0x7f91dc1b3de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:22.057 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.070 7f91e3985700 1 -- 172.21.15.102:0/1637806903 >> 172.21.15.102:0/1637806903 conn(0x7f91dc0cf430 msgr2=0x7f91dc0c2f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:29:22.057 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.070 7f91e3985700 1 -- 172.21.15.102:0/1637806903 shutdown_connections 2024-04-17T22:29:22.057 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.070 7f91e3985700 1 -- 172.21.15.102:0/1637806903 wait complete. 2024-04-17T22:29:22.067 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T22:29:22.194 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.205 7efd811c6700 1 Processor -- start 2024-04-17T22:29:22.194 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.205 7efd811c6700 1 -- start start 2024-04-17T22:29:22.194 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.205 7efd811c6700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7efd7c10e260 0x7efd7c10e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:22.194 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.205 7efd811c6700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7efd7c10ece0 0x7efd7c0f1750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:22.194 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.205 7efd811c6700 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd7c0f21d0 con 0x7efd7c0f1d80 2024-04-17T22:29:22.194 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.205 7efd811c6700 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd7c0f9f30 con 0x7efd7c10e260 2024-04-17T22:29:22.194 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.205 7efd811c6700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd7c0f9050 con 0x7efd7c10ece0 2024-04-17T22:29:22.194 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.205 7efd73fff700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7efd7c10ece0 0x7efd7c0f1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:22.194 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.205 7efd79dfc700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7efd7c10e260 0x7efd7c10e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:22.195 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.205 7efd79dfc700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7efd7c10e260 0x7efd7c10e6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:41164/0 (socket says 172.21.15.102:41164) 2024-04-17T22:29:22.195 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.205 7efd73fff700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7efd7c10ece0 0x7efd7c0f1750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:40624/0 (socket says 172.21.15.102:40624) 2024-04-17T22:29:22.195 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.205 7efd7a5fd700 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7efd7c0f1d80 0x7efd7c1aa4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:34850/0 (socket says 172.21.15.102:34850) 2024-04-17T22:29:22.195 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.205 7efd79dfc700 1 -- 172.21.15.102:0/3618646904 learned_addr learned my addr 172.21.15.102:0/3618646904 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:29:22.195 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.206 7efd795fb700 1 -- 172.21.15.102:0/3618646904 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1181276903 0 0) 0x7efd74001490 con 0x7efd7c0f1d80 2024-04-17T22:29:22.195 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.206 7efd795fb700 1 -- 172.21.15.102:0/3618646904 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efd5c003270 con 0x7efd7c0f1d80 2024-04-17T22:29:22.195 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.206 7efd73fff700 1 -- 172.21.15.102:0/3618646904 >> v2:172.21.15.102:3300/0 conn(0x7efd7c10e260 msgr2=0x7efd7c10e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:29:22.195 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.206 7efd73fff700 1 --2- 172.21.15.102:0/3618646904 >> v2:172.21.15.102:3300/0 conn(0x7efd7c10e260 0x7efd7c10e6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:22.195 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.206 7efd73fff700 1 -- 172.21.15.102:0/3618646904 >> v1:172.21.15.102:6789/0 conn(0x7efd7c0f1d80 legacy=0x7efd7c1aa4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:22.195 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.206 7efd73fff700 1 -- 172.21.15.102:0/3618646904 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd7c0f23f0 con 0x7efd7c10ece0 2024-04-17T22:29:22.195 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.206 7efd79dfc700 1 --2- 172.21.15.102:0/3618646904 >> v2:172.21.15.102:3300/0 conn(0x7efd7c10e260 0x7efd7c10e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:29:22.195 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.206 7efd73fff700 1 --2- 172.21.15.102:0/3618646904 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7efd7c10ece0 0x7efd7c0f1750 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7efd64002f70 tx=0x7efd6400b970).ready entity=mon.1 client_cookie=8b572e5663ec02fa server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:29:22.196 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.206 7efd795fb700 1 -- 172.21.15.102:0/3618646904 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7efd7c0f23f0 con 0x7efd7c10ece0 2024-04-17T22:29:22.196 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.206 7efd795fb700 1 -- 172.21.15.102:0/3618646904 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd6400d610 con 0x7efd7c10ece0 2024-04-17T22:29:22.196 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.206 7efd795fb700 1 -- 172.21.15.102:0/3618646904 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7efd6400d9e0 con 0x7efd7c10ece0 2024-04-17T22:29:22.196 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.206 7efd811c6700 1 -- 172.21.15.102:0/3618646904 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7efd7c10ece0 msgr2=0x7efd7c0f1750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:22.196 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.206 7efd811c6700 1 --2- 172.21.15.102:0/3618646904 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7efd7c10ece0 0x7efd7c0f1750 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7efd64002f70 tx=0x7efd6400b970).stop 2024-04-17T22:29:22.196 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.206 7efd811c6700 1 -- 172.21.15.102:0/3618646904 shutdown_connections 2024-04-17T22:29:22.196 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.206 7efd811c6700 1 --2- 172.21.15.102:0/3618646904 >> v2:172.21.15.102:3300/0 conn(0x7efd7c10e260 0x7efd7c10e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:22.196 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.206 7efd811c6700 1 --2- 172.21.15.102:0/3618646904 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7efd7c10ece0 0x7efd7c0f1750 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:22.196 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.207 7efd811c6700 1 -- 172.21.15.102:0/3618646904 >> 172.21.15.102:0/3618646904 conn(0x7efd7c1138c0 msgr2=0x7efd7c0f4f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:29:22.196 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.207 7efd811c6700 1 -- 172.21.15.102:0/3618646904 shutdown_connections 2024-04-17T22:29:22.196 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.207 7efd811c6700 1 -- 172.21.15.102:0/3618646904 wait complete. 2024-04-17T22:29:22.219 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.229 7efd811c6700 1 Processor -- start 2024-04-17T22:29:22.219 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.229 7efd811c6700 1 -- start start 2024-04-17T22:29:22.219 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.229 7efd811c6700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efd7c0cc360 0x7efd7c0d51d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:22.219 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.230 7efd811c6700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7efd7c0d5800 0x7efd7c1b42b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:22.219 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.230 7efd811c6700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7efd7c0d5c50 0x7efd7c1b48e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:22.219 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.230 7efd811c6700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd7c08f2e0 con 0x7efd7c0cc360 2024-04-17T22:29:22.219 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.230 7efd811c6700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd7c0fa1d0 con 0x7efd7c0d5c50 2024-04-17T22:29:22.219 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.230 7efd811c6700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7efd7c0dac10 con 0x7efd7c0d5800 2024-04-17T22:29:22.219 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.230 7efd79dfc700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efd7c0cc360 0x7efd7c0d51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:22.219 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.230 7efd7a5fd700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7efd7c0d5c50 0x7efd7c1b48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:22.220 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.230 7efd79dfc700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efd7c0cc360 0x7efd7c0d51d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:41170/0 (socket says 172.21.15.102:41170) 2024-04-17T22:29:22.220 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.230 7efd79dfc700 1 -- 172.21.15.102:0/3736010824 learned_addr learned my addr 172.21.15.102:0/3736010824 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:29:22.220 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.230 7efd73fff700 1 --2- 172.21.15.102:0/3736010824 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7efd7c0d5800 0x7efd7c1b42b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:22.220 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.230 7efd79dfc700 1 -- 172.21.15.102:0/3736010824 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7efd7c0d5800 msgr2=0x7efd7c1b42b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:22.220 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.230 7efd79dfc700 1 --2- 172.21.15.102:0/3736010824 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7efd7c0d5800 0x7efd7c1b42b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:22.220 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.230 7efd79dfc700 1 -- 172.21.15.102:0/3736010824 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7efd7c0d5c50 msgr2=0x7efd7c1b48e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:22.220 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.230 7efd79dfc700 1 --2- 172.21.15.102:0/3736010824 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7efd7c0d5c50 0x7efd7c1b48e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:22.220 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.230 7efd79dfc700 1 -- 172.21.15.102:0/3736010824 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd7c0cbca0 con 0x7efd7c0cc360 2024-04-17T22:29:22.220 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.230 7efd73fff700 1 --2- 172.21.15.102:0/3736010824 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7efd7c0d5800 0x7efd7c1b42b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:29:22.220 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.230 7efd7a5fd700 1 --2- 172.21.15.102:0/3736010824 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7efd7c0d5c50 0x7efd7c1b48e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:29:22.220 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.230 7efd79dfc700 1 --2- 172.21.15.102:0/3736010824 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efd7c0cc360 0x7efd7c0d51d0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 rx=0x7efd7c0cbc20 tx=0x7efd6c00ed30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:29:22.220 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.230 7efd72ffd700 1 -- 172.21.15.102:0/3736010824 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7efd7c0cbca0 con 0x7efd7c0cc360 2024-04-17T22:29:22.220 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.230 7efd72ffd700 1 -- 172.21.15.102:0/3736010824 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd6c00c7b0 con 0x7efd7c0cc360 2024-04-17T22:29:22.220 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.230 7efd811c6700 1 -- 172.21.15.102:0/3736010824 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd7c1b4f10 con 0x7efd7c0cc360 2024-04-17T22:29:22.221 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.230 7efd72ffd700 1 -- 172.21.15.102:0/3736010824 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7efd6c00f040 con 0x7efd7c0cc360 2024-04-17T22:29:22.221 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.230 7efd811c6700 1 -- 172.21.15.102:0/3736010824 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd7c1bbc00 con 0x7efd7c0cc360 2024-04-17T22:29:22.221 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.231 7efd72ffd700 1 -- 172.21.15.102:0/3736010824 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7efd6c011070 con 0x7efd7c0cc360 2024-04-17T22:29:22.221 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.231 7efd72ffd700 1 -- 172.21.15.102:0/3736010824 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 3194+0+0 (secure 0 0 0) 0x7efd6c010040 con 0x7efd7c0cc360 2024-04-17T22:29:22.221 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.231 7efd811c6700 1 -- 172.21.15.102:0/3736010824 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd7c0f0820 con 0x7efd7c0cc360 2024-04-17T22:29:22.221 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.235 7efd72ffd700 1 -- 172.21.15.102:0/3736010824 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+110455 (secure 0 0 0) 0x7efd6c00c5a0 con 0x7efd7c0cc360 2024-04-17T22:29:22.357 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.371 7efd811c6700 1 -- 172.21.15.102:0/3736010824 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efd7c0f5550 con 0x7efd7c0cc360 2024-04-17T22:29:22.358 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.371 7efd72ffd700 1 -- 172.21.15.102:0/3736010824 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) v1 ==== 93+0+1704 (secure 0 0 0) 0x7efd6c000c30 con 0x7efd7c0cc360 2024-04-17T22:29:22.358 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-17T22:29:22.358 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":5,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[],"filesystems":[{"mdsmap":{"epoch":5,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17 22:29:07.787730","modified":"2024-04-17 22:29:09.795249","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"-1 (unspecified)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4177},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4177":{"gid":4177,"name":"a","rank":0,"incarnation":3,"state":"up:active","state_seq":4,"addr":"172.21.15.102:6817/1944542598","addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6817","nonce":1944542598}]},"export_targets":[],"features":4611087854035861503,"flags":0}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-04-17T22:29:22.359 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.372 7efd811c6700 1 -- 172.21.15.102:0/3736010824 >> v1:172.21.15.102:6816/42360 conn(0x7efd600145a0 legacy=0x7efd60016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:22.359 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.372 7efd811c6700 1 -- 172.21.15.102:0/3736010824 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efd7c0cc360 msgr2=0x7efd7c0d51d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:22.359 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.372 7efd811c6700 1 --2- 172.21.15.102:0/3736010824 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efd7c0cc360 0x7efd7c0d51d0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 rx=0x7efd7c0cbc20 tx=0x7efd6c00ed30).stop 2024-04-17T22:29:22.359 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.372 7efd811c6700 1 -- 172.21.15.102:0/3736010824 shutdown_connections 2024-04-17T22:29:22.359 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.372 7efd811c6700 1 --2- 172.21.15.102:0/3736010824 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7efd7c0d5c50 0x7efd7c1b48e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:22.359 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.372 7efd811c6700 1 --2- 172.21.15.102:0/3736010824 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efd7c0cc360 0x7efd7c0d51d0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:22.359 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.372 7efd811c6700 1 --2- 172.21.15.102:0/3736010824 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7efd7c0d5800 0x7efd7c1b42b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:22.360 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.372 7efd811c6700 1 -- 172.21.15.102:0/3736010824 >> 172.21.15.102:0/3736010824 conn(0x7efd7c0c5690 msgr2=0x7efd7c0d62b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:29:22.360 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.372 7efd811c6700 1 -- 172.21.15.102:0/3736010824 shutdown_connections 2024-04-17T22:29:22.360 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.372 7efd811c6700 1 -- 172.21.15.102:0/3736010824 wait complete. 2024-04-17T22:29:22.360 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 5 2024-04-17T22:29:22.470 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 5, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-17 22:29:07.787730', 'modified': '2024-04-17 22:29:09.795249', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '-1 (unspecified)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4177}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4177': {'gid': 4177, 'name': 'a', 'rank': 0, 'incarnation': 3, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.102:6817/1944542598', 'addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.102:6817', 'nonce': 1944542598}]}, 'export_targets': [], 'features': 4611087854035861503, 'flags': 0}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2024-04-17T22:29:22.470 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-17T22:29:22.471 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-17T22:29:22.598 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.610 7f4538efd700 1 Processor -- start 2024-04-17T22:29:22.599 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.610 7f4538efd700 1 -- start start 2024-04-17T22:29:22.599 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.610 7f4538efd700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f453410e1d0 0x7f453410e620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:22.599 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.610 7f4538efd700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f453410ec50 0x7f45340f16c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:22.599 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.610 7f4538efd700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f45340f1cf0 0x7f45341aa460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:22.599 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.610 7f4538efd700 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45340f9ea0 con 0x7f453410ec50 2024-04-17T22:29:22.599 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.610 7f4538efd700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45340f8fc0 con 0x7f453410e1d0 2024-04-17T22:29:22.599 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.610 7f4538efd700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f45340fa140 con 0x7f45340f1cf0 2024-04-17T22:29:22.599 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.610 7f4532365700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f45340f1cf0 0x7f45341aa460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:22.599 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.610 7f4531363700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f453410ec50 0x7f45340f16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:22.599 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.610 7f4532365700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f45340f1cf0 0x7f45341aa460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:44594/0 (socket says 172.21.15.102:44594) 2024-04-17T22:29:22.599 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.610 7f4531b64700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f453410e1d0 0x7f453410e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:22.599 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.610 7f4532365700 1 -- 172.21.15.102:0/548804560 learned_addr learned my addr 172.21.15.102:0/548804560 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:29:22.599 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.611 7f4531363700 1 -- 172.21.15.102:0/548804560 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f45340f1cf0 msgr2=0x7f45341aa460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:29:22.599 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.611 7f4531363700 1 --2- 172.21.15.102:0/548804560 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f45340f1cf0 0x7f45341aa460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:22.600 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.611 7f4531363700 1 -- 172.21.15.102:0/548804560 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f453410e1d0 msgr2=0x7f453410e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:29:22.600 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.611 7f4531363700 1 --2- 172.21.15.102:0/548804560 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f453410e1d0 0x7f453410e620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:22.600 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.611 7f4531363700 1 -- 172.21.15.102:0/548804560 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45340f2140 con 0x7f453410ec50 2024-04-17T22:29:22.600 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.611 7f4532365700 1 --2- 172.21.15.102:0/548804560 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f45340f1cf0 0x7f45341aa460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:29:22.600 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.611 7f4531b64700 1 --2- 172.21.15.102:0/548804560 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f453410e1d0 0x7f453410e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:29:22.600 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.611 7f4531363700 1 --2- 172.21.15.102:0/548804560 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f453410ec50 0x7f45340f16c0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 rx=0x7f45340f4630 tx=0x7f452800ba50).ready entity=mon.0 client_cookie=c047a01abba52ce1 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:29:22.600 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.611 7f4530b62700 1 -- 172.21.15.102:0/548804560 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f45340f2140 con 0x7f453410ec50 2024-04-17T22:29:22.600 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.611 7f4530b62700 1 -- 172.21.15.102:0/548804560 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f452800d570 con 0x7f453410ec50 2024-04-17T22:29:22.600 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.611 7f4530b62700 1 -- 172.21.15.102:0/548804560 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f452800d940 con 0x7f453410ec50 2024-04-17T22:29:22.600 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.611 7f4538efd700 1 -- 172.21.15.102:0/548804560 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f453410ec50 msgr2=0x7f45340f16c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:22.600 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.611 7f4538efd700 1 --2- 172.21.15.102:0/548804560 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f453410ec50 0x7f45340f16c0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 rx=0x7f45340f4630 tx=0x7f452800ba50).stop 2024-04-17T22:29:22.600 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.612 7f4538efd700 1 -- 172.21.15.102:0/548804560 shutdown_connections 2024-04-17T22:29:22.600 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.612 7f4538efd700 1 --2- 172.21.15.102:0/548804560 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f45340f1cf0 0x7f45341aa460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:22.600 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.612 7f4538efd700 1 --2- 172.21.15.102:0/548804560 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f453410e1d0 0x7f453410e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:22.601 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.612 7f4538efd700 1 --2- 172.21.15.102:0/548804560 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f453410ec50 0x7f45340f16c0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:22.601 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.612 7f4538efd700 1 -- 172.21.15.102:0/548804560 >> 172.21.15.102:0/548804560 conn(0x7f4534113830 msgr2=0x7f45340f4ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:29:22.601 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.612 7f4538efd700 1 -- 172.21.15.102:0/548804560 shutdown_connections 2024-04-17T22:29:22.601 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.612 7f4538efd700 1 -- 172.21.15.102:0/548804560 wait complete. 2024-04-17T22:29:22.623 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.634 7f4538efd700 1 Processor -- start 2024-04-17T22:29:22.623 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.634 7f4538efd700 1 -- start start 2024-04-17T22:29:22.623 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.635 7f4538efd700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f45340cf290 0x7f45341ad230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:22.623 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.635 7f4538efd700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f45341ad860 0x7f45341b3c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:22.623 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.635 7f4538efd700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f45341adcb0 0x7f45341b4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:22.623 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.635 7f4538efd700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4534089940 con 0x7f45340cf290 2024-04-17T22:29:22.623 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.635 7f4538efd700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45340c4140 con 0x7f45341adcb0 2024-04-17T22:29:22.624 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.635 7f4538efd700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f45340cd420 con 0x7f45341ad860 2024-04-17T22:29:22.624 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.635 7f4531b64700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f45340cf290 0x7f45341ad230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:22.624 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.635 7f4531b64700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f45340cf290 0x7f45341ad230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:41186/0 (socket says 172.21.15.102:41186) 2024-04-17T22:29:22.624 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.635 7f4531b64700 1 -- 172.21.15.102:0/3870686672 learned_addr learned my addr 172.21.15.102:0/3870686672 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:29:22.624 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.635 7f4532365700 1 --2- 172.21.15.102:0/3870686672 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f45341adcb0 0x7f45341b4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:22.624 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.635 7f4531363700 1 --2- 172.21.15.102:0/3870686672 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f45341ad860 0x7f45341b3c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:22.624 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.635 7f4532365700 1 -- 172.21.15.102:0/3870686672 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f45341ad860 msgr2=0x7f45341b3c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:22.624 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.635 7f4532365700 1 --2- 172.21.15.102:0/3870686672 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f45341ad860 0x7f45341b3c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:22.624 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.635 7f4532365700 1 -- 172.21.15.102:0/3870686672 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f45340cf290 msgr2=0x7f45341ad230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:22.624 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.635 7f4532365700 1 --2- 172.21.15.102:0/3870686672 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f45340cf290 0x7f45341ad230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:22.624 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.635 7f4532365700 1 -- 172.21.15.102:0/3870686672 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45340cf6e0 con 0x7f45341adcb0 2024-04-17T22:29:22.624 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.635 7f4531b64700 1 --2- 172.21.15.102:0/3870686672 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f45340cf290 0x7f45341ad230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:29:22.624 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.635 7f4531363700 1 --2- 172.21.15.102:0/3870686672 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f45341ad860 0x7f45341b3c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:29:22.624 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.635 7f4532365700 1 --2- 172.21.15.102:0/3870686672 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f45341adcb0 0x7f45341b4240 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0x7f45340c5960 tx=0x7f452c00ea50).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:29:22.625 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.635 7f4522ffd700 1 -- 172.21.15.102:0/3870686672 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f452c00f040 con 0x7f45341adcb0 2024-04-17T22:29:22.625 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.635 7f4522ffd700 1 -- 172.21.15.102:0/3870686672 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f452c012600 con 0x7f45341adcb0 2024-04-17T22:29:22.625 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.635 7f4538efd700 1 -- 172.21.15.102:0/3870686672 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45341b4870 con 0x7f45341adcb0 2024-04-17T22:29:22.625 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.635 7f4522ffd700 1 -- 172.21.15.102:0/3870686672 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f452c0129d0 con 0x7f45341adcb0 2024-04-17T22:29:22.625 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.636 7f4538efd700 1 -- 172.21.15.102:0/3870686672 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45341bb300 con 0x7f45341adcb0 2024-04-17T22:29:22.625 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.636 7f4522ffd700 1 -- 172.21.15.102:0/3870686672 <== mon.1 v2:172.21.15.102:3301/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7f452c022060 con 0x7f45341adcb0 2024-04-17T22:29:22.625 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.636 7f4522ffd700 1 -- 172.21.15.102:0/3870686672 <== mon.1 v2:172.21.15.102:3301/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 3194+0+0 (secure 0 0 0) 0x7f452c010040 con 0x7f45341adcb0 2024-04-17T22:29:22.632 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.639 7f454b7b6b80 1 Processor -- start 2024-04-17T22:29:22.632 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.639 7f454b7b6b80 1 -- start start 2024-04-17T22:29:22.632 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.639 7f454b7b6b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562b8e986220 0x562b8e986670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:22.632 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.639 7f454b7b6b80 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x562b8e986ca0 0x562b8e9798e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:22.632 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.639 7f454b7b6b80 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x562b8e979f10 0x562b8e97a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:22.632 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.639 7f454b7b6b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b8e8b1c80 con 0x562b8e986220 2024-04-17T22:29:22.632 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.639 7f454b7b6b80 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x562b8e8cfa40 con 0x562b8e979f10 2024-04-17T22:29:22.632 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.639 7f454b7b6b80 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x562b8e89ce00 con 0x562b8e986ca0 2024-04-17T22:29:22.632 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.639 7f4531b64700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562b8e986220 0x562b8e986670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:22.632 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.639 7f4531363700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x562b8e986ca0 0x562b8e9798e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:22.632 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.639 7f4531b64700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562b8e986220 0x562b8e986670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:41192/0 (socket says 172.21.15.102:41192) 2024-04-17T22:29:22.632 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.639 7f4531b64700 1 -- 172.21.15.102:0/816543078 learned_addr learned my addr 172.21.15.102:0/816543078 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:29:22.632 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.639 7f4532365700 1 --2- 172.21.15.102:0/816543078 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x562b8e979f10 0x562b8e97a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:22.633 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.640 7f4531363700 1 -- 172.21.15.102:0/816543078 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x562b8e979f10 msgr2=0x562b8e97a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:29:22.633 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.640 7f4531363700 1 --2- 172.21.15.102:0/816543078 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x562b8e979f10 0x562b8e97a360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:22.633 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.640 7f4531363700 1 -- 172.21.15.102:0/816543078 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562b8e986220 msgr2=0x562b8e986670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:29:22.633 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.640 7f4531363700 1 --2- 172.21.15.102:0/816543078 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562b8e986220 0x562b8e986670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:22.633 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.640 7f4531363700 1 -- 172.21.15.102:0/816543078 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562b8e94f3a0 con 0x562b8e986ca0 2024-04-17T22:29:22.633 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.640 7f4531b64700 1 --2- 172.21.15.102:0/816543078 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562b8e986220 0x562b8e986670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:29:22.633 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.640 7f4531363700 1 --2- 172.21.15.102:0/816543078 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x562b8e986ca0 0x562b8e9798e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x562b8e8bd610 tx=0x7f4528006830).ready entity=mon.2 client_cookie=8ea89f490642c3fa server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:29:22.633 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.640 7f4538efd700 1 -- 172.21.15.102:0/816543078 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x562b8e94f3a0 con 0x562b8e986ca0 2024-04-17T22:29:22.633 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.640 7f4538efd700 1 -- 172.21.15.102:0/816543078 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4528006ab0 con 0x562b8e986ca0 2024-04-17T22:29:22.633 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.640 7f4538efd700 1 -- 172.21.15.102:0/816543078 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f4528017040 con 0x562b8e986ca0 2024-04-17T22:29:22.633 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.640 7f454b7b6b80 1 -- 172.21.15.102:0/816543078 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x562b8e986ca0 msgr2=0x562b8e9798e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:22.633 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.640 7f454b7b6b80 1 --2- 172.21.15.102:0/816543078 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x562b8e986ca0 0x562b8e9798e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x562b8e8bd610 tx=0x7f4528006830).stop 2024-04-17T22:29:22.633 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.640 7f454b7b6b80 1 -- 172.21.15.102:0/816543078 shutdown_connections 2024-04-17T22:29:22.634 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.640 7f454b7b6b80 1 --2- 172.21.15.102:0/816543078 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x562b8e979f10 0x562b8e97a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:22.634 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.640 7f454b7b6b80 1 --2- 172.21.15.102:0/816543078 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562b8e986220 0x562b8e986670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:22.634 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.640 7f454b7b6b80 1 --2- 172.21.15.102:0/816543078 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x562b8e986ca0 0x562b8e9798e0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:22.634 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.640 7f454b7b6b80 1 -- 172.21.15.102:0/816543078 >> 172.21.15.102:0/816543078 conn(0x562b8e972cc0 msgr2=0x562b8e994200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:29:22.634 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.641 7f454b7b6b80 1 -- 172.21.15.102:0/816543078 shutdown_connections 2024-04-17T22:29:22.634 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.641 7f454b7b6b80 1 -- 172.21.15.102:0/816543078 wait complete. 2024-04-17T22:29:22.634 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.641 7f454b7b6b80 1 -- start start 2024-04-17T22:29:22.634 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.641 7f454b7b6b80 1 Processor -- start 2024-04-17T22:29:22.634 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.641 7f454b7b6b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562b8e9a18a0 0x562b8e9a1cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:22.634 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.641 7f454b7b6b80 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x562b8e9a2320 0x562b8e988040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:22.634 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.641 7f454b7b6b80 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x562b8e988670 0x562b8e988ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:22.634 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.641 7f454b7b6b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b8e987200 con 0x562b8e9a18a0 2024-04-17T22:29:22.634 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.641 7f454b7b6b80 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x562b8e97a330 con 0x562b8e988670 2024-04-17T22:29:22.634 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.641 7f454b7b6b80 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x562b8e987ba0 con 0x562b8e9a2320 2024-04-17T22:29:22.635 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.641 7f454b7b6b80 10 client.0 ll_register_callbacks cb 0x562b8e8a7d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T22:29:22.635 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.642 7f4531b64700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562b8e9a18a0 0x562b8e9a1cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:22.635 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.642 7f4532365700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x562b8e988670 0x562b8e988ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:22.635 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.642 7f4531b64700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562b8e9a18a0 0x562b8e9a1cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:41208/0 (socket says 172.21.15.102:41208) 2024-04-17T22:29:22.635 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.642 7f4531b64700 1 -- 172.21.15.102:0/934902905 learned_addr learned my addr 172.21.15.102:0/934902905 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:29:22.635 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.642 7f4531363700 1 --2- 172.21.15.102:0/934902905 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x562b8e9a2320 0x562b8e988040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:22.635 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.642 7f4531b64700 1 -- 172.21.15.102:0/934902905 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x562b8e9a2320 msgr2=0x562b8e988040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:29:22.635 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.642 7f4531b64700 1 --2- 172.21.15.102:0/934902905 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x562b8e9a2320 0x562b8e988040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:22.635 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.642 7f4531b64700 1 -- 172.21.15.102:0/934902905 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x562b8e988670 msgr2=0x562b8e988ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:29:22.635 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.642 7f4531b64700 1 --2- 172.21.15.102:0/934902905 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x562b8e988670 0x562b8e988ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:22.635 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.642 7f4531b64700 1 -- 172.21.15.102:0/934902905 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562b8e7bfb60 con 0x562b8e9a18a0 2024-04-17T22:29:22.635 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.642 7f4532365700 1 --2- 172.21.15.102:0/934902905 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x562b8e988670 0x562b8e988ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:29:22.635 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.642 7f4531363700 1 --2- 172.21.15.102:0/934902905 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x562b8e9a2320 0x562b8e988040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:29:22.636 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.642 7f4531b64700 1 --2- 172.21.15.102:0/934902905 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562b8e9a18a0 0x562b8e9a1cf0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 rx=0x562b8e977cb0 tx=0x7f4524016910).ready entity=mon.0 client_cookie=4cb6e1264214dc45 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:29:22.636 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.642 7f450cff9700 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-04-17T22:29:22.636 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.642 7f450cff9700 1 -- 172.21.15.102:0/934902905 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x562b8e7bfb60 con 0x562b8e9a18a0 2024-04-17T22:29:22.636 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.642 7f450cff9700 1 -- 172.21.15.102:0/934902905 <== mon.0 v2:172.21.15.102:3300/0 2 ==== osd_map(16..16 src has 1..16) v4 ==== 3194+0+0 (secure 0 0 0) 0x7f4524010070 con 0x562b8e9a18a0 2024-04-17T22:29:22.636 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.642 7f454b7b6b80 1 -- 172.21.15.102:0/934902905 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562b8e8bacd0 con 0x562b8e9a18a0 2024-04-17T22:29:22.636 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.642 7f450cff9700 1 -- 172.21.15.102:0/934902905 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=5) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4524017650 con 0x562b8e9a18a0 2024-04-17T22:29:22.636 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.642 7f454b7b6b80 10 client.4617 fetch_fsmap learned FSMap version 5 2024-04-17T22:29:22.636 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.642 7f454b7b6b80 1 -- 172.21.15.102:0/934902905 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=5,monmap=2+}) v3 -- 0x562b8e979c50 con 0x562b8e9a18a0 2024-04-17T22:29:22.636 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.642 7f450cff9700 1 -- 172.21.15.102:0/934902905 <== mon.0 v2:172.21.15.102:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x562b8e8bacd0 con 0x562b8e9a18a0 2024-04-17T22:29:22.636 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.642 7f450cff9700 1 -- 172.21.15.102:0/934902905 <== mon.0 v2:172.21.15.102:3300/0 5 ==== fsmap(e 5) v1 ==== 1025+0+0 (secure 0 0 0) 0x7f452402a420 con 0x562b8e9a18a0 2024-04-17T22:29:22.636 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.642 7f454b7b6b80 10 client.4617 fetch_fsmap finished waiting for FSMap version 5 2024-04-17T22:29:22.636 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.642 7f454b7b6b80 10 client.4617 resolve_mds: resolved ID 'a' to GID 4177 2024-04-17T22:29:22.636 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.642 7f454b7b6b80 20 client.4617 populate_metadata read hostname 'smithi102' 2024-04-17T22:29:22.636 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.642 7f454b7b6b80 4 client.4617 mds_command: new command op to 4177 tid=0[{"prefix": "get_command_descriptions"}] 2024-04-17T22:29:22.637 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.643 7f454b7b6b80 1 -- 172.21.15.102:0/934902905 --> v1:172.21.15.102:6817/1944542598 -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562b8e7b3ba0 con 0x562b8e9a9780 2024-04-17T22:29:22.637 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.643 7f450cff9700 10 client.4617 ms_handle_connect on v1:172.21.15.102:6817/1944542598 2024-04-17T22:29:22.637 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.643 7f450cff9700 1 -- 172.21.15.102:0/934902905 <== mds.0 v1:172.21.15.102:6817/1944542598 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3729 (unknown 0 0 338808096) 0x562b8e7b3ba0 con 0x562b8e9a9780 2024-04-17T22:29:22.637 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.643 7f450cff9700 10 client.4617 handle_command_reply: tid=0 2024-04-17T22:29:22.637 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.643 7f454b7b6b80 1 client.4617 shutdown 2024-04-17T22:29:22.637 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.643 7f454b7b6b80 1 -- 172.21.15.102:0/934902905 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562b8e9a18a0 msgr2=0x562b8e9a1cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:22.637 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.643 7f454b7b6b80 1 --2- 172.21.15.102:0/934902905 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562b8e9a18a0 0x562b8e9a1cf0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 rx=0x562b8e977cb0 tx=0x7f4524016910).stop 2024-04-17T22:29:22.637 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.643 7f454b7b6b80 1 -- 172.21.15.102:0/934902905 shutdown_connections 2024-04-17T22:29:22.637 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.643 7f454b7b6b80 1 --2- 172.21.15.102:0/934902905 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x562b8e988670 0x562b8e988ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:22.637 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.643 7f450cff9700 0 client.4617 ms_handle_reset on v1:172.21.15.102:6817/1944542598 2024-04-17T22:29:22.637 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.643 7f454b7b6b80 1 --2- 172.21.15.102:0/934902905 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562b8e9a18a0 0x562b8e9a1cf0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:22.637 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.643 7f454b7b6b80 1 --2- 172.21.15.102:0/934902905 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x562b8e9a2320 0x562b8e988040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:22.637 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.643 7f454b7b6b80 1 -- 172.21.15.102:0/934902905 >> 172.21.15.102:0/934902905 conn(0x562b8e977f50 msgr2=0x562b8e994480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:29:22.637 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.643 7f454b7b6b80 1 -- 172.21.15.102:0/934902905 shutdown_connections 2024-04-17T22:29:22.638 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.643 7f454b7b6b80 1 -- 172.21.15.102:0/934902905 wait complete. 2024-04-17T22:29:22.638 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.644 7f454b7b6b80 20 client.4617 trim_cache size 0 max 16384 2024-04-17T22:29:22.640 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.653 7f450effd700 1 -- 172.21.15.102:0/3870686672 >> v1:172.21.15.102:6816/42360 conn(0x7f45100145a0 legacy=0x7f4510016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:22.640 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.653 7f450effd700 1 -- 172.21.15.102:0/3870686672 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f45341adcb0 msgr2=0x7f45341b4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:22.640 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.653 7f450effd700 1 --2- 172.21.15.102:0/3870686672 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f45341adcb0 0x7f45341b4240 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0x7f45340c5960 tx=0x7f452c00ea50).stop 2024-04-17T22:29:22.640 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.653 7f450effd700 1 -- 172.21.15.102:0/3870686672 shutdown_connections 2024-04-17T22:29:22.640 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.653 7f450effd700 1 --2- 172.21.15.102:0/3870686672 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f45341adcb0 0x7f45341b4240 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:22.640 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.653 7f450effd700 1 --2- 172.21.15.102:0/3870686672 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f45340cf290 0x7f45341ad230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:22.640 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.653 7f450effd700 1 --2- 172.21.15.102:0/3870686672 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f45341ad860 0x7f45341b3c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:22.640 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.653 7f450effd700 1 -- 172.21.15.102:0/3870686672 >> 172.21.15.102:0/3870686672 conn(0x7f45340e0dc0 msgr2=0x7f45340daae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:29:22.641 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.653 7f450effd700 1 -- 172.21.15.102:0/3870686672 shutdown_connections 2024-04-17T22:29:22.641 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.653 7f450effd700 1 -- 172.21.15.102:0/3870686672 wait complete. 2024-04-17T22:29:22.641 INFO:teuthology.orchestra.run.smithi102.stderr:no valid command found; 10 closest matches: 2024-04-17T22:29:22.641 INFO:teuthology.orchestra.run.smithi102.stderr:injectargs [...] 2024-04-17T22:29:22.641 INFO:teuthology.orchestra.run.smithi102.stderr:config set 2024-04-17T22:29:22.641 INFO:teuthology.orchestra.run.smithi102.stderr:config unset 2024-04-17T22:29:22.641 INFO:teuthology.orchestra.run.smithi102.stderr:exit 2024-04-17T22:29:22.641 INFO:teuthology.orchestra.run.smithi102.stderr:respawn 2024-04-17T22:29:22.641 INFO:teuthology.orchestra.run.smithi102.stderr:session kill 2024-04-17T22:29:22.642 INFO:teuthology.orchestra.run.smithi102.stderr:cpu_profiler status|flush 2024-04-17T22:29:22.642 INFO:teuthology.orchestra.run.smithi102.stderr:session ls { [...]} 2024-04-17T22:29:22.642 INFO:teuthology.orchestra.run.smithi102.stderr:client ls { [...]} 2024-04-17T22:29:22.642 INFO:teuthology.orchestra.run.smithi102.stderr:session evict { [...]} 2024-04-17T22:29:22.642 INFO:teuthology.orchestra.run.smithi102.stderr:Error EINVAL: invalid command 2024-04-17T22:29:22.650 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-17T22:29:22.651 INFO:teuthology.run_tasks:Running task exec... 2024-04-17T22:29:22.662 INFO:teuthology.task.exec:Executing custom commands... 2024-04-17T22:29:22.662 INFO:teuthology.task.exec:Running commands on role osd.0 host ubuntu@smithi102.front.sepia.ceph.com 2024-04-17T22:29:22.663 DEBUG:teuthology.orchestra.run.smithi102:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph osd set-require-min-compat-client nautilus' 2024-04-17T22:29:22.788 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.799 7f8486c1e700 1 Processor -- start 2024-04-17T22:29:22.788 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.799 7f8486c1e700 1 -- start start 2024-04-17T22:29:22.788 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.799 7f8486c1e700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f84800f5850 0x7f848010b020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:22.788 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.799 7f8486c1e700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f848010b650 0x7f848019fb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:22.788 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.800 7f8486c1e700 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8480103b50 con 0x7f848010baa0 2024-04-17T22:29:22.788 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.800 7f8486c1e700 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f848010bef0 con 0x7f84800f5850 2024-04-17T22:29:22.788 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.800 7f8486c1e700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84800fa380 con 0x7f848010b650 2024-04-17T22:29:22.789 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.800 7f847f7fe700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f84800f5850 0x7f848010b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:22.789 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.800 7f847effd700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f848010b650 0x7f848019fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:22.789 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.800 7f847ffff700 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f848010baa0 0x7f84801a0130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:34862/0 (socket says 172.21.15.102:34862) 2024-04-17T22:29:22.789 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.800 7f847ffff700 1 -- 172.21.15.102:0/3281479274 learned_addr learned my addr 172.21.15.102:0/3281479274 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-17T22:29:22.789 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.800 7f847effd700 1 -- 172.21.15.102:0/3281479274 >> v2:172.21.15.102:3300/0 conn(0x7f84800f5850 msgr2=0x7f848010b020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:29:22.789 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.800 7f847effd700 1 --2- 172.21.15.102:0/3281479274 >> v2:172.21.15.102:3300/0 conn(0x7f84800f5850 0x7f848010b020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:22.789 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.800 7f847effd700 1 -- 172.21.15.102:0/3281479274 >> v1:172.21.15.102:6789/0 conn(0x7f848010baa0 legacy=0x7f84801a0130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:22.789 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.800 7f847effd700 1 -- 172.21.15.102:0/3281479274 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84801a0930 con 0x7f848010b650 2024-04-17T22:29:22.789 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.800 7f847e7fc700 1 -- 172.21.15.102:0/3281479274 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2596310825 0 0) 0x7f8478001490 con 0x7f848010baa0 2024-04-17T22:29:22.789 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.800 7f847f7fe700 1 --2- 172.21.15.102:0/3281479274 >> v2:172.21.15.102:3300/0 conn(0x7f84800f5850 0x7f848010b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:29:22.789 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.800 7f847effd700 1 --2- 172.21.15.102:0/3281479274 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f848010b650 0x7f848019fb00 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 rx=0x7f8470002f70 tx=0x7f847000b970).ready entity=mon.1 client_cookie=b2c4609faa38512a server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:29:22.789 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.800 7f847e7fc700 1 -- 172.21.15.102:0/3281479274 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f84801a0930 con 0x7f848010b650 2024-04-17T22:29:22.789 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.800 7f847e7fc700 1 -- 172.21.15.102:0/3281479274 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f847000d5c0 con 0x7f848010b650 2024-04-17T22:29:22.789 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.800 7f847e7fc700 1 -- 172.21.15.102:0/3281479274 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f847000d990 con 0x7f848010b650 2024-04-17T22:29:22.789 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.800 7f8486c1e700 1 -- 172.21.15.102:0/3281479274 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f848010b650 msgr2=0x7f848019fb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:22.790 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.800 7f8486c1e700 1 --2- 172.21.15.102:0/3281479274 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f848010b650 0x7f848019fb00 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 rx=0x7f8470002f70 tx=0x7f847000b970).stop 2024-04-17T22:29:22.790 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.801 7f8486c1e700 1 -- 172.21.15.102:0/3281479274 shutdown_connections 2024-04-17T22:29:22.790 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.801 7f8486c1e700 1 --2- 172.21.15.102:0/3281479274 >> v2:172.21.15.102:3300/0 conn(0x7f84800f5850 0x7f848010b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:22.790 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.801 7f8486c1e700 1 --2- 172.21.15.102:0/3281479274 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f848010b650 0x7f848019fb00 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:22.790 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.801 7f8486c1e700 1 -- 172.21.15.102:0/3281479274 >> 172.21.15.102:0/3281479274 conn(0x7f84801138b0 msgr2=0x7f84800f4f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:29:22.790 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.801 7f8486c1e700 1 -- 172.21.15.102:0/3281479274 shutdown_connections 2024-04-17T22:29:22.790 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.801 7f8486c1e700 1 -- 172.21.15.102:0/3281479274 wait complete. 2024-04-17T22:29:22.812 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.823 7f8486c1e700 1 Processor -- start 2024-04-17T22:29:22.812 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.824 7f8486c1e700 1 -- start start 2024-04-17T22:29:22.812 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.824 7f8486c1e700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f84800dc4c0 0x7f84801a90c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:22.812 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.824 7f8486c1e700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f84801a96f0 0x7f84801afa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:22.812 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.824 7f8486c1e700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f84801a9b40 0x7f84801b0040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:22.812 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.824 7f8486c1e700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84800a8470 con 0x7f84800dc4c0 2024-04-17T22:29:22.812 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.824 7f8486c1e700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84800dbc00 con 0x7f84801a9b40 2024-04-17T22:29:22.812 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.824 7f8486c1e700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f84800db7e0 con 0x7f84801a96f0 2024-04-17T22:29:22.813 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.824 7f847f7fe700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f84800dc4c0 0x7f84801a90c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:22.813 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.824 7f847effd700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f84801a96f0 0x7f84801afa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:22.813 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.824 7f847ffff700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f84801a9b40 0x7f84801b0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:22.813 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.824 7f847effd700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f84801a96f0 0x7f84801afa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:44630/0 (socket says 172.21.15.102:44630) 2024-04-17T22:29:22.813 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.824 7f847f7fe700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f84800dc4c0 0x7f84801a90c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:41230/0 (socket says 172.21.15.102:41230) 2024-04-17T22:29:22.813 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.824 7f847effd700 1 -- 172.21.15.102:0/3903817048 learned_addr learned my addr 172.21.15.102:0/3903817048 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:29:22.813 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.824 7f847ffff700 1 -- 172.21.15.102:0/3903817048 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f84801a96f0 msgr2=0x7f84801afa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:22.813 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.824 7f847ffff700 1 --2- 172.21.15.102:0/3903817048 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f84801a96f0 0x7f84801afa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:22.813 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.824 7f847ffff700 1 -- 172.21.15.102:0/3903817048 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f84800dc4c0 msgr2=0x7f84801a90c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:22.813 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.824 7f847ffff700 1 --2- 172.21.15.102:0/3903817048 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f84800dc4c0 0x7f84801a90c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:22.813 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.824 7f847ffff700 1 -- 172.21.15.102:0/3903817048 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84800dbec0 con 0x7f84801a9b40 2024-04-17T22:29:22.813 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.824 7f847f7fe700 1 --2- 172.21.15.102:0/3903817048 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f84800dc4c0 0x7f84801a90c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:29:22.813 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.824 7f847effd700 1 --2- 172.21.15.102:0/3903817048 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f84801a96f0 0x7f84801afa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:29:22.813 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.825 7f847ffff700 1 --2- 172.21.15.102:0/3903817048 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f84801a9b40 0x7f84801b0040 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7f84800dbb30 tx=0x7f847800c9b0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:29:22.813 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.825 7f847cff9700 1 -- 172.21.15.102:0/3903817048 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f84800dbec0 con 0x7f84801a9b40 2024-04-17T22:29:22.814 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.825 7f847cff9700 1 -- 172.21.15.102:0/3903817048 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8478005bb0 con 0x7f84801a9b40 2024-04-17T22:29:22.814 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.825 7f8486c1e700 1 -- 172.21.15.102:0/3903817048 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84801b0670 con 0x7f84801a9b40 2024-04-17T22:29:22.814 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.825 7f847cff9700 1 -- 172.21.15.102:0/3903817048 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f84780070d0 con 0x7f84801a9b40 2024-04-17T22:29:22.814 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.825 7f8486c1e700 1 -- 172.21.15.102:0/3903817048 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84801b7210 con 0x7f84801a9b40 2024-04-17T22:29:22.814 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.826 7f847cff9700 1 -- 172.21.15.102:0/3903817048 <== mon.1 v2:172.21.15.102:3301/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7f84780072d0 con 0x7f84801a9b40 2024-04-17T22:29:22.815 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.826 7f847cff9700 1 -- 172.21.15.102:0/3903817048 <== mon.1 v2:172.21.15.102:3301/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 3194+0+0 (secure 0 0 0) 0x7f84780055a0 con 0x7f84801a9b40 2024-04-17T22:29:22.815 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.826 7f8486c1e700 1 -- 172.21.15.102:0/3903817048 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84800f0810 con 0x7f84801a9b40 2024-04-17T22:29:22.815 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.829 7f847cff9700 1 -- 172.21.15.102:0/3903817048 <== mon.1 v2:172.21.15.102:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+110455 (secure 0 0 0) 0x7f84780059a0 con 0x7f84801a9b40 2024-04-17T22:29:22.930 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:22.943 7f8486c1e700 1 -- 172.21.15.102:0/3903817048 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "osd set-require-min-compat-client", "version": "nautilus"} v 0) v1 -- 0x7f84800e22b0 con 0x7f84801a9b40 2024-04-17T22:29:23.247 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:23.261 7f847cff9700 1 -- 172.21.15.102:0/3903817048 <== mon.1 v2:172.21.15.102:3301/0 7 ==== mon_command_ack([{"prefix": "osd set-require-min-compat-client", "version": "nautilus"}]=0 set require_min_compat_client to nautilus v17) v1 ==== 145+0+0 (secure 0 0 0) 0x7f8480114430 con 0x7f84801a9b40 2024-04-17T22:29:23.250 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:23.264 7f8486c1e700 1 -- 172.21.15.102:0/3903817048 >> v1:172.21.15.102:6816/42360 conn(0x7f84640145a0 legacy=0x7f8464016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:23.250 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:23.264 7f8486c1e700 1 -- 172.21.15.102:0/3903817048 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f84801a9b40 msgr2=0x7f84801b0040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:23.250 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:23.264 7f8486c1e700 1 --2- 172.21.15.102:0/3903817048 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f84801a9b40 0x7f84801b0040 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7f84800dbb30 tx=0x7f847800c9b0).stop 2024-04-17T22:29:23.250 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:23.264 7f8486c1e700 1 -- 172.21.15.102:0/3903817048 shutdown_connections 2024-04-17T22:29:23.251 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:23.264 7f8486c1e700 1 --2- 172.21.15.102:0/3903817048 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f84801a9b40 0x7f84801b0040 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:23.251 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:23.264 7f8486c1e700 1 --2- 172.21.15.102:0/3903817048 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f84800dc4c0 0x7f84801a90c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:23.251 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:23.264 7f8486c1e700 1 --2- 172.21.15.102:0/3903817048 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f84801a96f0 0x7f84801afa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:23.251 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:23.264 7f8486c1e700 1 -- 172.21.15.102:0/3903817048 >> 172.21.15.102:0/3903817048 conn(0x7f84800e0de0 msgr2=0x7f84800cd860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:29:23.251 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:23.264 7f8486c1e700 1 -- 172.21.15.102:0/3903817048 shutdown_connections 2024-04-17T22:29:23.251 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:23.264 7f8486c1e700 1 -- 172.21.15.102:0/3903817048 wait complete. 2024-04-17T22:29:23.251 INFO:teuthology.orchestra.run.smithi102.stderr:set require_min_compat_client to nautilus 2024-04-17T22:29:23.261 INFO:teuthology.run_tasks:Running task print... 2024-04-17T22:29:23.271 INFO:teuthology.task.print:**** done ceph 2024-04-17T22:29:23.271 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-04-17T22:29:23.317 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-04-17T22:29:23.317 INFO:tasks.ceph_fuse:config is {'client.0': {}} 2024-04-17T22:29:23.317 INFO:tasks.ceph_fuse:client.0 config is {} 2024-04-17T22:29:23.317 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-17T22:29:23.317 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-17T22:29:23.317 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T22:29:23.317 DEBUG:teuthology.orchestra.run.smithi190:> ip netns list 2024-04-17T22:29:23.340 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T22:29:23.340 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip link delete ceph-brx 2024-04-17T22:29:23.415 INFO:teuthology.orchestra.run.smithi190.stderr:Cannot find device "ceph-brx" 2024-04-17T22:29:23.416 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-17T22:29:23.417 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-04-17T22:29:23.417 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-17T22:29:23.544 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:23.555 7f69ab5f8700 1 Processor -- start 2024-04-17T22:29:23.544 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:23.555 7f69ab5f8700 1 -- start start 2024-04-17T22:29:23.544 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:23.555 7f69ab5f8700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f69a40f5860 0x7f69a410b030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:23.544 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:23.555 7f69ab5f8700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f69a410b660 0x7f69a419fb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:23.544 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:23.555 7f69ab5f8700 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69a4103b60 con 0x7f69a410bab0 2024-04-17T22:29:23.544 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:23.555 7f69ab5f8700 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69a410bf00 con 0x7f69a40f5860 2024-04-17T22:29:23.544 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:23.555 7f69ab5f8700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f69a40fa390 con 0x7f69a410b660 2024-04-17T22:29:23.544 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:23.555 7f69a37fe700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f69a410b660 0x7f69a419fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:23.544 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:23.555 7f69a3fff700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f69a40f5860 0x7f69a410b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:23.544 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:23.555 7f69a37fe700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f69a410b660 0x7f69a419fb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:41550/0 (socket says 172.21.15.102:41550) 2024-04-17T22:29:23.544 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:23.555 7f69a3fff700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f69a40f5860 0x7f69a410b030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:37920/0 (socket says 172.21.15.102:37920) 2024-04-17T22:29:23.545 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:23.556 7f69a37fe700 1 -- 172.21.15.102:0/556768770 learned_addr learned my addr 172.21.15.102:0/556768770 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:29:23.545 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:23.556 7f69a2ffd700 1 -- 172.21.15.102:0/556768770 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 325184542 0 0) 0x7f699c001490 con 0x7f69a410bab0 2024-04-17T22:29:23.545 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:23.556 7f69a2ffd700 1 -- 172.21.15.102:0/556768770 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6984003270 con 0x7f69a410bab0 2024-04-17T22:29:23.545 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:23.556 7f69a37fe700 1 -- 172.21.15.102:0/556768770 >> v2:172.21.15.102:3300/0 conn(0x7f69a40f5860 msgr2=0x7f69a410b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:29:23.545 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:23.556 7f69a37fe700 1 --2- 172.21.15.102:0/556768770 >> v2:172.21.15.102:3300/0 conn(0x7f69a40f5860 0x7f69a410b030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:23.545 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:23.556 7f69a37fe700 1 -- 172.21.15.102:0/556768770 >> v1:172.21.15.102:6789/0 conn(0x7f69a410bab0 legacy=0x7f69a41a0140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:23.545 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:23.556 7f69a37fe700 1 -- 172.21.15.102:0/556768770 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69a41a0940 con 0x7f69a410b660 2024-04-17T22:29:23.545 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:23.556 7f69a3fff700 1 --2- 172.21.15.102:0/556768770 >> v2:172.21.15.102:3300/0 conn(0x7f69a40f5860 0x7f69a410b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:29:23.545 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:23.556 7f69a37fe700 1 --2- 172.21.15.102:0/556768770 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f69a410b660 0x7f69a419fb10 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7f6994002f70 tx=0x7f699400bc00).ready entity=mon.2 client_cookie=f6eedc6d9bdb8f94 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:29:23.545 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:23.556 7f69a2ffd700 1 -- 172.21.15.102:0/556768770 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f69a41a0940 con 0x7f69a410b660 2024-04-17T22:29:23.545 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:23.556 7f69a2ffd700 1 -- 172.21.15.102:0/556768770 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f699400dc80 con 0x7f69a410b660 2024-04-17T22:29:23.545 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:23.556 7f69a2ffd700 1 -- 172.21.15.102:0/556768770 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f6994010660 con 0x7f69a410b660 2024-04-17T22:29:23.545 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:23.556 7f69ab5f8700 1 -- 172.21.15.102:0/556768770 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f69a410b660 msgr2=0x7f69a419fb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:23.546 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:23.556 7f69ab5f8700 1 --2- 172.21.15.102:0/556768770 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f69a410b660 0x7f69a419fb10 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7f6994002f70 tx=0x7f699400bc00).stop 2024-04-17T22:29:23.546 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:23.557 7f69ab5f8700 1 -- 172.21.15.102:0/556768770 shutdown_connections 2024-04-17T22:29:23.546 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:23.557 7f69ab5f8700 1 --2- 172.21.15.102:0/556768770 >> v2:172.21.15.102:3300/0 conn(0x7f69a40f5860 0x7f69a410b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:23.546 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:23.557 7f69ab5f8700 1 --2- 172.21.15.102:0/556768770 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f69a410b660 0x7f69a419fb10 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:23.546 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:23.557 7f69ab5f8700 1 -- 172.21.15.102:0/556768770 >> 172.21.15.102:0/556768770 conn(0x7f69a41138c0 msgr2=0x7f69a40f4f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:29:23.546 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:23.557 7f69ab5f8700 1 -- 172.21.15.102:0/556768770 shutdown_connections 2024-04-17T22:29:23.546 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:23.557 7f69ab5f8700 1 -- 172.21.15.102:0/556768770 wait complete. 2024-04-17T22:29:23.566 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:23.579 7f69ab5f8700 1 Processor -- start 2024-04-17T22:29:23.566 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:23.579 7f69ab5f8700 1 -- start start 2024-04-17T22:29:23.568 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:23.580 7f69ab5f8700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f69a40d7080 0x7f69a40ce1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:23.568 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:23.580 7f69ab5f8700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f69a40ce810 0x7f69a41b0090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:23.568 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:23.580 7f69ab5f8700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f69a40cec60 0x7f69a41b06c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:23.569 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:23.580 7f69ab5f8700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69a40c3910 con 0x7f69a40ce810 2024-04-17T22:29:23.569 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:23.580 7f69ab5f8700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69a41b0cf0 con 0x7f69a40cec60 2024-04-17T22:29:23.569 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:23.580 7f69ab5f8700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f69a41b0ef0 con 0x7f69a40d7080 2024-04-17T22:29:23.569 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:23.580 7f69a37fe700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f69a40ce810 0x7f69a41b0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:23.569 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:23.580 7f69a8bf4700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f69a40cec60 0x7f69a41b06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:23.569 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:23.580 7f69a37fe700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f69a40ce810 0x7f69a41b0090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:37934/0 (socket says 172.21.15.102:37934) 2024-04-17T22:29:23.569 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:23.580 7f69a3fff700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f69a40d7080 0x7f69a40ce1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:23.569 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:23.580 7f69a37fe700 1 -- 172.21.15.102:0/2862105445 learned_addr learned my addr 172.21.15.102:0/2862105445 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:29:23.569 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:23.580 7f69a37fe700 1 -- 172.21.15.102:0/2862105445 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f69a40d7080 msgr2=0x7f69a40ce1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:23.569 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:23.580 7f69a37fe700 1 --2- 172.21.15.102:0/2862105445 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f69a40d7080 0x7f69a40ce1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:23.569 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:23.580 7f69a37fe700 1 -- 172.21.15.102:0/2862105445 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f69a40cec60 msgr2=0x7f69a41b06c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:23.569 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:23.580 7f69a37fe700 1 --2- 172.21.15.102:0/2862105445 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f69a40cec60 0x7f69a41b06c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:23.569 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:23.580 7f69a37fe700 1 -- 172.21.15.102:0/2862105445 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69a41b7800 con 0x7f69a40ce810 2024-04-17T22:29:23.569 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:23.580 7f69a8bf4700 1 --2- 172.21.15.102:0/2862105445 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f69a40cec60 0x7f69a41b06c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:29:23.570 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:23.580 7f69a37fe700 1 --2- 172.21.15.102:0/2862105445 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f69a40ce810 0x7f69a41b0090 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 rx=0x7f6994002f70 tx=0x7f6994006cd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:29:23.570 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:23.580 7f69a17fa700 1 -- 172.21.15.102:0/2862105445 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f69a41b7800 con 0x7f69a40ce810 2024-04-17T22:29:23.570 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:23.580 7f69a17fa700 1 -- 172.21.15.102:0/2862105445 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6994010cf0 con 0x7f69a40ce810 2024-04-17T22:29:23.570 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:23.581 7f69ab5f8700 1 -- 172.21.15.102:0/2862105445 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69a41b7a00 con 0x7f69a40ce810 2024-04-17T22:29:23.570 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:23.581 7f69a17fa700 1 -- 172.21.15.102:0/2862105445 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f69940185f0 con 0x7f69a40ce810 2024-04-17T22:29:23.570 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:23.581 7f69ab5f8700 1 -- 172.21.15.102:0/2862105445 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69a41b7de0 con 0x7f69a40ce810 2024-04-17T22:29:23.570 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:23.581 7f69a17fa700 1 -- 172.21.15.102:0/2862105445 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7f69940187f0 con 0x7f69a40ce810 2024-04-17T22:29:23.570 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:23.581 7f69a17fa700 1 -- 172.21.15.102:0/2862105445 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3194+0+0 (secure 0 0 0) 0x7f6994009db0 con 0x7f69a40ce810 2024-04-17T22:29:23.570 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:23.581 7f69ab5f8700 1 -- 172.21.15.102:0/2862105445 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69a40f0820 con 0x7f69a40ce810 2024-04-17T22:29:23.571 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:23.585 7f69a17fa700 1 -- 172.21.15.102:0/2862105445 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+110455 (secure 0 0 0) 0x7f699400d880 con 0x7f69a40ce810 2024-04-17T22:29:23.707 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:23.720 7f69ab5f8700 1 -- 172.21.15.102:0/2862105445 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f69a40f5550 con 0x7f69a40ce810 2024-04-17T22:29:23.707 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:23.721 7f69a17fa700 1 -- 172.21.15.102:0/2862105445 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v5) v1 ==== 53+0+73 (secure 0 0 0) 0x7f69940108f0 con 0x7f69a40ce810 2024-04-17T22:29:23.708 INFO:teuthology.orchestra.run.smithi102.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-17T22:29:23.709 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:23.722 7f69ab5f8700 1 -- 172.21.15.102:0/2862105445 >> v1:172.21.15.102:6816/42360 conn(0x7f69880145a0 legacy=0x7f6988016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:23.709 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:23.722 7f69ab5f8700 1 -- 172.21.15.102:0/2862105445 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f69a40ce810 msgr2=0x7f69a41b0090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:23.709 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:23.722 7f69ab5f8700 1 --2- 172.21.15.102:0/2862105445 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f69a40ce810 0x7f69a41b0090 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 rx=0x7f6994002f70 tx=0x7f6994006cd0).stop 2024-04-17T22:29:23.709 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:23.722 7f69ab5f8700 1 -- 172.21.15.102:0/2862105445 shutdown_connections 2024-04-17T22:29:23.709 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:23.722 7f69ab5f8700 1 --2- 172.21.15.102:0/2862105445 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f69a40cec60 0x7f69a41b06c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:23.709 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:23.722 7f69ab5f8700 1 --2- 172.21.15.102:0/2862105445 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f69a40d7080 0x7f69a40ce1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:23.709 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:23.722 7f69ab5f8700 1 --2- 172.21.15.102:0/2862105445 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f69a40ce810 0x7f69a41b0090 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:23.709 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:23.722 7f69ab5f8700 1 -- 172.21.15.102:0/2862105445 >> 172.21.15.102:0/2862105445 conn(0x7f69a40d6520 msgr2=0x7f69a41a42f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:29:23.709 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:23.722 7f69ab5f8700 1 -- 172.21.15.102:0/2862105445 shutdown_connections 2024-04-17T22:29:23.709 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:23.722 7f69ab5f8700 1 -- 172.21.15.102:0/2862105445 wait complete. 2024-04-17T22:29:23.719 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-17T22:29:23.719 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-17T22:29:23.719 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi190.front.sepia.ceph.com 2024-04-17T22:29:23.720 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-17T22:29:23.720 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-17T22:29:23.720 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-17T22:29:23.720 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-17T22:29:23.720 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-17T22:29:23.720 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-17T22:29:23.720 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T22:29:23.720 DEBUG:teuthology.orchestra.run.smithi190:> ip addr 2024-04-17T22:29:23.742 INFO:teuthology.orchestra.run.smithi190.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-17T22:29:23.742 INFO:teuthology.orchestra.run.smithi190.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-17T22:29:23.743 INFO:teuthology.orchestra.run.smithi190.stdout: inet 127.0.0.1/8 scope host lo 2024-04-17T22:29:23.743 INFO:teuthology.orchestra.run.smithi190.stdout: valid_lft forever preferred_lft forever 2024-04-17T22:29:23.743 INFO:teuthology.orchestra.run.smithi190.stdout: inet6 ::1/128 scope host 2024-04-17T22:29:23.743 INFO:teuthology.orchestra.run.smithi190.stdout: valid_lft forever preferred_lft forever 2024-04-17T22:29:23.743 INFO:teuthology.orchestra.run.smithi190.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-17T22:29:23.743 INFO:teuthology.orchestra.run.smithi190.stdout: link/ether 0c:c4:7a:d9:36:d0 brd ff:ff:ff:ff:ff:ff 2024-04-17T22:29:23.743 INFO:teuthology.orchestra.run.smithi190.stdout: altname enp7s0f0 2024-04-17T22:29:23.743 INFO:teuthology.orchestra.run.smithi190.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-17T22:29:23.743 INFO:teuthology.orchestra.run.smithi190.stdout: link/ether 0c:c4:7a:d9:36:d1 brd ff:ff:ff:ff:ff:ff 2024-04-17T22:29:23.743 INFO:teuthology.orchestra.run.smithi190.stdout: altname enp7s0f1 2024-04-17T22:29:23.743 INFO:teuthology.orchestra.run.smithi190.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-17T22:29:23.743 INFO:teuthology.orchestra.run.smithi190.stdout: link/ether 0c:c4:7a:88:77:62 brd ff:ff:ff:ff:ff:ff 2024-04-17T22:29:23.743 INFO:teuthology.orchestra.run.smithi190.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-17T22:29:23.743 INFO:teuthology.orchestra.run.smithi190.stdout: link/ether 0c:c4:7a:88:77:63 brd ff:ff:ff:ff:ff:ff 2024-04-17T22:29:23.743 INFO:teuthology.orchestra.run.smithi190.stdout: inet 172.21.15.190/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-17T22:29:23.744 INFO:teuthology.orchestra.run.smithi190.stdout: valid_lft 42661sec preferred_lft 42661sec 2024-04-17T22:29:23.744 INFO:teuthology.orchestra.run.smithi190.stdout: inet6 fe80::ec4:7aff:fe88:7763/64 scope link 2024-04-17T22:29:23.744 INFO:teuthology.orchestra.run.smithi190.stdout: valid_lft forever preferred_lft forever 2024-04-17T22:29:23.744 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-17T22:29:23.744 DEBUG:teuthology.orchestra.run.smithi190:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-17T22:29:23.744 DEBUG:teuthology.orchestra.run.smithi190:> set -e 2024-04-17T22:29:23.744 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip link add name ceph-brx type bridge 2024-04-17T22:29:23.744 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip addr flush dev ceph-brx 2024-04-17T22:29:23.745 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip link set ceph-brx up 2024-04-17T22:29:23.745 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-17T22:29:23.745 DEBUG:teuthology.orchestra.run.smithi190:> ') 2024-04-17T22:29:23.840 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T22:29:23 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-17T22:29:23.938 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T22:29:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T22:29:23.945 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T22:29:23.945 DEBUG:teuthology.orchestra.run.smithi190:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-17T22:29:23.973 INFO:teuthology.orchestra.run.smithi190.stdout:1 2024-04-17T22:29:23.974 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T22:29:23.974 DEBUG:teuthology.orchestra.run.smithi190:> route 2024-04-17T22:29:24.051 INFO:teuthology.orchestra.run.smithi190.stdout:Kernel IP routing table 2024-04-17T22:29:24.051 INFO:teuthology.orchestra.run.smithi190.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-17T22:29:24.051 INFO:teuthology.orchestra.run.smithi190.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-17T22:29:24.051 INFO:teuthology.orchestra.run.smithi190.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-17T22:29:24.051 INFO:teuthology.orchestra.run.smithi190.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-17T22:29:24.052 DEBUG:teuthology.orchestra.run.smithi190:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-17T22:29:24.052 DEBUG:teuthology.orchestra.run.smithi190:> set -e 2024-04-17T22:29:24.052 DEBUG:teuthology.orchestra.run.smithi190:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-17T22:29:24.052 DEBUG:teuthology.orchestra.run.smithi190:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-17T22:29:24.052 DEBUG:teuthology.orchestra.run.smithi190:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-17T22:29:24.052 DEBUG:teuthology.orchestra.run.smithi190:> ') 2024-04-17T22:29:24.142 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T22:29:24 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-17T22:29:24.347 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T22:29:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T22:29:24.352 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T22:29:24.353 DEBUG:teuthology.orchestra.run.smithi190:> ip netns list 2024-04-17T22:29:24.371 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T22:29:24.371 DEBUG:teuthology.orchestra.run.smithi190:> ip netns list-id 2024-04-17T22:29:24.428 DEBUG:teuthology.orchestra.run.smithi190:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-17T22:29:24.428 DEBUG:teuthology.orchestra.run.smithi190:> set -e 2024-04-17T22:29:24.428 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-17T22:29:24.428 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-04-17T22:29:24.428 DEBUG:teuthology.orchestra.run.smithi190:> ') 2024-04-17T22:29:24.518 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T22:29:24 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-17T22:29:24.556 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T22:29:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T22:29:24.561 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-04-17T22:29:24.561 DEBUG:teuthology.orchestra.run.smithi190:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-17T22:29:24.561 DEBUG:teuthology.orchestra.run.smithi190:> set -e 2024-04-17T22:29:24.562 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-04-17T22:29:24.562 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-17T22:29:24.562 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-04-17T22:29:24.562 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-04-17T22:29:24.562 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-04-17T22:29:24.562 DEBUG:teuthology.orchestra.run.smithi190:> ') 2024-04-17T22:29:24.612 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T22:29:24 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-17T22:29:24.780 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T22:29:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T22:29:24.786 DEBUG:teuthology.orchestra.run.smithi190:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-17T22:29:24.786 DEBUG:teuthology.orchestra.run.smithi190:> set -e 2024-04-17T22:29:24.786 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip link set brx.0 up 2024-04-17T22:29:24.786 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip link set dev brx.0 master ceph-brx 2024-04-17T22:29:24.786 DEBUG:teuthology.orchestra.run.smithi190:> ') 2024-04-17T22:29:24.837 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T22:29:24 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-17T22:29:24.916 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T22:29:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T22:29:24.922 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-17T22:29:24.922 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-17T22:29:24.922 DEBUG:teuthology.orchestra.run.smithi190:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-17T22:29:24.939 INFO:teuthology.orchestra.run.smithi190.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-17T22:29:24.939 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-17T22:29:24.939 DEBUG:teuthology.orchestra.run.smithi190:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-17T22:29:24.998 DEBUG:teuthology.orchestra.run.smithi190:> sudo modprobe fuse 2024-04-17T22:29:25.065 DEBUG:teuthology.orchestra.run.smithi190:> cat /proc/self/mounts | awk '{print $2}' 2024-04-17T22:29:25.124 INFO:teuthology.orchestra.run.smithi190.stdout:/sys 2024-04-17T22:29:25.124 INFO:teuthology.orchestra.run.smithi190.stdout:/proc 2024-04-17T22:29:25.125 INFO:teuthology.orchestra.run.smithi190.stdout:/dev 2024-04-17T22:29:25.125 INFO:teuthology.orchestra.run.smithi190.stdout:/sys/kernel/security 2024-04-17T22:29:25.125 INFO:teuthology.orchestra.run.smithi190.stdout:/dev/shm 2024-04-17T22:29:25.125 INFO:teuthology.orchestra.run.smithi190.stdout:/dev/pts 2024-04-17T22:29:25.125 INFO:teuthology.orchestra.run.smithi190.stdout:/run 2024-04-17T22:29:25.125 INFO:teuthology.orchestra.run.smithi190.stdout:/sys/fs/cgroup 2024-04-17T22:29:25.125 INFO:teuthology.orchestra.run.smithi190.stdout:/sys/fs/cgroup/systemd 2024-04-17T22:29:25.126 INFO:teuthology.orchestra.run.smithi190.stdout:/sys/fs/pstore 2024-04-17T22:29:25.126 INFO:teuthology.orchestra.run.smithi190.stdout:/sys/fs/bpf 2024-04-17T22:29:25.126 INFO:teuthology.orchestra.run.smithi190.stdout:/sys/fs/cgroup/pids 2024-04-17T22:29:25.126 INFO:teuthology.orchestra.run.smithi190.stdout:/sys/fs/cgroup/cpu,cpuacct 2024-04-17T22:29:25.126 INFO:teuthology.orchestra.run.smithi190.stdout:/sys/fs/cgroup/net_cls,net_prio 2024-04-17T22:29:25.126 INFO:teuthology.orchestra.run.smithi190.stdout:/sys/fs/cgroup/rdma 2024-04-17T22:29:25.127 INFO:teuthology.orchestra.run.smithi190.stdout:/sys/fs/cgroup/hugetlb 2024-04-17T22:29:25.127 INFO:teuthology.orchestra.run.smithi190.stdout:/sys/fs/cgroup/blkio 2024-04-17T22:29:25.127 INFO:teuthology.orchestra.run.smithi190.stdout:/sys/fs/cgroup/freezer 2024-04-17T22:29:25.127 INFO:teuthology.orchestra.run.smithi190.stdout:/sys/fs/cgroup/cpuset 2024-04-17T22:29:25.127 INFO:teuthology.orchestra.run.smithi190.stdout:/sys/fs/cgroup/devices 2024-04-17T22:29:25.127 INFO:teuthology.orchestra.run.smithi190.stdout:/sys/fs/cgroup/memory 2024-04-17T22:29:25.127 INFO:teuthology.orchestra.run.smithi190.stdout:/sys/fs/cgroup/perf_event 2024-04-17T22:29:25.127 INFO:teuthology.orchestra.run.smithi190.stdout:/sys/kernel/tracing 2024-04-17T22:29:25.128 INFO:teuthology.orchestra.run.smithi190.stdout:/sys/kernel/config 2024-04-17T22:29:25.128 INFO:teuthology.orchestra.run.smithi190.stdout:/ 2024-04-17T22:29:25.128 INFO:teuthology.orchestra.run.smithi190.stdout:/sys/fs/selinux 2024-04-17T22:29:25.128 INFO:teuthology.orchestra.run.smithi190.stdout:/sys/kernel/debug 2024-04-17T22:29:25.128 INFO:teuthology.orchestra.run.smithi190.stdout:/dev/hugepages 2024-04-17T22:29:25.128 INFO:teuthology.orchestra.run.smithi190.stdout:/dev/mqueue 2024-04-17T22:29:25.128 INFO:teuthology.orchestra.run.smithi190.stdout:/proc/sys/fs/binfmt_misc 2024-04-17T22:29:25.129 INFO:teuthology.orchestra.run.smithi190.stdout:/sys/fs/fuse/connections 2024-04-17T22:29:25.129 INFO:teuthology.orchestra.run.smithi190.stdout:/var/lib/nfs/rpc_pipefs 2024-04-17T22:29:25.129 INFO:teuthology.orchestra.run.smithi190.stdout:/run/user/1000 2024-04-17T22:29:25.129 INFO:teuthology.orchestra.run.smithi190.stdout:/var/lib/ceph 2024-04-17T22:29:25.129 INFO:teuthology.orchestra.run.smithi190.stdout:/run/netns 2024-04-17T22:29:25.129 INFO:teuthology.orchestra.run.smithi190.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-17T22:29:25.129 INFO:teuthology.orchestra.run.smithi190.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-17T22:29:25.130 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T22:29:25.130 DEBUG:teuthology.orchestra.run.smithi190:> ls /sys/fs/fuse/connections 2024-04-17T22:29:25.182 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-17T22:29:25.182 DEBUG:teuthology.orchestra.run.smithi190:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-17T22:29:25.224 DEBUG:teuthology.orchestra.run.smithi190:> sudo modprobe fuse 2024-04-17T22:29:25.250 DEBUG:teuthology.orchestra.run.smithi190:> cat /proc/self/mounts | awk '{print $2}' 2024-04-17T22:29:25.305 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi190.stderr:warning: line 130: 'mon_down_mkfs_grace' in section 'mon' redefined 2024-04-17T22:29:25.309 INFO:teuthology.orchestra.run.smithi190.stdout:/sys 2024-04-17T22:29:25.309 INFO:teuthology.orchestra.run.smithi190.stdout:/proc 2024-04-17T22:29:25.309 INFO:teuthology.orchestra.run.smithi190.stdout:/dev 2024-04-17T22:29:25.309 INFO:teuthology.orchestra.run.smithi190.stdout:/sys/kernel/security 2024-04-17T22:29:25.309 INFO:teuthology.orchestra.run.smithi190.stdout:/dev/shm 2024-04-17T22:29:25.309 INFO:teuthology.orchestra.run.smithi190.stdout:/dev/pts 2024-04-17T22:29:25.310 INFO:teuthology.orchestra.run.smithi190.stdout:/run 2024-04-17T22:29:25.310 INFO:teuthology.orchestra.run.smithi190.stdout:/sys/fs/cgroup 2024-04-17T22:29:25.310 INFO:teuthology.orchestra.run.smithi190.stdout:/sys/fs/cgroup/systemd 2024-04-17T22:29:25.310 INFO:teuthology.orchestra.run.smithi190.stdout:/sys/fs/pstore 2024-04-17T22:29:25.310 INFO:teuthology.orchestra.run.smithi190.stdout:/sys/fs/bpf 2024-04-17T22:29:25.310 INFO:teuthology.orchestra.run.smithi190.stdout:/sys/fs/cgroup/pids 2024-04-17T22:29:25.310 INFO:teuthology.orchestra.run.smithi190.stdout:/sys/fs/cgroup/cpu,cpuacct 2024-04-17T22:29:25.310 INFO:teuthology.orchestra.run.smithi190.stdout:/sys/fs/cgroup/net_cls,net_prio 2024-04-17T22:29:25.310 INFO:teuthology.orchestra.run.smithi190.stdout:/sys/fs/cgroup/rdma 2024-04-17T22:29:25.310 INFO:teuthology.orchestra.run.smithi190.stdout:/sys/fs/cgroup/hugetlb 2024-04-17T22:29:25.310 INFO:teuthology.orchestra.run.smithi190.stdout:/sys/fs/cgroup/blkio 2024-04-17T22:29:25.310 INFO:teuthology.orchestra.run.smithi190.stdout:/sys/fs/cgroup/freezer 2024-04-17T22:29:25.310 INFO:teuthology.orchestra.run.smithi190.stdout:/sys/fs/cgroup/cpuset 2024-04-17T22:29:25.310 INFO:teuthology.orchestra.run.smithi190.stdout:/sys/fs/cgroup/devices 2024-04-17T22:29:25.310 INFO:teuthology.orchestra.run.smithi190.stdout:/sys/fs/cgroup/memory 2024-04-17T22:29:25.311 INFO:teuthology.orchestra.run.smithi190.stdout:/sys/fs/cgroup/perf_event 2024-04-17T22:29:25.311 INFO:teuthology.orchestra.run.smithi190.stdout:/sys/kernel/tracing 2024-04-17T22:29:25.311 INFO:teuthology.orchestra.run.smithi190.stdout:/sys/kernel/config 2024-04-17T22:29:25.311 INFO:teuthology.orchestra.run.smithi190.stdout:/ 2024-04-17T22:29:25.311 INFO:teuthology.orchestra.run.smithi190.stdout:/sys/fs/selinux 2024-04-17T22:29:25.311 INFO:teuthology.orchestra.run.smithi190.stdout:/sys/kernel/debug 2024-04-17T22:29:25.311 INFO:teuthology.orchestra.run.smithi190.stdout:/dev/hugepages 2024-04-17T22:29:25.311 INFO:teuthology.orchestra.run.smithi190.stdout:/dev/mqueue 2024-04-17T22:29:25.311 INFO:teuthology.orchestra.run.smithi190.stdout:/proc/sys/fs/binfmt_misc 2024-04-17T22:29:25.311 INFO:teuthology.orchestra.run.smithi190.stdout:/sys/fs/fuse/connections 2024-04-17T22:29:25.311 INFO:teuthology.orchestra.run.smithi190.stdout:/var/lib/nfs/rpc_pipefs 2024-04-17T22:29:25.311 INFO:teuthology.orchestra.run.smithi190.stdout:/run/user/1000 2024-04-17T22:29:25.311 INFO:teuthology.orchestra.run.smithi190.stdout:/var/lib/ceph 2024-04-17T22:29:25.311 INFO:teuthology.orchestra.run.smithi190.stdout:/run/netns 2024-04-17T22:29:25.311 INFO:teuthology.orchestra.run.smithi190.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-17T22:29:25.311 INFO:teuthology.orchestra.run.smithi190.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-17T22:29:25.312 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T22:29:25.312 DEBUG:teuthology.orchestra.run.smithi190:> ls /sys/fs/fuse/connections 2024-04-17T22:29:25.357 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi190.stderr:2024-04-17 22:29:25.371 7f7ca547a200 -1 init, newargv = 0x55f3aea34a10 newargc=9 2024-04-17T22:29:25.357 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi190.stderr:ceph-fuse[40530]: starting ceph client 2024-04-17T22:29:25.362 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi190.stderr:ceph-fuse[40530]: starting fuse 2024-04-17T22:29:25.368 INFO:teuthology.orchestra.run.smithi190.stdout:46 2024-04-17T22:29:25.368 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [46] 2024-04-17T22:29:25.368 DEBUG:teuthology.orchestra.run.smithi190:> sudo stdin-killer -- python3 -c ' 2024-04-17T22:29:25.368 DEBUG:teuthology.orchestra.run.smithi190:> import glob 2024-04-17T22:29:25.368 DEBUG:teuthology.orchestra.run.smithi190:> import re 2024-04-17T22:29:25.368 DEBUG:teuthology.orchestra.run.smithi190:> import os 2024-04-17T22:29:25.368 DEBUG:teuthology.orchestra.run.smithi190:> import subprocess 2024-04-17T22:29:25.368 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T22:29:25.369 DEBUG:teuthology.orchestra.run.smithi190:> def _find_admin_socket(client_name): 2024-04-17T22:29:25.369 DEBUG:teuthology.orchestra.run.smithi190:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-17T22:29:25.369 DEBUG:teuthology.orchestra.run.smithi190:> files = glob.glob(asok_path) 2024-04-17T22:29:25.369 DEBUG:teuthology.orchestra.run.smithi190:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-17T22:29:25.369 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T22:29:25.369 DEBUG:teuthology.orchestra.run.smithi190:> # Given a non-glob path, it better be there 2024-04-17T22:29:25.369 DEBUG:teuthology.orchestra.run.smithi190:> if "*" not in asok_path: 2024-04-17T22:29:25.369 DEBUG:teuthology.orchestra.run.smithi190:> assert(len(files) == 1) 2024-04-17T22:29:25.369 DEBUG:teuthology.orchestra.run.smithi190:> return files[0] 2024-04-17T22:29:25.369 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T22:29:25.369 DEBUG:teuthology.orchestra.run.smithi190:> for f in files: 2024-04-17T22:29:25.369 DEBUG:teuthology.orchestra.run.smithi190:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-17T22:29:25.369 DEBUG:teuthology.orchestra.run.smithi190:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-17T22:29:25.369 DEBUG:teuthology.orchestra.run.smithi190:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-17T22:29:25.369 DEBUG:teuthology.orchestra.run.smithi190:> contents = proc_f.read() 2024-04-17T22:29:25.369 DEBUG:teuthology.orchestra.run.smithi190:> if mountpoint in contents: 2024-04-17T22:29:25.369 DEBUG:teuthology.orchestra.run.smithi190:> return f 2024-04-17T22:29:25.369 DEBUG:teuthology.orchestra.run.smithi190:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-17T22:29:25.370 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T22:29:25.370 DEBUG:teuthology.orchestra.run.smithi190:> print(_find_admin_socket("client.0")) 2024-04-17T22:29:25.370 DEBUG:teuthology.orchestra.run.smithi190:> ' 2024-04-17T22:29:25.492 INFO:teuthology.orchestra.run.smithi190.stdout:/var/run/ceph/ceph-client.0.40530.asok 2024-04-17T22:29:25.495 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T22:29:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T22:29:25.501 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.40530.asok 2024-04-17T22:29:25.501 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T22:29:25.501 DEBUG:teuthology.orchestra.run.smithi190:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.40530.asok status 2024-04-17T22:29:25.641 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-04-17T22:29:25.641 INFO:teuthology.orchestra.run.smithi190.stdout: "metadata": { 2024-04-17T22:29:25.641 INFO:teuthology.orchestra.run.smithi190.stdout: "ceph_sha1": "ca74598065096e6fcbd8433c8779a2be0c889351", 2024-04-17T22:29:25.641 INFO:teuthology.orchestra.run.smithi190.stdout: "ceph_version": "ceph version 14.2.22 (ca74598065096e6fcbd8433c8779a2be0c889351) nautilus (stable)", 2024-04-17T22:29:25.641 INFO:teuthology.orchestra.run.smithi190.stdout: "entity_id": "0", 2024-04-17T22:29:25.641 INFO:teuthology.orchestra.run.smithi190.stdout: "hostname": "smithi190", 2024-04-17T22:29:25.641 INFO:teuthology.orchestra.run.smithi190.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-17T22:29:25.642 INFO:teuthology.orchestra.run.smithi190.stdout: "pid": "40530", 2024-04-17T22:29:25.642 INFO:teuthology.orchestra.run.smithi190.stdout: "root": "/" 2024-04-17T22:29:25.642 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-04-17T22:29:25.642 INFO:teuthology.orchestra.run.smithi190.stdout: "dentry_count": 0, 2024-04-17T22:29:25.642 INFO:teuthology.orchestra.run.smithi190.stdout: "dentry_pinned_count": 0, 2024-04-17T22:29:25.642 INFO:teuthology.orchestra.run.smithi190.stdout: "id": 4483, 2024-04-17T22:29:25.642 INFO:teuthology.orchestra.run.smithi190.stdout: "inst": { 2024-04-17T22:29:25.642 INFO:teuthology.orchestra.run.smithi190.stdout: "name": { 2024-04-17T22:29:25.642 INFO:teuthology.orchestra.run.smithi190.stdout: "type": "client", 2024-04-17T22:29:25.642 INFO:teuthology.orchestra.run.smithi190.stdout: "num": 4483 2024-04-17T22:29:25.642 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-04-17T22:29:25.642 INFO:teuthology.orchestra.run.smithi190.stdout: "addr": { 2024-04-17T22:29:25.642 INFO:teuthology.orchestra.run.smithi190.stdout: "type": "v1", 2024-04-17T22:29:25.642 INFO:teuthology.orchestra.run.smithi190.stdout: "addr": "172.21.15.190:0", 2024-04-17T22:29:25.643 INFO:teuthology.orchestra.run.smithi190.stdout: "nonce": 122713038 2024-04-17T22:29:25.643 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-04-17T22:29:25.643 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-04-17T22:29:25.643 INFO:teuthology.orchestra.run.smithi190.stdout: "addr": { 2024-04-17T22:29:25.643 INFO:teuthology.orchestra.run.smithi190.stdout: "type": "v1", 2024-04-17T22:29:25.643 INFO:teuthology.orchestra.run.smithi190.stdout: "addr": "172.21.15.190:0", 2024-04-17T22:29:25.643 INFO:teuthology.orchestra.run.smithi190.stdout: "nonce": 122713038 2024-04-17T22:29:25.643 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-04-17T22:29:25.643 INFO:teuthology.orchestra.run.smithi190.stdout: "inst_str": "client.4483 172.21.15.190:0/122713038", 2024-04-17T22:29:25.643 INFO:teuthology.orchestra.run.smithi190.stdout: "addr_str": "172.21.15.190:0/122713038", 2024-04-17T22:29:25.643 INFO:teuthology.orchestra.run.smithi190.stdout: "inode_count": 1, 2024-04-17T22:29:25.643 INFO:teuthology.orchestra.run.smithi190.stdout: "mds_epoch": 5, 2024-04-17T22:29:25.643 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_epoch": 17, 2024-04-17T22:29:25.644 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_epoch_barrier": 0, 2024-04-17T22:29:25.644 INFO:teuthology.orchestra.run.smithi190.stdout: "blacklisted": false, 2024-04-17T22:29:25.644 INFO:teuthology.orchestra.run.smithi190.stdout: "fs_name": "cephfs" 2024-04-17T22:29:25.644 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-04-17T22:29:25.650 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T22:29:25.650 DEBUG:teuthology.orchestra.run.smithi190:> stat --file-system '--printf=%T 2024-04-17T22:29:25.650 DEBUG:teuthology.orchestra.run.smithi190:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-17T22:29:25.668 INFO:teuthology.orchestra.run.smithi190.stdout:fuseblk 2024-04-17T22:29:25.668 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-17T22:29:25.669 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T22:29:25.669 DEBUG:teuthology.orchestra.run.smithi190:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-17T22:29:25.736 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-04-17T22:29:25.746 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-04-17T22:29:25.746 INFO:tasks.ceph_fuse:config is {'client.0': {'mounted': False}} 2024-04-17T22:29:25.746 INFO:tasks.ceph_fuse:client.0 config is {'mounted': False} 2024-04-17T22:29:25.746 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-04-17T22:29:25.746 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T22:29:25.795 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-04-17T22:29:25.795 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T22:29:25.853 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T22:29:25.991 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.002 7f8024d40700 1 Processor -- start 2024-04-17T22:29:25.991 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.002 7f8024d40700 1 -- start start 2024-04-17T22:29:25.991 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.003 7f8024d40700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f802010c880 0x7f80201a6270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:25.992 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.003 7f8024d40700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f80201a68a0 0x7f80201a8d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:25.992 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.003 7f8024d40700 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80201a1b80 con 0x7f80200f56e0 2024-04-17T22:29:25.992 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.003 7f8024d40700 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80200f9350 con 0x7f80201a68a0 2024-04-17T22:29:25.992 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.003 7f8024d40700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f802010ccd0 con 0x7f802010c880 2024-04-17T22:29:25.992 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.003 7f8017fff700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f802010c880 0x7f80201a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:25.992 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.003 7f801e311700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f80201a68a0 0x7f80201a8d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:25.992 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.003 7f8017fff700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f802010c880 0x7f80201a6270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:55998/0 (socket says 172.21.15.102:55998) 2024-04-17T22:29:25.992 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.003 7f8017fff700 1 -- 172.21.15.102:0/1795431 learned_addr learned my addr 172.21.15.102:0/1795431 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:29:25.992 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.003 7f801db10700 1 --1- 172.21.15.102:0/1795431 >> v1:172.21.15.102:6789/0 conn(0x7f80200f56e0 0x7f802010c080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:49186/0 (socket says 172.21.15.102:49186) 2024-04-17T22:29:25.992 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.003 7f801e311700 1 --2- 172.21.15.102:0/1795431 >> v2:172.21.15.102:3300/0 conn(0x7f80201a68a0 0x7f80201a8d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:37948/0 (socket says 172.21.15.102:37948) 2024-04-17T22:29:25.992 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.003 7f8017fff700 1 -- 172.21.15.102:0/1795431 >> v2:172.21.15.102:3300/0 conn(0x7f80201a68a0 msgr2=0x7f80201a8d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:29:25.992 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.003 7f8017fff700 1 --2- 172.21.15.102:0/1795431 >> v2:172.21.15.102:3300/0 conn(0x7f80201a68a0 0x7f80201a8d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:25.992 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.003 7f8017fff700 1 -- 172.21.15.102:0/1795431 >> v1:172.21.15.102:6789/0 conn(0x7f80200f56e0 legacy=0x7f802010c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:25.992 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.003 7f8017fff700 1 -- 172.21.15.102:0/1795431 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80201aa340 con 0x7f802010c880 2024-04-17T22:29:25.993 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.003 7f801d30f700 1 -- 172.21.15.102:0/1795431 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3231577658 0 0) 0x7f80201a1b80 con 0x7f80200f56e0 2024-04-17T22:29:25.993 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.003 7f801e311700 1 --2- 172.21.15.102:0/1795431 >> v2:172.21.15.102:3300/0 conn(0x7f80201a68a0 0x7f80201a8d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:29:25.993 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.003 7f8017fff700 1 --2- 172.21.15.102:0/1795431 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f802010c880 0x7f80201a6270 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7f8008002f70 tx=0x7f80080099d0).ready entity=mon.1 client_cookie=6f122377e36662dd server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:29:25.993 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.003 7f801d30f700 1 -- 172.21.15.102:0/1795431 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f80201aa340 con 0x7f802010c880 2024-04-17T22:29:25.993 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.003 7f801d30f700 1 -- 172.21.15.102:0/1795431 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f800800f510 con 0x7f802010c880 2024-04-17T22:29:25.993 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.003 7f801d30f700 1 -- 172.21.15.102:0/1795431 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f800800f6f0 con 0x7f802010c880 2024-04-17T22:29:25.993 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.003 7f8024d40700 1 -- 172.21.15.102:0/1795431 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f802010c880 msgr2=0x7f80201a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:25.993 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.003 7f8024d40700 1 --2- 172.21.15.102:0/1795431 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f802010c880 0x7f80201a6270 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7f8008002f70 tx=0x7f80080099d0).stop 2024-04-17T22:29:25.993 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.004 7f8024d40700 1 -- 172.21.15.102:0/1795431 shutdown_connections 2024-04-17T22:29:25.993 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.004 7f8024d40700 1 --2- 172.21.15.102:0/1795431 >> v2:172.21.15.102:3300/0 conn(0x7f80201a68a0 0x7f80201a8d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:25.993 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.004 7f8024d40700 1 --2- 172.21.15.102:0/1795431 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f802010c880 0x7f80201a6270 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:25.993 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.004 7f8024d40700 1 -- 172.21.15.102:0/1795431 >> 172.21.15.102:0/1795431 conn(0x7f8020113830 msgr2=0x7f80200f4ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:29:25.993 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.004 7f8024d40700 1 -- 172.21.15.102:0/1795431 shutdown_connections 2024-04-17T22:29:25.994 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.004 7f8024d40700 1 -- 172.21.15.102:0/1795431 wait complete. 2024-04-17T22:29:26.016 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.027 7f8024d40700 1 Processor -- start 2024-04-17T22:29:26.016 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.027 7f8024d40700 1 -- start start 2024-04-17T22:29:26.016 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.027 7f8024d40700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f80200d01f0 0x7f80200d7e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:26.016 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.027 7f8024d40700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f80200d8490 0x7f80201b3d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:26.016 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.027 7f8024d40700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f80200d88e0 0x7f80201b4350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:26.016 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.027 7f8024d40700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80200f7730 con 0x7f80200d01f0 2024-04-17T22:29:26.016 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.027 7f8024d40700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80200de250 con 0x7f80200d8490 2024-04-17T22:29:26.016 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.027 7f8024d40700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f80201b4980 con 0x7f80200d88e0 2024-04-17T22:29:26.016 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.027 7f801db10700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f80200d01f0 0x7f80200d7e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:26.016 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.027 7f8017fff700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f80200d8490 0x7f80201b3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:26.016 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.027 7f8017fff700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f80200d8490 0x7f80201b3d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:56010/0 (socket says 172.21.15.102:56010) 2024-04-17T22:29:26.016 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.027 7f8017fff700 1 -- 172.21.15.102:0/1424169775 learned_addr learned my addr 172.21.15.102:0/1424169775 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:29:26.016 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.027 7f801db10700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f80200d01f0 0x7f80200d7e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:37958/0 (socket says 172.21.15.102:37958) 2024-04-17T22:29:26.016 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.027 7f801e311700 1 --2- 172.21.15.102:0/1424169775 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f80200d88e0 0x7f80201b4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:26.017 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.028 7f8017fff700 1 -- 172.21.15.102:0/1424169775 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f80200d88e0 msgr2=0x7f80201b4350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:26.017 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.028 7f8017fff700 1 --2- 172.21.15.102:0/1424169775 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f80200d88e0 0x7f80201b4350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:26.017 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.028 7f8017fff700 1 -- 172.21.15.102:0/1424169775 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f80200d01f0 msgr2=0x7f80200d7e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:26.017 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.028 7f8017fff700 1 --2- 172.21.15.102:0/1424169775 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f80200d01f0 0x7f80200d7e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:26.017 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.028 7f8017fff700 1 -- 172.21.15.102:0/1424169775 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80201b4bb0 con 0x7f80200d8490 2024-04-17T22:29:26.017 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.028 7f801e311700 1 --2- 172.21.15.102:0/1424169775 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f80200d88e0 0x7f80201b4350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:29:26.017 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.028 7f801db10700 1 --2- 172.21.15.102:0/1424169775 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f80200d01f0 0x7f80200d7e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:29:26.017 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.028 7f8017fff700 1 --2- 172.21.15.102:0/1424169775 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f80200d8490 0x7f80201b3d20 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0x7f80200d9500 tx=0x7f80080068c0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:29:26.017 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.028 7f8016ffd700 1 -- 172.21.15.102:0/1424169775 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f80201b4bb0 con 0x7f80200d8490 2024-04-17T22:29:26.017 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.028 7f8016ffd700 1 -- 172.21.15.102:0/1424169775 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8008011860 con 0x7f80200d8490 2024-04-17T22:29:26.017 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.028 7f8024d40700 1 -- 172.21.15.102:0/1424169775 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80201bb410 con 0x7f80200d8490 2024-04-17T22:29:26.017 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.028 7f8016ffd700 1 -- 172.21.15.102:0/1424169775 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8008011c30 con 0x7f80200d8490 2024-04-17T22:29:26.017 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.028 7f8024d40700 1 -- 172.21.15.102:0/1424169775 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80201bb7f0 con 0x7f80200d8490 2024-04-17T22:29:26.017 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.028 7f8016ffd700 1 -- 172.21.15.102:0/1424169775 <== mon.1 v2:172.21.15.102:3301/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7f8008019030 con 0x7f80200d8490 2024-04-17T22:29:26.018 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.028 7f8016ffd700 1 -- 172.21.15.102:0/1424169775 <== mon.1 v2:172.21.15.102:3301/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3194+0+0 (secure 0 0 0) 0x7f8008018040 con 0x7f80200d8490 2024-04-17T22:29:26.018 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.029 7f8024d40700 1 -- 172.21.15.102:0/1424169775 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80200f0790 con 0x7f80200d8490 2024-04-17T22:29:26.018 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.032 7f8016ffd700 1 -- 172.21.15.102:0/1424169775 <== mon.1 v2:172.21.15.102:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+110455 (secure 0 0 0) 0x7f8008011650 con 0x7f80200d8490 2024-04-17T22:29:26.123 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.137 7f8024d40700 1 -- 172.21.15.102:0/1424169775 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f80201ae390 con 0x7f80200d8490 2024-04-17T22:29:26.123 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.137 7f8016ffd700 1 -- 172.21.15.102:0/1424169775 <== mon.1 v2:172.21.15.102:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) v1 ==== 74+0+8764 (secure 0 0 0) 0x7f8008037250 con 0x7f80200d8490 2024-04-17T22:29:26.124 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-17T22:29:26.124 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":17,"fsid":"cb63e789-091d-49b1-b120-55f9db526871","created":"2024-04-17 22:28:48.209635","modified":"2024-04-17 22:29:23.257387","last_up_change":"2024-04-17 22:28:54.236215","last_in_change":"2024-04-17 22:28:54.236215","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":4,"require_min_compat_client":"nautilus","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-04-17 22:28:57.478545","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-17 22:29:06.144708","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-17 22:29:06.748924","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3351ef85-ade4-4b90-a8f4-213f4e00ba01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6812","nonce":42955}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6813","nonce":42955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6815","nonce":42955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6814","nonce":42955}]},"public_addr":"172.21.15.102:6812/42955","cluster_addr":"172.21.15.102:6813/42955","heartbeat_back_addr":"172.21.15.102:6815/42955","heartbeat_front_addr":"172.21.15.102:6814/42955","state":["exists","up"]},{"osd":1,"uuid":"04f67eac-f36b-48f5-982d-4cfc776596d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6808","nonce":42958}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6809","nonce":42958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6811","nonce":42958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6810","nonce":42958}]},"public_addr":"172.21.15.102:6808/42958","cluster_addr":"172.21.15.102:6809/42958","heartbeat_back_addr":"172.21.15.102:6811/42958","heartbeat_front_addr":"172.21.15.102:6810/42958","state":["exists","up"]},{"osd":2,"uuid":"d524204b-70b2-40a8-8f73-f9e7d44318af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6804","nonce":42954}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6805","nonce":42954}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6807","nonce":42954}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6806","nonce":42954}]},"public_addr":"172.21.15.102:6804/42954","cluster_addr":"172.21.15.102:6805/42954","heartbeat_back_addr":"172.21.15.102:6807/42954","heartbeat_front_addr":"172.21.15.102:6806/42954","state":["exists","up"]},{"osd":3,"uuid":"4fc9d20c-1ad0-484f-aafa-2c9a7574245b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6800","nonce":42953}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6801","nonce":42953}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6803","nonce":42953}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6802","nonce":42953}]},"public_addr":"172.21.15.102:6800/42953","cluster_addr":"172.21.15.102:6801/42953","heartbeat_back_addr":"172.21.15.102:6803/42953","heartbeat_front_addr":"172.21.15.102:6802/42953","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2024-04-17T22:29:26.125 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.138 7f8024d40700 1 -- 172.21.15.102:0/1424169775 >> v1:172.21.15.102:6816/42360 conn(0x7f80040145a0 legacy=0x7f8004016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:26.126 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.138 7f8024d40700 1 -- 172.21.15.102:0/1424169775 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f80200d8490 msgr2=0x7f80201b3d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:26.126 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.138 7f8024d40700 1 --2- 172.21.15.102:0/1424169775 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f80200d8490 0x7f80201b3d20 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0x7f80200d9500 tx=0x7f80080068c0).stop 2024-04-17T22:29:26.126 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.139 7f8024d40700 1 -- 172.21.15.102:0/1424169775 shutdown_connections 2024-04-17T22:29:26.126 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.139 7f8024d40700 1 --2- 172.21.15.102:0/1424169775 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f80200d88e0 0x7f80201b4350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:26.126 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.139 7f8024d40700 1 --2- 172.21.15.102:0/1424169775 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f80200d01f0 0x7f80200d7e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:26.126 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.139 7f8024d40700 1 --2- 172.21.15.102:0/1424169775 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f80200d8490 0x7f80201b3d20 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:26.126 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.139 7f8024d40700 1 -- 172.21.15.102:0/1424169775 >> 172.21.15.102:0/1424169775 conn(0x7f80200c52b0 msgr2=0x7f80200cc580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:29:26.126 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.139 7f8024d40700 1 -- 172.21.15.102:0/1424169775 shutdown_connections 2024-04-17T22:29:26.126 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.139 7f8024d40700 1 -- 172.21.15.102:0/1424169775 wait complete. 2024-04-17T22:29:26.136 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2024-04-17 22:28:57.478545', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '12', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-17 22:29:06.144708', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-17 22:29:06.748924', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-17T22:29:26.137 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2024-04-17T22:29:26.266 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.278 7fd2e9842700 1 Processor -- start 2024-04-17T22:29:26.267 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.278 7fd2e9842700 1 -- start start 2024-04-17T22:29:26.267 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.278 7fd2e9842700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fd2e419fde0 0x7fd2e41a0230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:26.267 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.278 7fd2e9842700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd2e41a0860 0x7fd2e41a3020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:26.267 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.278 7fd2e9842700 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2e40ad030 con 0x7fd2e41a3650 2024-04-17T22:29:26.267 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.278 7fd2e9842700 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2e40f9d40 con 0x7fd2e419fde0 2024-04-17T22:29:26.267 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.278 7fd2e9842700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2e40f2c90 con 0x7fd2e41a0860 2024-04-17T22:29:26.267 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.279 7fd2e1dc4700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd2e41a0860 0x7fd2e41a3020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:26.267 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.279 7fd2e2dc6700 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7fd2e41a3650 0x7fd2e41aa6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:49188/0 (socket says 172.21.15.102:49188) 2024-04-17T22:29:26.267 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.279 7fd2e2dc6700 1 -- 172.21.15.102:0/856157096 learned_addr learned my addr 172.21.15.102:0/856157096 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-17T22:29:26.267 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.279 7fd2e25c5700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fd2e419fde0 0x7fd2e41a0230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:26.267 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.279 7fd2e1dc4700 1 -- 172.21.15.102:0/856157096 >> v2:172.21.15.102:3300/0 conn(0x7fd2e419fde0 msgr2=0x7fd2e41a0230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:29:26.267 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.279 7fd2e1dc4700 1 --2- 172.21.15.102:0/856157096 >> v2:172.21.15.102:3300/0 conn(0x7fd2e419fde0 0x7fd2e41a0230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:26.267 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.279 7fd2e1dc4700 1 -- 172.21.15.102:0/856157096 >> v1:172.21.15.102:6789/0 conn(0x7fd2e41a3650 legacy=0x7fd2e41aa6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:26.267 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.279 7fd2e1dc4700 1 -- 172.21.15.102:0/856157096 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2e40f9a90 con 0x7fd2e41a0860 2024-04-17T22:29:26.268 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.279 7fd2e1dc4700 1 --2- 172.21.15.102:0/856157096 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd2e41a0860 0x7fd2e41a3020 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0x7fd2d4002f70 tx=0x7fd2d400b970).ready entity=mon.1 client_cookie=c2755e2a45200e18 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:29:26.268 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.279 7fd2e15c3700 1 -- 172.21.15.102:0/856157096 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd2e40f9a90 con 0x7fd2e41a0860 2024-04-17T22:29:26.268 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.279 7fd2e15c3700 1 -- 172.21.15.102:0/856157096 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2d400d5c0 con 0x7fd2e41a0860 2024-04-17T22:29:26.268 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.279 7fd2e15c3700 1 -- 172.21.15.102:0/856157096 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd2d400d990 con 0x7fd2e41a0860 2024-04-17T22:29:26.268 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.279 7fd2e9842700 1 -- 172.21.15.102:0/856157096 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd2e41a0860 msgr2=0x7fd2e41a3020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:26.268 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.279 7fd2e9842700 1 --2- 172.21.15.102:0/856157096 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd2e41a0860 0x7fd2e41a3020 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0x7fd2d4002f70 tx=0x7fd2d400b970).stop 2024-04-17T22:29:26.268 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.280 7fd2e9842700 1 -- 172.21.15.102:0/856157096 shutdown_connections 2024-04-17T22:29:26.268 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.280 7fd2e9842700 1 --2- 172.21.15.102:0/856157096 >> v2:172.21.15.102:3300/0 conn(0x7fd2e419fde0 0x7fd2e41a0230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:26.268 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.280 7fd2e9842700 1 --2- 172.21.15.102:0/856157096 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd2e41a0860 0x7fd2e41a3020 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:26.268 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.280 7fd2e9842700 1 -- 172.21.15.102:0/856157096 >> 172.21.15.102:0/856157096 conn(0x7fd2e4113000 msgr2=0x7fd2e4113450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:29:26.268 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.280 7fd2e9842700 1 -- 172.21.15.102:0/856157096 shutdown_connections 2024-04-17T22:29:26.268 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.280 7fd2e9842700 1 -- 172.21.15.102:0/856157096 wait complete. 2024-04-17T22:29:26.291 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.303 7fd2e9842700 1 Processor -- start 2024-04-17T22:29:26.291 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.303 7fd2e9842700 1 -- start start 2024-04-17T22:29:26.291 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.303 7fd2e9842700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd2e40dc990 0x7fd2e419ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:26.291 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.303 7fd2e9842700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd2e419e3f0 0x7fd2e41b3e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:26.291 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.303 7fd2e9842700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd2e419e840 0x7fd2e41b44c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:26.291 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.303 7fd2e9842700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2e40d7590 con 0x7fd2e419e3f0 2024-04-17T22:29:26.291 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.303 7fd2e9842700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2e41a2e20 con 0x7fd2e40dc990 2024-04-17T22:29:26.291 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.303 7fd2e9842700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd2e41b4af0 con 0x7fd2e419e840 2024-04-17T22:29:26.291 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.303 7fd2e25c5700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd2e40dc990 0x7fd2e419ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:26.292 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.303 7fd2e1dc4700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd2e419e3f0 0x7fd2e41b3e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:26.292 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.303 7fd2e1dc4700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd2e419e3f0 0x7fd2e41b3e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:37962/0 (socket says 172.21.15.102:37962) 2024-04-17T22:29:26.292 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.303 7fd2e1dc4700 1 -- 172.21.15.102:0/3901098991 learned_addr learned my addr 172.21.15.102:0/3901098991 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:29:26.292 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.303 7fd2e2dc6700 1 --2- 172.21.15.102:0/3901098991 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd2e419e840 0x7fd2e41b44c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:26.292 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.303 7fd2e25c5700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd2e40dc990 0x7fd2e419ddc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:56036/0 (socket says 172.21.15.102:56036) 2024-04-17T22:29:26.292 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.303 7fd2e25c5700 1 -- 172.21.15.102:0/3901098991 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd2e419e840 msgr2=0x7fd2e41b44c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:26.292 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.303 7fd2e25c5700 1 --2- 172.21.15.102:0/3901098991 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd2e419e840 0x7fd2e41b44c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:26.292 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.303 7fd2e25c5700 1 -- 172.21.15.102:0/3901098991 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd2e419e3f0 msgr2=0x7fd2e41b3e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:26.292 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.303 7fd2e25c5700 1 --2- 172.21.15.102:0/3901098991 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd2e419e3f0 0x7fd2e41b3e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:26.292 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.303 7fd2e25c5700 1 -- 172.21.15.102:0/3901098991 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2e41b4d20 con 0x7fd2e40dc990 2024-04-17T22:29:26.292 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.303 7fd2e1dc4700 1 --2- 172.21.15.102:0/3901098991 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd2e419e3f0 0x7fd2e41b3e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:29:26.292 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.303 7fd2e2dc6700 1 --2- 172.21.15.102:0/3901098991 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd2e419e840 0x7fd2e41b44c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:29:26.292 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.304 7fd2e25c5700 1 --2- 172.21.15.102:0/3901098991 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd2e40dc990 0x7fd2e419ddc0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7fd2d800c000 tx=0x7fd2d800ec70).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:29:26.292 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.304 7fd2d37fe700 1 -- 172.21.15.102:0/3901098991 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd2e41b4d20 con 0x7fd2e40dc990 2024-04-17T22:29:26.292 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.304 7fd2d37fe700 1 -- 172.21.15.102:0/3901098991 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2d800c9a0 con 0x7fd2e40dc990 2024-04-17T22:29:26.292 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.304 7fd2e9842700 1 -- 172.21.15.102:0/3901098991 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2e41bb690 con 0x7fd2e40dc990 2024-04-17T22:29:26.293 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.304 7fd2d37fe700 1 -- 172.21.15.102:0/3901098991 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd2d800f040 con 0x7fd2e40dc990 2024-04-17T22:29:26.293 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.304 7fd2e9842700 1 -- 172.21.15.102:0/3901098991 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2e41bba70 con 0x7fd2e40dc990 2024-04-17T22:29:26.293 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.304 7fd2d37fe700 1 -- 172.21.15.102:0/3901098991 <== mon.1 v2:172.21.15.102:3301/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7fd2d8011070 con 0x7fd2e40dc990 2024-04-17T22:29:26.293 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.304 7fd2d37fe700 1 -- 172.21.15.102:0/3901098991 <== mon.1 v2:172.21.15.102:3301/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3194+0+0 (secure 0 0 0) 0x7fd2d8010040 con 0x7fd2e40dc990 2024-04-17T22:29:26.293 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.305 7fd2e9842700 1 -- 172.21.15.102:0/3901098991 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2e40f09f0 con 0x7fd2e40dc990 2024-04-17T22:29:26.294 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.308 7fd2d37fe700 1 -- 172.21.15.102:0/3901098991 <== mon.1 v2:172.21.15.102:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+110455 (secure 0 0 0) 0x7fd2d800c5a0 con 0x7fd2e40dc990 2024-04-17T22:29:26.400 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.413 7fd2e9842700 1 -- 172.21.15.102:0/3901098991 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fd2e41ae5b0 con 0x7fd2e40dc990 2024-04-17T22:29:26.400 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.414 7fd2d37fe700 1 -- 172.21.15.102:0/3901098991 <== mon.1 v2:172.21.15.102:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v17) v1 ==== 92+0+10 (secure 0 0 0) 0x7fd2d8000c30 con 0x7fd2e40dc990 2024-04-17T22:29:26.400 INFO:teuthology.orchestra.run.smithi102.stdout:pg_num: 8 2024-04-17T22:29:26.402 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.415 7fd2e9842700 1 -- 172.21.15.102:0/3901098991 >> v1:172.21.15.102:6816/42360 conn(0x7fd2c00145a0 legacy=0x7fd2c0016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:26.402 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.415 7fd2e9842700 1 -- 172.21.15.102:0/3901098991 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd2e40dc990 msgr2=0x7fd2e419ddc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:26.402 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.415 7fd2e9842700 1 --2- 172.21.15.102:0/3901098991 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd2e40dc990 0x7fd2e419ddc0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7fd2d800c000 tx=0x7fd2d800ec70).stop 2024-04-17T22:29:26.402 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.415 7fd2e9842700 1 -- 172.21.15.102:0/3901098991 shutdown_connections 2024-04-17T22:29:26.402 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.415 7fd2e9842700 1 --2- 172.21.15.102:0/3901098991 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd2e419e840 0x7fd2e41b44c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:26.402 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.415 7fd2e9842700 1 --2- 172.21.15.102:0/3901098991 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd2e40dc990 0x7fd2e419ddc0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:26.402 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.415 7fd2e9842700 1 --2- 172.21.15.102:0/3901098991 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd2e419e3f0 0x7fd2e41b3e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:26.402 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.415 7fd2e9842700 1 -- 172.21.15.102:0/3901098991 >> 172.21.15.102:0/3901098991 conn(0x7fd2e40d7960 msgr2=0x7fd2e40d2510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:29:26.402 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.415 7fd2e9842700 1 -- 172.21.15.102:0/3901098991 shutdown_connections 2024-04-17T22:29:26.402 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.415 7fd2e9842700 1 -- 172.21.15.102:0/3901098991 wait complete. 2024-04-17T22:29:26.412 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-17T22:29:26.541 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.552 7fe98c2fd700 1 Processor -- start 2024-04-17T22:29:26.542 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.552 7fe98c2fd700 1 -- start start 2024-04-17T22:29:26.542 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.553 7fe98c2fd700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe98410b810 0x7fe9841a52c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:26.542 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.553 7fe98c2fd700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fe98410bc60 0x7fe9841a7900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:26.542 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.553 7fe98c2fd700 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe984103b40 con 0x7fe9840f5840 2024-04-17T22:29:26.542 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.553 7fe98c2fd700 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9840fa370 con 0x7fe98410bc60 2024-04-17T22:29:26.542 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.553 7fe98c2fd700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9841a0b00 con 0x7fe98410b810 2024-04-17T22:29:26.542 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.553 7fe9898f9700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fe98410bc60 0x7fe9841a7900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:26.542 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.553 7fe9888f7700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe98410b810 0x7fe9841a52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:26.542 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.553 7fe9898f9700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fe98410bc60 0x7fe9841a7900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:37964/0 (socket says 172.21.15.102:37964) 2024-04-17T22:29:26.542 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.553 7fe9898f9700 1 -- 172.21.15.102:0/3539131256 learned_addr learned my addr 172.21.15.102:0/3539131256 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:29:26.543 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.553 7fe9888f7700 1 --2- 172.21.15.102:0/3539131256 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe98410b810 0x7fe9841a52c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:56038/0 (socket says 172.21.15.102:56038) 2024-04-17T22:29:26.543 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.553 7fe9898f9700 1 -- 172.21.15.102:0/3539131256 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe98410b810 msgr2=0x7fe9841a52c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:29:26.543 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.553 7fe9898f9700 1 --2- 172.21.15.102:0/3539131256 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe98410b810 0x7fe9841a52c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:26.543 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.553 7fe9898f9700 1 -- 172.21.15.102:0/3539131256 >> v1:172.21.15.102:6789/0 conn(0x7fe9840f5840 legacy=0x7fe98410b010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:26.543 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.553 7fe9898f9700 1 -- 172.21.15.102:0/3539131256 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9841a8f40 con 0x7fe98410bc60 2024-04-17T22:29:26.543 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.553 7fe973fff700 1 -- 172.21.15.102:0/3539131256 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1781608938 0 0) 0x7fe974001110 con 0x7fe9840f5840 2024-04-17T22:29:26.543 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.553 7fe9888f7700 1 --2- 172.21.15.102:0/3539131256 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe98410b810 0x7fe9841a52c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:29:26.543 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.553 7fe9898f9700 1 --2- 172.21.15.102:0/3539131256 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe98410bc60 0x7fe9841a7900 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 rx=0x7fe97c0032f0 tx=0x7fe97c009de0).ready entity=mon.0 client_cookie=e88f0bf26c7a4c48 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:29:26.543 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.553 7fe973fff700 1 -- 172.21.15.102:0/3539131256 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe9841a8f40 con 0x7fe98410bc60 2024-04-17T22:29:26.543 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.553 7fe973fff700 1 -- 172.21.15.102:0/3539131256 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe97c00bac0 con 0x7fe98410bc60 2024-04-17T22:29:26.543 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.554 7fe973fff700 1 -- 172.21.15.102:0/3539131256 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe97c010470 con 0x7fe98410bc60 2024-04-17T22:29:26.543 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.554 7fe98c2fd700 1 -- 172.21.15.102:0/3539131256 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe98410bc60 msgr2=0x7fe9841a7900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:26.544 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.554 7fe98c2fd700 1 --2- 172.21.15.102:0/3539131256 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe98410bc60 0x7fe9841a7900 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 rx=0x7fe97c0032f0 tx=0x7fe97c009de0).stop 2024-04-17T22:29:26.544 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.554 7fe98c2fd700 1 -- 172.21.15.102:0/3539131256 shutdown_connections 2024-04-17T22:29:26.544 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.554 7fe98c2fd700 1 --2- 172.21.15.102:0/3539131256 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe98410bc60 0x7fe9841a7900 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:26.544 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.554 7fe98c2fd700 1 --2- 172.21.15.102:0/3539131256 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe98410b810 0x7fe9841a52c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:26.544 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.554 7fe98c2fd700 1 -- 172.21.15.102:0/3539131256 >> 172.21.15.102:0/3539131256 conn(0x7fe9841138a0 msgr2=0x7fe9840f4f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:29:26.544 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.554 7fe98c2fd700 1 -- 172.21.15.102:0/3539131256 shutdown_connections 2024-04-17T22:29:26.544 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.554 7fe98c2fd700 1 -- 172.21.15.102:0/3539131256 wait complete. 2024-04-17T22:29:26.563 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.577 7fe98c2fd700 1 Processor -- start 2024-04-17T22:29:26.564 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.577 7fe98c2fd700 1 -- start start 2024-04-17T22:29:26.566 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.577 7fe98c2fd700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fe9840d3540 0x7fe9841ad4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:26.566 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.578 7fe98c2fd700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe9841adb00 0x7fe9841b4050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:26.566 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.578 7fe98c2fd700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe9841adf50 0x7fe9841b4680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:26.566 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.578 7fe98c2fd700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9840d96d0 con 0x7fe9841adb00 2024-04-17T22:29:26.566 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.578 7fe98c2fd700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9840d9340 con 0x7fe9841adf50 2024-04-17T22:29:26.567 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.578 7fe98c2fd700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe9840d9990 con 0x7fe9840d3540 2024-04-17T22:29:26.567 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.578 7fe9888f7700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe9841adb00 0x7fe9841b4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:26.567 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.578 7fe9890f8700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fe9840d3540 0x7fe9841ad4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:26.567 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.578 7fe9888f7700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe9841adb00 0x7fe9841b4050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:37972/0 (socket says 172.21.15.102:37972) 2024-04-17T22:29:26.567 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.578 7fe9888f7700 1 -- 172.21.15.102:0/4060263383 learned_addr learned my addr 172.21.15.102:0/4060263383 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:29:26.567 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.578 7fe9898f9700 1 --2- 172.21.15.102:0/4060263383 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe9841adf50 0x7fe9841b4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:26.567 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.578 7fe9888f7700 1 -- 172.21.15.102:0/4060263383 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fe9840d3540 msgr2=0x7fe9841ad4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:26.567 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.578 7fe9888f7700 1 --2- 172.21.15.102:0/4060263383 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fe9840d3540 0x7fe9841ad4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:26.567 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.578 7fe9888f7700 1 -- 172.21.15.102:0/4060263383 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe9841adf50 msgr2=0x7fe9841b4680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:26.567 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.578 7fe9888f7700 1 --2- 172.21.15.102:0/4060263383 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe9841adf50 0x7fe9841b4680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:26.567 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.578 7fe9888f7700 1 -- 172.21.15.102:0/4060263383 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9841b4cb0 con 0x7fe9841adb00 2024-04-17T22:29:26.567 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.578 7fe9890f8700 1 --2- 172.21.15.102:0/4060263383 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fe9840d3540 0x7fe9841ad4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:29:26.567 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.578 7fe9898f9700 1 --2- 172.21.15.102:0/4060263383 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe9841adf50 0x7fe9841b4680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:29:26.567 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.578 7fe9888f7700 1 --2- 172.21.15.102:0/4060263383 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe9841adb00 0x7fe9841b4050 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 rx=0x7fe9840d94f0 tx=0x7fe98000bf70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:29:26.568 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.578 7fe9727fc700 1 -- 172.21.15.102:0/4060263383 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe9841b4cb0 con 0x7fe9841adb00 2024-04-17T22:29:26.568 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.578 7fe9727fc700 1 -- 172.21.15.102:0/4060263383 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe98000c9a0 con 0x7fe9841adb00 2024-04-17T22:29:26.568 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.578 7fe98c2fd700 1 -- 172.21.15.102:0/4060263383 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9841b4fd0 con 0x7fe9841adb00 2024-04-17T22:29:26.568 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.578 7fe9727fc700 1 -- 172.21.15.102:0/4060263383 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe98000f040 con 0x7fe9841adb00 2024-04-17T22:29:26.568 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.579 7fe98c2fd700 1 -- 172.21.15.102:0/4060263383 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9841bb920 con 0x7fe9841adb00 2024-04-17T22:29:26.568 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.579 7fe9727fc700 1 -- 172.21.15.102:0/4060263383 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7fe980011070 con 0x7fe9841adb00 2024-04-17T22:29:26.568 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.579 7fe9727fc700 1 -- 172.21.15.102:0/4060263383 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3194+0+0 (secure 0 0 0) 0x7fe980010040 con 0x7fe9841adb00 2024-04-17T22:29:26.568 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.580 7fe98c2fd700 1 -- 172.21.15.102:0/4060263383 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9840f0800 con 0x7fe9841adb00 2024-04-17T22:29:26.569 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.583 7fe9727fc700 1 -- 172.21.15.102:0/4060263383 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+110455 (secure 0 0 0) 0x7fe98000c5a0 con 0x7fe9841adb00 2024-04-17T22:29:26.675 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.689 7fe98c2fd700 1 -- 172.21.15.102:0/4060263383 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe9840e21f0 con 0x7fe9841adb00 2024-04-17T22:29:26.676 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.689 7fe9727fc700 1 -- 172.21.15.102:0/4060263383 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v17) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe980000c30 con 0x7fe9841adb00 2024-04-17T22:29:26.676 INFO:teuthology.orchestra.run.smithi102.stdout:pg_num: 64 2024-04-17T22:29:26.677 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.690 7fe98c2fd700 1 -- 172.21.15.102:0/4060263383 >> v1:172.21.15.102:6816/42360 conn(0x7fe9640145a0 legacy=0x7fe964016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:26.677 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.690 7fe98c2fd700 1 -- 172.21.15.102:0/4060263383 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe9841adb00 msgr2=0x7fe9841b4050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:26.677 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.690 7fe98c2fd700 1 --2- 172.21.15.102:0/4060263383 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe9841adb00 0x7fe9841b4050 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 rx=0x7fe9840d94f0 tx=0x7fe98000bf70).stop 2024-04-17T22:29:26.677 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.690 7fe98c2fd700 1 -- 172.21.15.102:0/4060263383 shutdown_connections 2024-04-17T22:29:26.677 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.690 7fe98c2fd700 1 --2- 172.21.15.102:0/4060263383 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe9841adf50 0x7fe9841b4680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:26.677 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.690 7fe98c2fd700 1 --2- 172.21.15.102:0/4060263383 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fe9840d3540 0x7fe9841ad4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:26.677 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.690 7fe98c2fd700 1 --2- 172.21.15.102:0/4060263383 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe9841adb00 0x7fe9841b4050 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:26.678 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.690 7fe98c2fd700 1 -- 172.21.15.102:0/4060263383 >> 172.21.15.102:0/4060263383 conn(0x7fe9840e0f40 msgr2=0x7fe9840dfe90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:29:26.678 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.690 7fe98c2fd700 1 -- 172.21.15.102:0/4060263383 shutdown_connections 2024-04-17T22:29:26.678 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.690 7fe98c2fd700 1 -- 172.21.15.102:0/4060263383 wait complete. 2024-04-17T22:29:26.687 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-17T22:29:26.817 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.828 7fdccbfb6700 1 Processor -- start 2024-04-17T22:29:26.817 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.828 7fdccbfb6700 1 -- start start 2024-04-17T22:29:26.817 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.828 7fdccbfb6700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fdcc40f5840 0x7fdcc410b010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:26.817 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.828 7fdccbfb6700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fdcc410b640 0x7fdcc419faf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:26.817 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.828 7fdccbfb6700 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdcc4103b40 con 0x7fdcc410ba90 2024-04-17T22:29:26.817 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.828 7fdccbfb6700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcc410bee0 con 0x7fdcc410b640 2024-04-17T22:29:26.817 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.828 7fdccbfb6700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdcc40fa370 con 0x7fdcc40f5840 2024-04-17T22:29:26.817 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.829 7fdcc8db1700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fdcc40f5840 0x7fdcc410b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:26.817 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.829 7fdcc3fff700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fdcc410b640 0x7fdcc419faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:26.818 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.829 7fdcc8db1700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fdcc40f5840 0x7fdcc410b010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:41596/0 (socket says 172.21.15.102:41596) 2024-04-17T22:29:26.818 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.829 7fdcc8db1700 1 -- 172.21.15.102:0/2875183924 learned_addr learned my addr 172.21.15.102:0/2875183924 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:29:26.818 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.829 7fdcc95b2700 1 --1- 172.21.15.102:0/2875183924 >> v1:172.21.15.102:6789/0 conn(0x7fdcc410ba90 0x7fdcc41a0120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:49200/0 (socket says 172.21.15.102:49200) 2024-04-17T22:29:26.818 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.829 7fdcc37fe700 1 -- 172.21.15.102:0/2875183924 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2379834629 0 0) 0x7fdcbc001490 con 0x7fdcc410ba90 2024-04-17T22:29:26.818 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.829 7fdcc37fe700 1 -- 172.21.15.102:0/2875183924 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdca8003270 con 0x7fdcc410ba90 2024-04-17T22:29:26.818 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.829 7fdcc8db1700 1 -- 172.21.15.102:0/2875183924 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fdcc410b640 msgr2=0x7fdcc419faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:29:26.818 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.829 7fdcc8db1700 1 --2- 172.21.15.102:0/2875183924 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fdcc410b640 0x7fdcc419faf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:26.818 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.829 7fdcc8db1700 1 -- 172.21.15.102:0/2875183924 >> v1:172.21.15.102:6789/0 conn(0x7fdcc410ba90 legacy=0x7fdcc41a0120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:26.818 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.829 7fdcc8db1700 1 -- 172.21.15.102:0/2875183924 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcc41a0920 con 0x7fdcc40f5840 2024-04-17T22:29:26.818 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.829 7fdcc3fff700 1 --2- 172.21.15.102:0/2875183924 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fdcc410b640 0x7fdcc419faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:29:26.818 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.829 7fdcc8db1700 1 --2- 172.21.15.102:0/2875183924 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fdcc40f5840 0x7fdcc410b010 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7fdcb8002f70 tx=0x7fdcb8009a00).ready entity=mon.2 client_cookie=649b2088767ef066 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:29:26.818 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.829 7fdcc37fe700 1 -- 172.21.15.102:0/2875183924 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fdcc41a0920 con 0x7fdcc40f5840 2024-04-17T22:29:26.818 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.829 7fdcc37fe700 1 -- 172.21.15.102:0/2875183924 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcb800f510 con 0x7fdcc40f5840 2024-04-17T22:29:26.818 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.829 7fdcc37fe700 1 -- 172.21.15.102:0/2875183924 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fdcb800f6f0 con 0x7fdcc40f5840 2024-04-17T22:29:26.819 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.829 7fdccbfb6700 1 -- 172.21.15.102:0/2875183924 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fdcc40f5840 msgr2=0x7fdcc410b010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:26.819 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.829 7fdccbfb6700 1 --2- 172.21.15.102:0/2875183924 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fdcc40f5840 0x7fdcc410b010 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7fdcb8002f70 tx=0x7fdcb8009a00).stop 2024-04-17T22:29:26.819 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.830 7fdccbfb6700 1 -- 172.21.15.102:0/2875183924 shutdown_connections 2024-04-17T22:29:26.819 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.830 7fdccbfb6700 1 --2- 172.21.15.102:0/2875183924 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fdcc40f5840 0x7fdcc410b010 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:26.819 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.830 7fdccbfb6700 1 --2- 172.21.15.102:0/2875183924 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fdcc410b640 0x7fdcc419faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:26.819 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.830 7fdccbfb6700 1 -- 172.21.15.102:0/2875183924 >> 172.21.15.102:0/2875183924 conn(0x7fdcc41138a0 msgr2=0x7fdcc40f4f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:29:26.819 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.830 7fdccbfb6700 1 -- 172.21.15.102:0/2875183924 shutdown_connections 2024-04-17T22:29:26.819 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.830 7fdccbfb6700 1 -- 172.21.15.102:0/2875183924 wait complete. 2024-04-17T22:29:26.838 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.852 7fdccbfb6700 1 Processor -- start 2024-04-17T22:29:26.839 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.852 7fdccbfb6700 1 -- start start 2024-04-17T22:29:26.841 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.853 7fdccbfb6700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fdcc40cd3e0 0x7fdcc40cd830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:26.841 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.853 7fdccbfb6700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdcc40cde60 0x7fdcc41b2780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:26.841 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.853 7fdccbfb6700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fdcc41b2db0 0x7fdcc41b3200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:26.841 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.853 7fdccbfb6700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcc4088fc0 con 0x7fdcc40cde60 2024-04-17T22:29:26.842 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.853 7fdccbfb6700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcc40d7250 con 0x7fdcc40cd3e0 2024-04-17T22:29:26.842 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.853 7fdccbfb6700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdcc40d6c00 con 0x7fdcc41b2db0 2024-04-17T22:29:26.842 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.853 7fdcc3fff700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdcc40cde60 0x7fdcc41b2780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:26.842 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.853 7fdcc95b2700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fdcc41b2db0 0x7fdcc41b3200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:26.842 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.853 7fdcc3fff700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdcc40cde60 0x7fdcc41b2780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:37978/0 (socket says 172.21.15.102:37978) 2024-04-17T22:29:26.842 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.853 7fdcc3fff700 1 -- 172.21.15.102:0/1616602688 learned_addr learned my addr 172.21.15.102:0/1616602688 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:29:26.842 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.853 7fdcc8db1700 1 --2- 172.21.15.102:0/1616602688 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fdcc40cd3e0 0x7fdcc40cd830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:26.842 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.853 7fdcc95b2700 1 -- 172.21.15.102:0/1616602688 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fdcc40cd3e0 msgr2=0x7fdcc40cd830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:26.842 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.853 7fdcc95b2700 1 --2- 172.21.15.102:0/1616602688 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fdcc40cd3e0 0x7fdcc40cd830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:26.842 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.853 7fdcc95b2700 1 -- 172.21.15.102:0/1616602688 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdcc40cde60 msgr2=0x7fdcc41b2780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:26.842 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.853 7fdcc95b2700 1 --2- 172.21.15.102:0/1616602688 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdcc40cde60 0x7fdcc41b2780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:26.842 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.853 7fdcc95b2700 1 -- 172.21.15.102:0/1616602688 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcc40d6730 con 0x7fdcc41b2db0 2024-04-17T22:29:26.842 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.853 7fdcc3fff700 1 --2- 172.21.15.102:0/1616602688 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdcc40cde60 0x7fdcc41b2780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:29:26.842 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.853 7fdcc95b2700 1 --2- 172.21.15.102:0/1616602688 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fdcc41b2db0 0x7fdcc41b3200 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7fdcc40d8b70 tx=0x7fdcbc00c930).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:29:26.843 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.853 7fdcc1ffb700 1 -- 172.21.15.102:0/1616602688 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fdcc40d6730 con 0x7fdcc41b2db0 2024-04-17T22:29:26.843 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.853 7fdcc1ffb700 1 -- 172.21.15.102:0/1616602688 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcbc005af0 con 0x7fdcc41b2db0 2024-04-17T22:29:26.843 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.853 7fdccbfb6700 1 -- 172.21.15.102:0/1616602688 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdcc41b9d90 con 0x7fdcc41b2db0 2024-04-17T22:29:26.843 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.853 7fdcc1ffb700 1 -- 172.21.15.102:0/1616602688 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fdcbc0070d0 con 0x7fdcc41b2db0 2024-04-17T22:29:26.843 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.853 7fdccbfb6700 1 -- 172.21.15.102:0/1616602688 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcc41ba170 con 0x7fdcc41b2db0 2024-04-17T22:29:26.843 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.854 7fdcc1ffb700 1 -- 172.21.15.102:0/1616602688 <== mon.2 v2:172.21.15.102:3302/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7fdcbc0072d0 con 0x7fdcc41b2db0 2024-04-17T22:29:26.843 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.854 7fdcc1ffb700 1 -- 172.21.15.102:0/1616602688 <== mon.2 v2:172.21.15.102:3302/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3194+0+0 (secure 0 0 0) 0x7fdcbc0055a0 con 0x7fdcc41b2db0 2024-04-17T22:29:26.843 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.854 7fdccbfb6700 1 -- 172.21.15.102:0/1616602688 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdcc40f0800 con 0x7fdcc41b2db0 2024-04-17T22:29:26.844 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.857 7fdcc1ffb700 1 -- 172.21.15.102:0/1616602688 <== mon.2 v2:172.21.15.102:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+110455 (secure 0 0 0) 0x7fdcbc0058e0 con 0x7fdcc41b2db0 2024-04-17T22:29:26.948 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.961 7fdccbfb6700 1 -- 172.21.15.102:0/1616602688 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fdcc40e21f0 con 0x7fdcc41b2db0 2024-04-17T22:29:26.948 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.962 7fdcc1ffb700 1 -- 172.21.15.102:0/1616602688 <== mon.2 v2:172.21.15.102:3302/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v17) v1 ==== 100+0+11 (secure 0 0 0) 0x7fdcbc000fb0 con 0x7fdcc41b2db0 2024-04-17T22:29:26.949 INFO:teuthology.orchestra.run.smithi102.stdout:pg_num: 64 2024-04-17T22:29:26.950 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.963 7fdccbfb6700 1 -- 172.21.15.102:0/1616602688 >> v1:172.21.15.102:6816/42360 conn(0x7fdcac0145a0 legacy=0x7fdcac016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:26.950 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.963 7fdccbfb6700 1 -- 172.21.15.102:0/1616602688 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fdcc41b2db0 msgr2=0x7fdcc41b3200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:26.950 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.963 7fdccbfb6700 1 --2- 172.21.15.102:0/1616602688 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fdcc41b2db0 0x7fdcc41b3200 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7fdcc40d8b70 tx=0x7fdcbc00c930).stop 2024-04-17T22:29:26.950 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.963 7fdccbfb6700 1 -- 172.21.15.102:0/1616602688 shutdown_connections 2024-04-17T22:29:26.950 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.963 7fdccbfb6700 1 --2- 172.21.15.102:0/1616602688 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fdcc41b2db0 0x7fdcc41b3200 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:26.950 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.963 7fdccbfb6700 1 --2- 172.21.15.102:0/1616602688 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fdcc40cd3e0 0x7fdcc40cd830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:26.950 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.963 7fdccbfb6700 1 --2- 172.21.15.102:0/1616602688 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdcc40cde60 0x7fdcc41b2780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:26.950 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.963 7fdccbfb6700 1 -- 172.21.15.102:0/1616602688 >> 172.21.15.102:0/1616602688 conn(0x7fdcc40e0ed0 msgr2=0x7fdcc40db020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:29:26.950 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.963 7fdccbfb6700 1 -- 172.21.15.102:0/1616602688 shutdown_connections 2024-04-17T22:29:26.951 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:26.963 7fdccbfb6700 1 -- 172.21.15.102:0/1616602688 wait complete. 2024-04-17T22:29:26.961 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-17T22:29:27.087 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.099 7f539fcf7700 1 Processor -- start 2024-04-17T22:29:27.088 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.099 7f539fcf7700 1 -- start start 2024-04-17T22:29:27.088 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.099 7f539fcf7700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f539810c880 0x7f53981a6270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:27.088 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.099 7f539fcf7700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f53981a68a0 0x7f53981a8d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:27.088 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.099 7f539fcf7700 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53981a1b80 con 0x7f53980f56e0 2024-04-17T22:29:27.088 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.099 7f539fcf7700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53980f9350 con 0x7f539810c880 2024-04-17T22:29:27.088 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.099 7f539fcf7700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f539810ccd0 con 0x7f53981a68a0 2024-04-17T22:29:27.088 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.099 7f5397fff700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f539810c880 0x7f53981a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:27.088 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.099 7f539d2f3700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f53981a68a0 0x7f53981a8d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:27.088 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.099 7f539d2f3700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f53981a68a0 0x7f53981a8d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:41612/0 (socket says 172.21.15.102:41612) 2024-04-17T22:29:27.088 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.099 7f539d2f3700 1 -- 172.21.15.102:0/3187060099 learned_addr learned my addr 172.21.15.102:0/3187060099 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:29:27.089 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.099 7f5397fff700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f539810c880 0x7f53981a6270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:56078/0 (socket says 172.21.15.102:56078) 2024-04-17T22:29:27.089 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.099 7f539caf2700 1 --1- 172.21.15.102:0/3187060099 >> v1:172.21.15.102:6789/0 conn(0x7f53980f56e0 0x7f539810c080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:49208/0 (socket says 172.21.15.102:49208) 2024-04-17T22:29:27.089 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.099 7f539d2f3700 1 -- 172.21.15.102:0/3187060099 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f539810c880 msgr2=0x7f53981a6270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:29:27.089 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.099 7f539d2f3700 1 --2- 172.21.15.102:0/3187060099 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f539810c880 0x7f53981a6270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:27.089 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.099 7f539d2f3700 1 -- 172.21.15.102:0/3187060099 >> v1:172.21.15.102:6789/0 conn(0x7f53980f56e0 legacy=0x7f539810c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:27.089 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.099 7f539d2f3700 1 -- 172.21.15.102:0/3187060099 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53981aa340 con 0x7f53981a68a0 2024-04-17T22:29:27.089 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.099 7f53977fe700 1 -- 172.21.15.102:0/3187060099 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1884201237 0 0) 0x7f53981a1b80 con 0x7f53980f56e0 2024-04-17T22:29:27.089 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.099 7f5397fff700 1 --2- 172.21.15.102:0/3187060099 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f539810c880 0x7f53981a6270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:29:27.089 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.100 7f539d2f3700 1 --2- 172.21.15.102:0/3187060099 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f53981a68a0 0x7f53981a8d00 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7f5398089fc0 tx=0x7f539000f330).ready entity=mon.2 client_cookie=b3582d43f6a9a6cf server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:29:27.089 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.100 7f53977fe700 1 -- 172.21.15.102:0/3187060099 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f53981aa340 con 0x7f53981a68a0 2024-04-17T22:29:27.089 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.100 7f53977fe700 1 -- 172.21.15.102:0/3187060099 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f539000fa90 con 0x7f53981a68a0 2024-04-17T22:29:27.089 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.100 7f53977fe700 1 -- 172.21.15.102:0/3187060099 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f5390013070 con 0x7f53981a68a0 2024-04-17T22:29:27.089 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.100 7f539fcf7700 1 -- 172.21.15.102:0/3187060099 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f53981a68a0 msgr2=0x7f53981a8d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:27.089 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.100 7f539fcf7700 1 --2- 172.21.15.102:0/3187060099 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f53981a68a0 0x7f53981a8d00 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7f5398089fc0 tx=0x7f539000f330).stop 2024-04-17T22:29:27.090 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.100 7f539fcf7700 1 -- 172.21.15.102:0/3187060099 shutdown_connections 2024-04-17T22:29:27.090 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.100 7f539fcf7700 1 --2- 172.21.15.102:0/3187060099 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f53981a68a0 0x7f53981a8d00 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:27.090 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.100 7f539fcf7700 1 --2- 172.21.15.102:0/3187060099 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f539810c880 0x7f53981a6270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:27.090 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.100 7f539fcf7700 1 -- 172.21.15.102:0/3187060099 >> 172.21.15.102:0/3187060099 conn(0x7f5398113830 msgr2=0x7f53980f4ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:29:27.090 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.100 7f539fcf7700 1 -- 172.21.15.102:0/3187060099 shutdown_connections 2024-04-17T22:29:27.090 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.100 7f539fcf7700 1 -- 172.21.15.102:0/3187060099 wait complete. 2024-04-17T22:29:27.109 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.123 7f539fcf7700 1 Processor -- start 2024-04-17T22:29:27.110 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.123 7f539fcf7700 1 -- start start 2024-04-17T22:29:27.112 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.123 7f539fcf7700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f53981a9030 0x7f53981a9480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:27.112 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.124 7f539fcf7700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f53981a9ab0 0x7f53981af6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:27.112 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.124 7f539fcf7700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f53981afce0 0x7f53981b0130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:27.113 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.124 7f539fcf7700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53980c1d60 con 0x7f53981a9ab0 2024-04-17T22:29:27.113 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.124 7f539fcf7700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53980d76b0 con 0x7f53981afce0 2024-04-17T22:29:27.113 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.124 7f539fcf7700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f53980d7c70 con 0x7f53981a9030 2024-04-17T22:29:27.113 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.124 7f539caf2700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f53981a9030 0x7f53981a9480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:27.113 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.124 7f539d2f3700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f53981afce0 0x7f53981b0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:27.113 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.124 7f5397fff700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f53981a9ab0 0x7f53981af6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:27.113 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.124 7f539caf2700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f53981a9030 0x7f53981a9480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:41628/0 (socket says 172.21.15.102:41628) 2024-04-17T22:29:27.113 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.124 7f539caf2700 1 -- 172.21.15.102:0/1992197960 learned_addr learned my addr 172.21.15.102:0/1992197960 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:29:27.113 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.124 7f5397fff700 1 -- 172.21.15.102:0/1992197960 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f53981a9030 msgr2=0x7f53981a9480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:27.113 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.124 7f5397fff700 1 --2- 172.21.15.102:0/1992197960 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f53981a9030 0x7f53981a9480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:27.113 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.124 7f5397fff700 1 -- 172.21.15.102:0/1992197960 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f53981afce0 msgr2=0x7f53981b0130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:27.113 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.124 7f5397fff700 1 --2- 172.21.15.102:0/1992197960 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f53981afce0 0x7f53981b0130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:27.113 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.124 7f5397fff700 1 -- 172.21.15.102:0/1992197960 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53980d79b0 con 0x7f53981a9ab0 2024-04-17T22:29:27.113 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.124 7f539d2f3700 1 --2- 172.21.15.102:0/1992197960 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f53981afce0 0x7f53981b0130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:29:27.114 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.124 7f539caf2700 1 --2- 172.21.15.102:0/1992197960 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f53981a9030 0x7f53981a9480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:29:27.114 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.124 7f5397fff700 1 --2- 172.21.15.102:0/1992197960 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f53981a9ab0 0x7f53981af6b0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 rx=0x7f53980d7860 tx=0x7f538400ec70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:29:27.114 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.124 7f5395ffb700 1 -- 172.21.15.102:0/1992197960 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f53980d79b0 con 0x7f53981a9ab0 2024-04-17T22:29:27.114 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.124 7f5395ffb700 1 -- 172.21.15.102:0/1992197960 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f538400c790 con 0x7f53981a9ab0 2024-04-17T22:29:27.114 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.124 7f539fcf7700 1 -- 172.21.15.102:0/1992197960 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53981b6e30 con 0x7f53981a9ab0 2024-04-17T22:29:27.114 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.124 7f5395ffb700 1 -- 172.21.15.102:0/1992197960 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f538400f040 con 0x7f53981a9ab0 2024-04-17T22:29:27.114 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.124 7f539fcf7700 1 -- 172.21.15.102:0/1992197960 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53981b7210 con 0x7f53981a9ab0 2024-04-17T22:29:27.114 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.125 7f5395ffb700 1 -- 172.21.15.102:0/1992197960 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7f5384011070 con 0x7f53981a9ab0 2024-04-17T22:29:27.114 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.125 7f5395ffb700 1 -- 172.21.15.102:0/1992197960 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3194+0+0 (secure 0 0 0) 0x7f5384010040 con 0x7f53981a9ab0 2024-04-17T22:29:27.114 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.125 7f539fcf7700 1 -- 172.21.15.102:0/1992197960 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53980f0790 con 0x7f53981a9ab0 2024-04-17T22:29:27.115 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.128 7f5395ffb700 1 -- 172.21.15.102:0/1992197960 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+110455 (secure 0 0 0) 0x7f538400edb0 con 0x7f53981a9ab0 2024-04-17T22:29:27.261 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.274 7f539fcf7700 1 -- 172.21.15.102:0/1992197960 >> v1:172.21.15.102:6816/42360 conn(0x7f53800145a0 legacy=0x7f5380016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:27.261 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.274 7f539fcf7700 1 -- 172.21.15.102:0/1992197960 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f53981a9ab0 msgr2=0x7f53981af6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:27.262 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.274 7f539fcf7700 1 --2- 172.21.15.102:0/1992197960 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f53981a9ab0 0x7f53981af6b0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 rx=0x7f53980d7860 tx=0x7f538400ec70).stop 2024-04-17T22:29:27.262 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.274 7f539fcf7700 1 -- 172.21.15.102:0/1992197960 shutdown_connections 2024-04-17T22:29:27.262 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.274 7f539fcf7700 1 --2- 172.21.15.102:0/1992197960 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f53981afce0 0x7f53981b0130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:27.262 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.274 7f539fcf7700 1 --2- 172.21.15.102:0/1992197960 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f53981a9030 0x7f53981a9480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:27.262 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.274 7f539fcf7700 1 --2- 172.21.15.102:0/1992197960 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f53981a9ab0 0x7f53981af6b0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:27.262 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.274 7f539fcf7700 1 -- 172.21.15.102:0/1992197960 >> 172.21.15.102:0/1992197960 conn(0x7f53980df450 msgr2=0x7f53980c4700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:29:27.262 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.274 7f539fcf7700 1 -- 172.21.15.102:0/1992197960 shutdown_connections 2024-04-17T22:29:27.262 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.274 7f539fcf7700 1 -- 172.21.15.102:0/1992197960 wait complete. 2024-04-17T22:29:27.262 INFO:teuthology.orchestra.run.smithi102.stderr:no valid command found; 10 closest matches: 2024-04-17T22:29:27.263 INFO:teuthology.orchestra.run.smithi102.stderr:osd perf 2024-04-17T22:29:27.263 INFO:teuthology.orchestra.run.smithi102.stderr:osd df {plain|tree} {class|name} {} 2024-04-17T22:29:27.263 INFO:teuthology.orchestra.run.smithi102.stderr:osd blocked-by 2024-04-17T22:29:27.263 INFO:teuthology.orchestra.run.smithi102.stderr:osd pool stats {} 2024-04-17T22:29:27.263 INFO:teuthology.orchestra.run.smithi102.stderr:osd pool scrub [...] 2024-04-17T22:29:27.263 INFO:teuthology.orchestra.run.smithi102.stderr:osd pool deep-scrub [...] 2024-04-17T22:29:27.263 INFO:teuthology.orchestra.run.smithi102.stderr:osd pool repair [...] 2024-04-17T22:29:27.263 INFO:teuthology.orchestra.run.smithi102.stderr:osd pool force-recovery [...] 2024-04-17T22:29:27.263 INFO:teuthology.orchestra.run.smithi102.stderr:osd pool force-backfill [...] 2024-04-17T22:29:27.263 INFO:teuthology.orchestra.run.smithi102.stderr:osd pool cancel-force-recovery [...] 2024-04-17T22:29:27.263 INFO:teuthology.orchestra.run.smithi102.stderr:Error EINVAL: invalid command 2024-04-17T22:29:27.271 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-17T22:29:27.272 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blacklist ls 2024-04-17T22:29:27.399 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.411 7f0673c42700 1 Processor -- start 2024-04-17T22:29:27.399 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.411 7f0673c42700 1 -- start start 2024-04-17T22:29:27.400 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.411 7f0673c42700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f066c1a64b0 0x7f066c1a8910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:27.400 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.411 7f0673c42700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f066c1a8f40 0x7f066c1ab3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:27.400 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.411 7f0673c42700 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f066c10ee90 con 0x7f066c10e240 2024-04-17T22:29:27.400 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.411 7f0673c42700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f066c0f9f10 con 0x7f066c1a64b0 2024-04-17T22:29:27.400 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.411 7f0673c42700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f066c0f9030 con 0x7f066c1a8f40 2024-04-17T22:29:27.400 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.411 7f066bfff700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f066c1a64b0 0x7f066c1a8910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:27.400 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.411 7f067123e700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f066c1a8f40 0x7f066c1ab3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:27.400 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.411 7f0670a3d700 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f066c10e240 0x7f066c10e690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:49216/0 (socket says 172.21.15.102:49216) 2024-04-17T22:29:27.400 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.411 7f0670a3d700 1 -- 172.21.15.102:0/260020794 learned_addr learned my addr 172.21.15.102:0/260020794 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-17T22:29:27.400 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.411 7f066bfff700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f066c1a64b0 0x7f066c1a8910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:56098/0 (socket says 172.21.15.102:56098) 2024-04-17T22:29:27.400 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.412 7f067123e700 1 -- 172.21.15.102:0/260020794 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f066c1a64b0 msgr2=0x7f066c1a8910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:29:27.400 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.412 7f066b7fe700 1 -- 172.21.15.102:0/260020794 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2624503853 0 0) 0x7f0660001110 con 0x7f066c10e240 2024-04-17T22:29:27.400 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.412 7f067123e700 1 --2- 172.21.15.102:0/260020794 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f066c1a64b0 0x7f066c1a8910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:27.400 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.412 7f067123e700 1 -- 172.21.15.102:0/260020794 >> v1:172.21.15.102:6789/0 conn(0x7f066c10e240 legacy=0x7f066c10e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:27.401 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.412 7f067123e700 1 -- 172.21.15.102:0/260020794 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f066c1ac9e0 con 0x7f066c1a8f40 2024-04-17T22:29:27.401 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.412 7f066bfff700 1 --2- 172.21.15.102:0/260020794 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f066c1a64b0 0x7f066c1a8910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:29:27.401 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.412 7f067123e700 1 --2- 172.21.15.102:0/260020794 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f066c1a8f40 0x7f066c1ab3a0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7f066c0f95d0 tx=0x7f0664009cf0).ready entity=mon.2 client_cookie=11311c2c999d3da6 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:29:27.401 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.412 7f066b7fe700 1 -- 172.21.15.102:0/260020794 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f066c1ac9e0 con 0x7f066c1a8f40 2024-04-17T22:29:27.401 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.412 7f066b7fe700 1 -- 172.21.15.102:0/260020794 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f066400b840 con 0x7f066c1a8f40 2024-04-17T22:29:27.401 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.412 7f066b7fe700 1 -- 172.21.15.102:0/260020794 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f066400ba20 con 0x7f066c1a8f40 2024-04-17T22:29:27.401 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.412 7f0673c42700 1 -- 172.21.15.102:0/260020794 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f066c1a8f40 msgr2=0x7f066c1ab3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:27.401 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.412 7f0673c42700 1 --2- 172.21.15.102:0/260020794 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f066c1a8f40 0x7f066c1ab3a0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7f066c0f95d0 tx=0x7f0664009cf0).stop 2024-04-17T22:29:27.401 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.413 7f0673c42700 1 -- 172.21.15.102:0/260020794 shutdown_connections 2024-04-17T22:29:27.401 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.413 7f0673c42700 1 --2- 172.21.15.102:0/260020794 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f066c1a8f40 0x7f066c1ab3a0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:27.401 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.413 7f0673c42700 1 --2- 172.21.15.102:0/260020794 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f066c1a64b0 0x7f066c1a8910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:27.401 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.413 7f0673c42700 1 -- 172.21.15.102:0/260020794 >> 172.21.15.102:0/260020794 conn(0x7f066c1138a0 msgr2=0x7f066c0f4f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:29:27.401 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.413 7f0673c42700 1 -- 172.21.15.102:0/260020794 shutdown_connections 2024-04-17T22:29:27.401 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.413 7f0673c42700 1 -- 172.21.15.102:0/260020794 wait complete. 2024-04-17T22:29:27.422 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.435 7f0673c42700 1 Processor -- start 2024-04-17T22:29:27.422 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.435 7f0673c42700 1 -- start start 2024-04-17T22:29:27.424 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.436 7f0673c42700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f066c0cdad0 0x7f066c1ad900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:27.424 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.436 7f0673c42700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f066c1adf30 0x7f066c1b3d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:27.424 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.436 7f0673c42700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f066c1b43b0 0x7f066c1b4800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:27.424 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.436 7f0673c42700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f066c0a8560 con 0x7f066c0cdad0 2024-04-17T22:29:27.424 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.436 7f0673c42700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f066c0fa1b0 con 0x7f066c1b43b0 2024-04-17T22:29:27.424 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.436 7f0673c42700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f066c0d6a70 con 0x7f066c1adf30 2024-04-17T22:29:27.424 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.436 7f066bfff700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f066c1adf30 0x7f066c1b3d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:27.424 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.436 7f067123e700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f066c1b43b0 0x7f066c1b4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:27.425 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.436 7f0670a3d700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f066c0cdad0 0x7f066c1ad900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:27.425 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.436 7f066bfff700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f066c1adf30 0x7f066c1b3d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:41652/0 (socket says 172.21.15.102:41652) 2024-04-17T22:29:27.425 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.436 7f066bfff700 1 -- 172.21.15.102:0/3207693169 learned_addr learned my addr 172.21.15.102:0/3207693169 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:29:27.425 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.436 7f067123e700 1 -- 172.21.15.102:0/3207693169 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f066c1adf30 msgr2=0x7f066c1b3d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:27.425 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.436 7f067123e700 1 --2- 172.21.15.102:0/3207693169 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f066c1adf30 0x7f066c1b3d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:27.425 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.436 7f067123e700 1 -- 172.21.15.102:0/3207693169 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f066c0cdad0 msgr2=0x7f066c1ad900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:27.425 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.436 7f067123e700 1 --2- 172.21.15.102:0/3207693169 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f066c0cdad0 0x7f066c1ad900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:27.425 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.436 7f067123e700 1 -- 172.21.15.102:0/3207693169 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f066c1bb260 con 0x7f066c1b43b0 2024-04-17T22:29:27.425 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.436 7f066bfff700 1 --2- 172.21.15.102:0/3207693169 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f066c1adf30 0x7f066c1b3d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:29:27.425 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.436 7f067123e700 1 --2- 172.21.15.102:0/3207693169 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f066c1b43b0 0x7f066c1b4800 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0x7f066c0d7120 tx=0x7f0664006c40).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:29:27.425 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.437 7f0669ffb700 1 -- 172.21.15.102:0/3207693169 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f066c1bb260 con 0x7f066c1b43b0 2024-04-17T22:29:27.425 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.437 7f0669ffb700 1 -- 172.21.15.102:0/3207693169 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0664011b20 con 0x7f066c1b43b0 2024-04-17T22:29:27.425 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.437 7f0673c42700 1 -- 172.21.15.102:0/3207693169 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f066c1bb460 con 0x7f066c1b43b0 2024-04-17T22:29:27.425 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.437 7f0669ffb700 1 -- 172.21.15.102:0/3207693169 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f0664017040 con 0x7f066c1b43b0 2024-04-17T22:29:27.425 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.437 7f0673c42700 1 -- 172.21.15.102:0/3207693169 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f066c1bb660 con 0x7f066c1b43b0 2024-04-17T22:29:27.426 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.437 7f0669ffb700 1 -- 172.21.15.102:0/3207693169 <== mon.1 v2:172.21.15.102:3301/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7f066401c470 con 0x7f066c1b43b0 2024-04-17T22:29:27.426 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.437 7f0669ffb700 1 -- 172.21.15.102:0/3207693169 <== mon.1 v2:172.21.15.102:3301/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3194+0+0 (secure 0 0 0) 0x7f0664018030 con 0x7f066c1b43b0 2024-04-17T22:29:27.426 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.438 7f0673c42700 1 -- 172.21.15.102:0/3207693169 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f066c0f0800 con 0x7f066c1b43b0 2024-04-17T22:29:27.427 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.441 7f0669ffb700 1 -- 172.21.15.102:0/3207693169 <== mon.1 v2:172.21.15.102:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+110455 (secure 0 0 0) 0x7f0664011910 con 0x7f066c1b43b0 2024-04-17T22:29:27.546 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.559 7f0673c42700 1 -- 172.21.15.102:0/3207693169 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "osd blacklist ls"} v 0) v1 -- 0x7f066c0a6510 con 0x7f066c1b43b0 2024-04-17T22:29:27.546 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.559 7f0669ffb700 1 -- 172.21.15.102:0/3207693169 <== mon.1 v2:172.21.15.102:3301/0 7 ==== mon_command_ack([{"prefix": "osd blacklist ls"}]=0 listed 0 entries v17) v1 ==== 80+0+0 (secure 0 0 0) 0x7f066400b630 con 0x7f066c1b43b0 2024-04-17T22:29:27.547 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.561 7f0673c42700 1 -- 172.21.15.102:0/3207693169 >> v1:172.21.15.102:6816/42360 conn(0x7f06540145a0 legacy=0x7f0654016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:27.548 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.561 7f0673c42700 1 -- 172.21.15.102:0/3207693169 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f066c1b43b0 msgr2=0x7f066c1b4800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:27.548 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.561 7f0673c42700 1 --2- 172.21.15.102:0/3207693169 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f066c1b43b0 0x7f066c1b4800 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0x7f066c0d7120 tx=0x7f0664006c40).stop 2024-04-17T22:29:27.548 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.561 7f0673c42700 1 -- 172.21.15.102:0/3207693169 shutdown_connections 2024-04-17T22:29:27.548 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.561 7f0673c42700 1 --2- 172.21.15.102:0/3207693169 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f066c1b43b0 0x7f066c1b4800 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:27.548 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.561 7f0673c42700 1 --2- 172.21.15.102:0/3207693169 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f066c0cdad0 0x7f066c1ad900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:27.548 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.561 7f0673c42700 1 --2- 172.21.15.102:0/3207693169 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f066c1adf30 0x7f066c1b3d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:27.548 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.561 7f0673c42700 1 -- 172.21.15.102:0/3207693169 >> 172.21.15.102:0/3207693169 conn(0x7f066c0e0f20 msgr2=0x7f066c0dac40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:29:27.548 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.561 7f0673c42700 1 -- 172.21.15.102:0/3207693169 shutdown_connections 2024-04-17T22:29:27.548 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.561 7f0673c42700 1 -- 172.21.15.102:0/3207693169 wait complete. 2024-04-17T22:29:27.548 INFO:teuthology.orchestra.run.smithi102.stderr:listed 0 entries 2024-04-17T22:29:27.558 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-04-17T22:29:27.558 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T22:29:27.581 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T22:29:27.707 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.719 7fc4951d8700 1 Processor -- start 2024-04-17T22:29:27.708 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.719 7fc4951d8700 1 -- start start 2024-04-17T22:29:27.708 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.719 7fc4951d8700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc4900f57d0 0x7fc49010afa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:27.708 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.719 7fc4951d8700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fc49010b5d0 0x7fc49019fa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:27.708 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.719 7fc4951d8700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fc49010ba20 0x7fc4901a00b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:27.708 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.719 7fc4951d8700 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc49010be70 con 0x7fc49010ba20 2024-04-17T22:29:27.708 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.719 7fc4951d8700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4900fa300 con 0x7fc4900f57d0 2024-04-17T22:29:27.708 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.719 7fc4951d8700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc490103ad0 con 0x7fc49010b5d0 2024-04-17T22:29:27.708 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.719 7fc48ddfc700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc4900f57d0 0x7fc49010afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:27.708 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.720 7fc48e5fd700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fc49010ba20 0x7fc4901a00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:27.708 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.720 7fc48d5fb700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fc49010b5d0 0x7fc49019fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:27.708 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.720 7fc48ddfc700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc4900f57d0 0x7fc49010afa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:56106/0 (socket says 172.21.15.102:56106) 2024-04-17T22:29:27.708 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.720 7fc48ddfc700 1 -- 172.21.15.102:0/3213402696 learned_addr learned my addr 172.21.15.102:0/3213402696 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:29:27.709 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.720 7fc48e5fd700 1 -- 172.21.15.102:0/3213402696 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fc49010b5d0 msgr2=0x7fc49019fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:29:27.709 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.720 7fc48e5fd700 1 --2- 172.21.15.102:0/3213402696 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fc49010b5d0 0x7fc49019fa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:27.709 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.720 7fc48e5fd700 1 -- 172.21.15.102:0/3213402696 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc4900f57d0 msgr2=0x7fc49010afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:29:27.709 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.720 7fc48e5fd700 1 --2- 172.21.15.102:0/3213402696 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc4900f57d0 0x7fc49010afa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:27.709 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.720 7fc48e5fd700 1 -- 172.21.15.102:0/3213402696 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4901a06e0 con 0x7fc49010ba20 2024-04-17T22:29:27.709 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.720 7fc48ddfc700 1 --2- 172.21.15.102:0/3213402696 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc4900f57d0 0x7fc49010afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:29:27.709 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.720 7fc48e5fd700 1 --2- 172.21.15.102:0/3213402696 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc49010ba20 0x7fc4901a00b0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 rx=0x7fc4900fa4b0 tx=0x7fc48000d2f0).ready entity=mon.0 client_cookie=14e4a6fc0b8b8fed server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:29:27.709 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.720 7fc48cdfa700 1 -- 172.21.15.102:0/3213402696 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc4901a06e0 con 0x7fc49010ba20 2024-04-17T22:29:27.709 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.720 7fc48cdfa700 1 -- 172.21.15.102:0/3213402696 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc480010470 con 0x7fc49010ba20 2024-04-17T22:29:27.709 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.720 7fc48cdfa700 1 -- 172.21.15.102:0/3213402696 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc480010840 con 0x7fc49010ba20 2024-04-17T22:29:27.709 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.720 7fc4951d8700 1 -- 172.21.15.102:0/3213402696 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc49010ba20 msgr2=0x7fc4901a00b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:27.709 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.720 7fc4951d8700 1 --2- 172.21.15.102:0/3213402696 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc49010ba20 0x7fc4901a00b0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 rx=0x7fc4900fa4b0 tx=0x7fc48000d2f0).stop 2024-04-17T22:29:27.709 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.721 7fc4951d8700 1 -- 172.21.15.102:0/3213402696 shutdown_connections 2024-04-17T22:29:27.709 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.721 7fc4951d8700 1 --2- 172.21.15.102:0/3213402696 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc49010ba20 0x7fc4901a00b0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:27.709 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.721 7fc4951d8700 1 --2- 172.21.15.102:0/3213402696 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc4900f57d0 0x7fc49010afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:27.709 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.721 7fc4951d8700 1 --2- 172.21.15.102:0/3213402696 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fc49010b5d0 0x7fc49019fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:27.710 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.721 7fc4951d8700 1 -- 172.21.15.102:0/3213402696 >> 172.21.15.102:0/3213402696 conn(0x7fc490113830 msgr2=0x7fc4900f4ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:29:27.710 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.721 7fc4951d8700 1 -- 172.21.15.102:0/3213402696 shutdown_connections 2024-04-17T22:29:27.710 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.721 7fc4951d8700 1 -- 172.21.15.102:0/3213402696 wait complete. 2024-04-17T22:29:27.732 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.743 7fc4951d8700 1 Processor -- start 2024-04-17T22:29:27.732 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.743 7fc4951d8700 1 -- start start 2024-04-17T22:29:27.732 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.744 7fc4951d8700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fc4900d5240 0x7fc4901ad1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:27.732 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.744 7fc4951d8700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc4901ad810 0x7fc4901b3be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:27.732 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.744 7fc4951d8700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc4901adc60 0x7fc4901b4210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:27.732 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.744 7fc4951d8700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4900e2d70 con 0x7fc4901adc60 2024-04-17T22:29:27.732 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.744 7fc4951d8700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4900dcb40 con 0x7fc4901ad810 2024-04-17T22:29:27.732 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.744 7fc4951d8700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc4900d4500 con 0x7fc4900d5240 2024-04-17T22:29:27.732 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.744 7fc48ddfc700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fc4900d5240 0x7fc4901ad1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:27.732 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.744 7fc48e5fd700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc4901adc60 0x7fc4901b4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:27.732 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.744 7fc48d5fb700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc4901ad810 0x7fc4901b3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:27.732 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.744 7fc48ddfc700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fc4900d5240 0x7fc4901ad1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:41660/0 (socket says 172.21.15.102:41660) 2024-04-17T22:29:27.732 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.744 7fc48e5fd700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc4901adc60 0x7fc4901b4210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:37994/0 (socket says 172.21.15.102:37994) 2024-04-17T22:29:27.733 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.744 7fc48ddfc700 1 -- 172.21.15.102:0/2403868666 learned_addr learned my addr 172.21.15.102:0/2403868666 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:29:27.733 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.744 7fc48d5fb700 1 -- 172.21.15.102:0/2403868666 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fc4900d5240 msgr2=0x7fc4901ad1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:27.733 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.744 7fc48d5fb700 1 --2- 172.21.15.102:0/2403868666 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fc4900d5240 0x7fc4901ad1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:27.733 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.744 7fc48d5fb700 1 -- 172.21.15.102:0/2403868666 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc4901adc60 msgr2=0x7fc4901b4210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:27.733 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.744 7fc48d5fb700 1 --2- 172.21.15.102:0/2403868666 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc4901adc60 0x7fc4901b4210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:27.733 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.744 7fc48d5fb700 1 -- 172.21.15.102:0/2403868666 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4901b4840 con 0x7fc4901ad810 2024-04-17T22:29:27.733 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.744 7fc48ddfc700 1 --2- 172.21.15.102:0/2403868666 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fc4900d5240 0x7fc4901ad1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:29:27.733 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.744 7fc48d5fb700 1 --2- 172.21.15.102:0/2403868666 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc4901ad810 0x7fc4901b3be0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7fc4900c3870 tx=0x7fc48400ebf0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:29:27.733 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.744 7fc47effd700 1 -- 172.21.15.102:0/2403868666 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc4901b4840 con 0x7fc4901ad810 2024-04-17T22:29:27.733 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.745 7fc47effd700 1 -- 172.21.15.102:0/2403868666 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc48400ee60 con 0x7fc4901ad810 2024-04-17T22:29:27.733 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.745 7fc47effd700 1 -- 172.21.15.102:0/2403868666 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc48400f040 con 0x7fc4901ad810 2024-04-17T22:29:27.733 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.745 7fc4951d8700 1 -- 172.21.15.102:0/2403868666 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4901b4b60 con 0x7fc4901ad810 2024-04-17T22:29:27.733 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.745 7fc4951d8700 1 -- 172.21.15.102:0/2403868666 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4901bb530 con 0x7fc4901ad810 2024-04-17T22:29:27.733 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.745 7fc47effd700 1 -- 172.21.15.102:0/2403868666 <== mon.1 v2:172.21.15.102:3301/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7fc484011070 con 0x7fc4901ad810 2024-04-17T22:29:27.733 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.745 7fc47effd700 1 -- 172.21.15.102:0/2403868666 <== mon.1 v2:172.21.15.102:3301/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3194+0+0 (secure 0 0 0) 0x7fc484010040 con 0x7fc4901ad810 2024-04-17T22:29:27.734 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.746 7fc4951d8700 1 -- 172.21.15.102:0/2403868666 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4900f0790 con 0x7fc4901ad810 2024-04-17T22:29:27.735 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.749 7fc47effd700 1 -- 172.21.15.102:0/2403868666 <== mon.1 v2:172.21.15.102:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+110455 (secure 0 0 0) 0x7fc48400c5a0 con 0x7fc4901ad810 2024-04-17T22:29:27.842 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.855 7fc4951d8700 1 -- 172.21.15.102:0/2403868666 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc4900abea0 con 0x7fc4901ad810 2024-04-17T22:29:27.842 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.856 7fc47effd700 1 -- 172.21.15.102:0/2403868666 <== mon.1 v2:172.21.15.102:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) v1 ==== 74+0+8764 (secure 0 0 0) 0x7fc484000c30 con 0x7fc4901ad810 2024-04-17T22:29:27.842 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-17T22:29:27.843 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":17,"fsid":"cb63e789-091d-49b1-b120-55f9db526871","created":"2024-04-17 22:28:48.209635","modified":"2024-04-17 22:29:23.257387","last_up_change":"2024-04-17 22:28:54.236215","last_in_change":"2024-04-17 22:28:54.236215","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":4,"require_min_compat_client":"nautilus","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-04-17 22:28:57.478545","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-17 22:29:06.144708","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-17 22:29:06.748924","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3351ef85-ade4-4b90-a8f4-213f4e00ba01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6812","nonce":42955}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6813","nonce":42955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6815","nonce":42955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6814","nonce":42955}]},"public_addr":"172.21.15.102:6812/42955","cluster_addr":"172.21.15.102:6813/42955","heartbeat_back_addr":"172.21.15.102:6815/42955","heartbeat_front_addr":"172.21.15.102:6814/42955","state":["exists","up"]},{"osd":1,"uuid":"04f67eac-f36b-48f5-982d-4cfc776596d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6808","nonce":42958}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6809","nonce":42958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6811","nonce":42958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6810","nonce":42958}]},"public_addr":"172.21.15.102:6808/42958","cluster_addr":"172.21.15.102:6809/42958","heartbeat_back_addr":"172.21.15.102:6811/42958","heartbeat_front_addr":"172.21.15.102:6810/42958","state":["exists","up"]},{"osd":2,"uuid":"d524204b-70b2-40a8-8f73-f9e7d44318af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6804","nonce":42954}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6805","nonce":42954}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6807","nonce":42954}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6806","nonce":42954}]},"public_addr":"172.21.15.102:6804/42954","cluster_addr":"172.21.15.102:6805/42954","heartbeat_back_addr":"172.21.15.102:6807/42954","heartbeat_front_addr":"172.21.15.102:6806/42954","state":["exists","up"]},{"osd":3,"uuid":"4fc9d20c-1ad0-484f-aafa-2c9a7574245b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6800","nonce":42953}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6801","nonce":42953}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6803","nonce":42953}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6802","nonce":42953}]},"public_addr":"172.21.15.102:6800/42953","cluster_addr":"172.21.15.102:6801/42953","heartbeat_back_addr":"172.21.15.102:6803/42953","heartbeat_front_addr":"172.21.15.102:6802/42953","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2024-04-17T22:29:27.844 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.857 7fc4951d8700 1 -- 172.21.15.102:0/2403868666 >> v1:172.21.15.102:6816/42360 conn(0x7fc46c0145a0 legacy=0x7fc46c016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:27.844 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.857 7fc4951d8700 1 -- 172.21.15.102:0/2403868666 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc4901ad810 msgr2=0x7fc4901b3be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:27.844 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.857 7fc4951d8700 1 --2- 172.21.15.102:0/2403868666 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc4901ad810 0x7fc4901b3be0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7fc4900c3870 tx=0x7fc48400ebf0).stop 2024-04-17T22:29:27.844 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.857 7fc4951d8700 1 -- 172.21.15.102:0/2403868666 shutdown_connections 2024-04-17T22:29:27.844 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.857 7fc4951d8700 1 --2- 172.21.15.102:0/2403868666 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc4901adc60 0x7fc4901b4210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:27.844 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.857 7fc4951d8700 1 --2- 172.21.15.102:0/2403868666 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fc4900d5240 0x7fc4901ad1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:27.844 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.857 7fc4951d8700 1 --2- 172.21.15.102:0/2403868666 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc4901ad810 0x7fc4901b3be0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:27.844 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.857 7fc4951d8700 1 -- 172.21.15.102:0/2403868666 >> 172.21.15.102:0/2403868666 conn(0x7fc4900c3fb0 msgr2=0x7fc4900c46d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:29:27.844 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.857 7fc4951d8700 1 -- 172.21.15.102:0/2403868666 shutdown_connections 2024-04-17T22:29:27.844 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.857 7fc4951d8700 1 -- 172.21.15.102:0/2403868666 wait complete. 2024-04-17T22:29:27.854 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2024-04-17 22:28:57.478545', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '12', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-17 22:29:06.144708', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-17 22:29:06.748924', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-17T22:29:27.855 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2024-04-17T22:29:27.984 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.995 7f3f722c0700 1 Processor -- start 2024-04-17T22:29:27.984 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.995 7f3f722c0700 1 -- start start 2024-04-17T22:29:27.984 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.996 7f3f722c0700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f3f6c1055c0 0x7f3f6c10b130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:27.984 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.996 7f3f722c0700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f3f6c0f59b0 0x7f3f6c10b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:27.984 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.996 7f3f722c0700 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f6c0ad030 con 0x7f3f6c19fc80 2024-04-17T22:29:27.984 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.996 7f3f722c0700 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f6c10bd90 con 0x7f3f6c1055c0 2024-04-17T22:29:27.985 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.996 7f3f722c0700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3f6c1a08d0 con 0x7f3f6c0f59b0 2024-04-17T22:29:27.985 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.996 7f3f6a885700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f3f6c0f59b0 0x7f3f6c10b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:27.985 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.996 7f3f6b086700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f3f6c1055c0 0x7f3f6c10b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:27.985 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.996 7f3f6b086700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f3f6c1055c0 0x7f3f6c10b130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:38002/0 (socket says 172.21.15.102:38002) 2024-04-17T22:29:27.985 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.996 7f3f6a885700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f3f6c0f59b0 0x7f3f6c10b760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:41666/0 (socket says 172.21.15.102:41666) 2024-04-17T22:29:27.985 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.996 7f3f6b086700 1 -- 172.21.15.102:0/2833944144 learned_addr learned my addr 172.21.15.102:0/2833944144 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:29:27.985 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.996 7f3f6b086700 1 -- 172.21.15.102:0/2833944144 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f3f6c0f59b0 msgr2=0x7f3f6c10b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:29:27.985 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.996 7f3f6b086700 1 --2- 172.21.15.102:0/2833944144 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f3f6c0f59b0 0x7f3f6c10b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:27.985 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.996 7f3f6b086700 1 -- 172.21.15.102:0/2833944144 >> v1:172.21.15.102:6789/0 conn(0x7f3f6c19fc80 legacy=0x7f3f6c1a00d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:27.985 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.996 7f3f6b086700 1 -- 172.21.15.102:0/2833944144 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f6c1a0b00 con 0x7f3f6c1055c0 2024-04-17T22:29:27.985 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.996 7f3f6a084700 1 -- 172.21.15.102:0/2833944144 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2186244036 0 0) 0x7f3f64001490 con 0x7f3f6c19fc80 2024-04-17T22:29:27.985 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.996 7f3f6a885700 1 --2- 172.21.15.102:0/2833944144 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f3f6c0f59b0 0x7f3f6c10b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:29:27.985 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.996 7f3f6b086700 1 --2- 172.21.15.102:0/2833944144 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3f6c1055c0 0x7f3f6c10b130 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 rx=0x7f3f60002f70 tx=0x7f3f60009a50).ready entity=mon.0 client_cookie=e2a74a671f71f166 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:29:27.985 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.997 7f3f6a084700 1 -- 172.21.15.102:0/2833944144 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f3f6c1a0b00 con 0x7f3f6c1055c0 2024-04-17T22:29:27.985 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.997 7f3f6a084700 1 -- 172.21.15.102:0/2833944144 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f6000f510 con 0x7f3f6c1055c0 2024-04-17T22:29:27.986 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.997 7f3f6a084700 1 -- 172.21.15.102:0/2833944144 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f3f6000f6f0 con 0x7f3f6c1055c0 2024-04-17T22:29:27.986 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.997 7f3f722c0700 1 -- 172.21.15.102:0/2833944144 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3f6c1055c0 msgr2=0x7f3f6c10b130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:27.986 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.997 7f3f722c0700 1 --2- 172.21.15.102:0/2833944144 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3f6c1055c0 0x7f3f6c10b130 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 rx=0x7f3f60002f70 tx=0x7f3f60009a50).stop 2024-04-17T22:29:27.986 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.997 7f3f722c0700 1 -- 172.21.15.102:0/2833944144 shutdown_connections 2024-04-17T22:29:27.986 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.997 7f3f722c0700 1 --2- 172.21.15.102:0/2833944144 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3f6c1055c0 0x7f3f6c10b130 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:27.986 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.997 7f3f722c0700 1 --2- 172.21.15.102:0/2833944144 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f3f6c0f59b0 0x7f3f6c10b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:27.986 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.997 7f3f722c0700 1 -- 172.21.15.102:0/2833944144 >> 172.21.15.102:0/2833944144 conn(0x7f3f6c113a90 msgr2=0x7f3f6c0f5140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:29:27.986 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.997 7f3f722c0700 1 -- 172.21.15.102:0/2833944144 shutdown_connections 2024-04-17T22:29:27.986 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:27.997 7f3f722c0700 1 -- 172.21.15.102:0/2833944144 wait complete. 2024-04-17T22:29:28.008 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.019 7f3f722c0700 1 Processor -- start 2024-04-17T22:29:28.008 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.020 7f3f722c0700 1 -- start start 2024-04-17T22:29:28.008 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.020 7f3f722c0700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3f6c0ce4e0 0x7f3f6c1ad830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:28.008 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.020 7f3f722c0700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f3f6c1ade60 0x7f3f6c1b3cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:28.008 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.020 7f3f722c0700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f3f6c1b42e0 0x7f3f6c1b4730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:28.008 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.020 7f3f722c0700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f6c1a6c20 con 0x7f3f6c0ce4e0 2024-04-17T22:29:28.008 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.020 7f3f722c0700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f6c0cdf10 con 0x7f3f6c1ade60 2024-04-17T22:29:28.008 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.020 7f3f722c0700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3f6c1b4d60 con 0x7f3f6c1b42e0 2024-04-17T22:29:28.008 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.020 7f3f6b887700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f3f6c1b42e0 0x7f3f6c1b4730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:28.008 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.020 7f3f6b086700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3f6c0ce4e0 0x7f3f6c1ad830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:28.008 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.020 7f3f6a885700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f3f6c1ade60 0x7f3f6c1b3cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:28.009 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.020 7f3f6b887700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f3f6c1b42e0 0x7f3f6c1b4730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:41676/0 (socket says 172.21.15.102:41676) 2024-04-17T22:29:28.009 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.020 7f3f6b086700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3f6c0ce4e0 0x7f3f6c1ad830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:38016/0 (socket says 172.21.15.102:38016) 2024-04-17T22:29:28.009 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.020 7f3f6b887700 1 -- 172.21.15.102:0/3694982668 learned_addr learned my addr 172.21.15.102:0/3694982668 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:29:28.009 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.020 7f3f6b887700 1 -- 172.21.15.102:0/3694982668 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f3f6c1ade60 msgr2=0x7f3f6c1b3cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:28.009 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.020 7f3f6b887700 1 --2- 172.21.15.102:0/3694982668 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f3f6c1ade60 0x7f3f6c1b3cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:28.009 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.020 7f3f6b887700 1 -- 172.21.15.102:0/3694982668 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3f6c0ce4e0 msgr2=0x7f3f6c1ad830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:28.009 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.020 7f3f6b887700 1 --2- 172.21.15.102:0/3694982668 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3f6c0ce4e0 0x7f3f6c1ad830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:28.009 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.020 7f3f6b887700 1 -- 172.21.15.102:0/3694982668 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f6c1bb190 con 0x7f3f6c1b42e0 2024-04-17T22:29:28.009 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.020 7f3f6b086700 1 --2- 172.21.15.102:0/3694982668 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3f6c0ce4e0 0x7f3f6c1ad830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:29:28.009 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.020 7f3f6a885700 1 --2- 172.21.15.102:0/3694982668 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f3f6c1ade60 0x7f3f6c1b3cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:29:28.009 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.020 7f3f6b887700 1 --2- 172.21.15.102:0/3694982668 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f3f6c1b42e0 0x7f3f6c1b4730 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7f3f6c0f2530 tx=0x7f3f6400cb60).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:29:28.009 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.021 7f3f68881700 1 -- 172.21.15.102:0/3694982668 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f3f6c1bb190 con 0x7f3f6c1b42e0 2024-04-17T22:29:28.009 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.021 7f3f68881700 1 -- 172.21.15.102:0/3694982668 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f64006ab0 con 0x7f3f6c1b42e0 2024-04-17T22:29:28.009 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.021 7f3f68881700 1 -- 172.21.15.102:0/3694982668 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f3f64007030 con 0x7f3f6c1b42e0 2024-04-17T22:29:28.010 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.021 7f3f722c0700 1 -- 172.21.15.102:0/3694982668 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f6c1bb390 con 0x7f3f6c1b42e0 2024-04-17T22:29:28.010 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.021 7f3f722c0700 1 -- 172.21.15.102:0/3694982668 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f6c1bb770 con 0x7f3f6c1b42e0 2024-04-17T22:29:28.010 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.021 7f3f68881700 1 -- 172.21.15.102:0/3694982668 <== mon.2 v2:172.21.15.102:3302/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7f3f64007230 con 0x7f3f6c1b42e0 2024-04-17T22:29:28.010 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.021 7f3f68881700 1 -- 172.21.15.102:0/3694982668 <== mon.2 v2:172.21.15.102:3302/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3194+0+0 (secure 0 0 0) 0x7f3f64002f70 con 0x7f3f6c1b42e0 2024-04-17T22:29:28.010 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.022 7f3f722c0700 1 -- 172.21.15.102:0/3694982668 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f6c0f09f0 con 0x7f3f6c1b42e0 2024-04-17T22:29:28.011 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.025 7f3f68881700 1 -- 172.21.15.102:0/3694982668 <== mon.2 v2:172.21.15.102:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+110455 (secure 0 0 0) 0x7f3f640068a0 con 0x7f3f6c1b42e0 2024-04-17T22:29:28.115 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.129 7f3f722c0700 1 -- 172.21.15.102:0/3694982668 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f3f6c0f00c0 con 0x7f3f6c1b42e0 2024-04-17T22:29:28.116 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.129 7f3f68881700 1 -- 172.21.15.102:0/3694982668 <== mon.2 v2:172.21.15.102:3302/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v17) v1 ==== 92+0+10 (secure 0 0 0) 0x7f3f64004020 con 0x7f3f6c1b42e0 2024-04-17T22:29:28.116 INFO:teuthology.orchestra.run.smithi102.stdout:pg_num: 8 2024-04-17T22:29:28.117 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.130 7f3f722c0700 1 -- 172.21.15.102:0/3694982668 >> v1:172.21.15.102:6816/42360 conn(0x7f3f4c0145a0 legacy=0x7f3f4c016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:28.117 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.130 7f3f722c0700 1 -- 172.21.15.102:0/3694982668 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f3f6c1b42e0 msgr2=0x7f3f6c1b4730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:28.117 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.130 7f3f722c0700 1 --2- 172.21.15.102:0/3694982668 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f3f6c1b42e0 0x7f3f6c1b4730 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7f3f6c0f2530 tx=0x7f3f6400cb60).stop 2024-04-17T22:29:28.117 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.130 7f3f722c0700 1 -- 172.21.15.102:0/3694982668 shutdown_connections 2024-04-17T22:29:28.117 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.130 7f3f722c0700 1 --2- 172.21.15.102:0/3694982668 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f3f6c1b42e0 0x7f3f6c1b4730 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:28.117 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.130 7f3f722c0700 1 --2- 172.21.15.102:0/3694982668 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3f6c0ce4e0 0x7f3f6c1ad830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:28.118 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.130 7f3f722c0700 1 --2- 172.21.15.102:0/3694982668 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f3f6c1ade60 0x7f3f6c1b3cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:28.118 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.130 7f3f722c0700 1 -- 172.21.15.102:0/3694982668 >> 172.21.15.102:0/3694982668 conn(0x7f3f6c0e10e0 msgr2=0x7f3f6c0dde20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:29:28.118 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.131 7f3f722c0700 1 -- 172.21.15.102:0/3694982668 shutdown_connections 2024-04-17T22:29:28.118 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.131 7f3f722c0700 1 -- 172.21.15.102:0/3694982668 wait complete. 2024-04-17T22:29:28.128 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-17T22:29:28.255 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.269 7f4ea51be700 1 Processor -- start 2024-04-17T22:29:28.256 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.269 7f4ea51be700 1 -- start start 2024-04-17T22:29:28.258 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.269 7f4ea51be700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f4ea010e240 0x7f4ea010e690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:28.258 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.269 7f4ea51be700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f4ea010ecc0 0x7f4ea00f1730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:28.258 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.269 7f4ea51be700 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ea00f21b0 con 0x7f4ea00f1d60 2024-04-17T22:29:28.258 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.269 7f4ea51be700 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ea00f9f10 con 0x7f4ea010ecc0 2024-04-17T22:29:28.258 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.269 7f4ea51be700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4ea00f9030 con 0x7f4ea010e240 2024-04-17T22:29:28.258 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.269 7f4e9d5fb700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f4ea010ecc0 0x7f4ea00f1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:28.258 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.270 7f4e9e5fd700 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f4ea00f1d60 0x7f4ea01aa4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:49224/0 (socket says 172.21.15.102:49224) 2024-04-17T22:29:28.258 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.270 7f4e9e5fd700 1 -- 172.21.15.102:0/3461079038 learned_addr learned my addr 172.21.15.102:0/3461079038 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-17T22:29:28.258 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.270 7f4e9ddfc700 1 --2- 172.21.15.102:0/3461079038 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f4ea010e240 0x7f4ea010e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:28.258 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.270 7f4e9d5fb700 1 -- 172.21.15.102:0/3461079038 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f4ea010e240 msgr2=0x7f4ea010e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:29:28.259 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.270 7f4e9d5fb700 1 --2- 172.21.15.102:0/3461079038 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f4ea010e240 0x7f4ea010e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:28.259 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.270 7f4e9d5fb700 1 -- 172.21.15.102:0/3461079038 >> v1:172.21.15.102:6789/0 conn(0x7f4ea00f1d60 legacy=0x7f4ea01aa4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:28.259 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.270 7f4e9cdfa700 1 -- 172.21.15.102:0/3461079038 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3499690665 0 0) 0x7f4e98001490 con 0x7f4ea00f1d60 2024-04-17T22:29:28.259 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.270 7f4e9d5fb700 1 -- 172.21.15.102:0/3461079038 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ea00f23d0 con 0x7f4ea010ecc0 2024-04-17T22:29:28.259 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.270 7f4e9ddfc700 1 --2- 172.21.15.102:0/3461079038 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f4ea010e240 0x7f4ea010e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:29:28.259 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.270 7f4e9d5fb700 1 --2- 172.21.15.102:0/3461079038 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4ea010ecc0 0x7f4ea00f1730 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 rx=0x7f4ea0103580 tx=0x7f4e9400ba20).ready entity=mon.0 client_cookie=c1ef906c262a5aa3 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:29:28.259 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.270 7f4e9cdfa700 1 -- 172.21.15.102:0/3461079038 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f4ea00f23d0 con 0x7f4ea010ecc0 2024-04-17T22:29:28.259 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.270 7f4e9cdfa700 1 -- 172.21.15.102:0/3461079038 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e9400d5c0 con 0x7f4ea010ecc0 2024-04-17T22:29:28.259 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.270 7f4e9cdfa700 1 -- 172.21.15.102:0/3461079038 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f4e9400d9e0 con 0x7f4ea010ecc0 2024-04-17T22:29:28.259 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.270 7f4ea51be700 1 -- 172.21.15.102:0/3461079038 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4ea010ecc0 msgr2=0x7f4ea00f1730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:28.259 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.270 7f4ea51be700 1 --2- 172.21.15.102:0/3461079038 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4ea010ecc0 0x7f4ea00f1730 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 rx=0x7f4ea0103580 tx=0x7f4e9400ba20).stop 2024-04-17T22:29:28.259 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.271 7f4ea51be700 1 -- 172.21.15.102:0/3461079038 shutdown_connections 2024-04-17T22:29:28.259 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.271 7f4ea51be700 1 --2- 172.21.15.102:0/3461079038 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f4ea010e240 0x7f4ea010e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:28.259 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.271 7f4ea51be700 1 --2- 172.21.15.102:0/3461079038 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4ea010ecc0 0x7f4ea00f1730 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:28.259 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.271 7f4ea51be700 1 -- 172.21.15.102:0/3461079038 >> 172.21.15.102:0/3461079038 conn(0x7f4ea01138a0 msgr2=0x7f4ea00f4f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:29:28.260 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.271 7f4ea51be700 1 -- 172.21.15.102:0/3461079038 shutdown_connections 2024-04-17T22:29:28.260 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.271 7f4ea51be700 1 -- 172.21.15.102:0/3461079038 wait complete. 2024-04-17T22:29:28.282 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.294 7f4ea51be700 1 Processor -- start 2024-04-17T22:29:28.282 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.294 7f4ea51be700 1 -- start start 2024-04-17T22:29:28.282 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.294 7f4ea51be700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4ea00ce200 0x7f4ea00d5ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:28.282 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.294 7f4ea51be700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4ea00d6620 0x7f4ea01b42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:28.283 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.294 7f4ea51be700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f4ea00d6a70 0x7f4ea01b4900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:28.283 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.294 7f4ea51be700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ea00fa1b0 con 0x7f4ea00ce200 2024-04-17T22:29:28.283 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.294 7f4ea51be700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ea00cd640 con 0x7f4ea00d6620 2024-04-17T22:29:28.283 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.294 7f4ea51be700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4ea00cdc90 con 0x7f4ea00d6a70 2024-04-17T22:29:28.283 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.294 7f4e9d5fb700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4ea00d6620 0x7f4ea01b42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:28.283 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.294 7f4e9e5fd700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f4ea00d6a70 0x7f4ea01b4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:28.283 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.294 7f4e9ddfc700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4ea00ce200 0x7f4ea00d5ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:28.283 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.294 7f4e9d5fb700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4ea00d6620 0x7f4ea01b42d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:56128/0 (socket says 172.21.15.102:56128) 2024-04-17T22:29:28.283 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.294 7f4e9d5fb700 1 -- 172.21.15.102:0/823881431 learned_addr learned my addr 172.21.15.102:0/823881431 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:29:28.283 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.294 7f4e9ddfc700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4ea00ce200 0x7f4ea00d5ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:38048/0 (socket says 172.21.15.102:38048) 2024-04-17T22:29:28.283 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.294 7f4e9e5fd700 1 -- 172.21.15.102:0/823881431 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4ea00d6620 msgr2=0x7f4ea01b42d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:28.283 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.294 7f4e9e5fd700 1 --2- 172.21.15.102:0/823881431 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4ea00d6620 0x7f4ea01b42d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:28.283 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.294 7f4e9e5fd700 1 -- 172.21.15.102:0/823881431 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4ea00ce200 msgr2=0x7f4ea00d5ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:28.283 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.294 7f4e9e5fd700 1 --2- 172.21.15.102:0/823881431 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4ea00ce200 0x7f4ea00d5ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:28.283 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.294 7f4e9e5fd700 1 -- 172.21.15.102:0/823881431 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ea00c4500 con 0x7f4ea00d6a70 2024-04-17T22:29:28.284 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.294 7f4e9ddfc700 1 --2- 172.21.15.102:0/823881431 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4ea00ce200 0x7f4ea00d5ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:29:28.284 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.295 7f4e9e5fd700 1 --2- 172.21.15.102:0/823881431 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f4ea00d6a70 0x7f4ea01b4900 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7f4ea00d75e0 tx=0x7f4e9800c960).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:29:28.284 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.295 7f4e8effd700 1 -- 172.21.15.102:0/823881431 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f4e98005bb0 con 0x7f4ea00d6a70 2024-04-17T22:29:28.284 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.295 7f4e8effd700 1 -- 172.21.15.102:0/823881431 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e98005db0 con 0x7f4ea00d6a70 2024-04-17T22:29:28.284 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.295 7f4ea51be700 1 -- 172.21.15.102:0/823881431 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ea01b4f30 con 0x7f4ea00d6a70 2024-04-17T22:29:28.284 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.295 7f4e8effd700 1 -- 172.21.15.102:0/823881431 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f4e98004aa0 con 0x7f4ea00d6a70 2024-04-17T22:29:28.284 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.295 7f4ea51be700 1 -- 172.21.15.102:0/823881431 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ea01bbba0 con 0x7f4ea00d6a70 2024-04-17T22:29:28.284 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.295 7f4e8effd700 1 -- 172.21.15.102:0/823881431 <== mon.2 v2:172.21.15.102:3302/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7f4e98004ca0 con 0x7f4ea00d6a70 2024-04-17T22:29:28.284 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.295 7f4e8effd700 1 -- 172.21.15.102:0/823881431 <== mon.2 v2:172.21.15.102:3302/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3194+0+0 (secure 0 0 0) 0x7f4e9800bcb0 con 0x7f4ea00d6a70 2024-04-17T22:29:28.284 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.296 7f4ea51be700 1 -- 172.21.15.102:0/823881431 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ea00f0800 con 0x7f4ea00d6a70 2024-04-17T22:29:28.285 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.299 7f4e8effd700 1 -- 172.21.15.102:0/823881431 <== mon.2 v2:172.21.15.102:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+110455 (secure 0 0 0) 0x7f4e980059a0 con 0x7f4ea00d6a70 2024-04-17T22:29:28.389 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.403 7f4ea51be700 1 -- 172.21.15.102:0/823881431 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f4ea00efc30 con 0x7f4ea00d6a70 2024-04-17T22:29:28.390 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.403 7f4e8effd700 1 -- 172.21.15.102:0/823881431 <== mon.2 v2:172.21.15.102:3302/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v17) v1 ==== 104+0+11 (secure 0 0 0) 0x7f4ea0114420 con 0x7f4ea00d6a70 2024-04-17T22:29:28.390 INFO:teuthology.orchestra.run.smithi102.stdout:pg_num: 64 2024-04-17T22:29:28.394 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.404 7f4ea51be700 1 -- 172.21.15.102:0/823881431 >> v1:172.21.15.102:6816/42360 conn(0x7f4e7c0145a0 legacy=0x7f4e7c016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:28.394 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.404 7f4ea51be700 1 -- 172.21.15.102:0/823881431 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f4ea00d6a70 msgr2=0x7f4ea01b4900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:28.394 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.404 7f4ea51be700 1 --2- 172.21.15.102:0/823881431 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f4ea00d6a70 0x7f4ea01b4900 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7f4ea00d75e0 tx=0x7f4e9800c960).stop 2024-04-17T22:29:28.394 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.404 7f4ea51be700 1 -- 172.21.15.102:0/823881431 shutdown_connections 2024-04-17T22:29:28.394 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.404 7f4ea51be700 1 --2- 172.21.15.102:0/823881431 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f4ea00d6a70 0x7f4ea01b4900 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:28.395 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.404 7f4ea51be700 1 --2- 172.21.15.102:0/823881431 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4ea00ce200 0x7f4ea00d5ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:28.395 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.404 7f4ea51be700 1 --2- 172.21.15.102:0/823881431 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4ea00d6620 0x7f4ea01b42d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:28.395 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.405 7f4ea51be700 1 -- 172.21.15.102:0/823881431 >> 172.21.15.102:0/823881431 conn(0x7f4ea00c38d0 msgr2=0x7f4ea00dae70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:29:28.395 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.405 7f4ea51be700 1 -- 172.21.15.102:0/823881431 shutdown_connections 2024-04-17T22:29:28.395 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.405 7f4ea51be700 1 -- 172.21.15.102:0/823881431 wait complete. 2024-04-17T22:29:28.402 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-17T22:29:28.532 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.543 7f9eac72d700 1 Processor -- start 2024-04-17T22:29:28.532 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.543 7f9eac72d700 1 -- start start 2024-04-17T22:29:28.532 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.543 7f9eac72d700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f9ea40f5640 0x7f9ea410b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:28.532 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.544 7f9eac72d700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f9ea41041c0 0x7f9ea41a80d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:28.532 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.544 7f9eac72d700 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ea40ad030 con 0x7f9ea410bc10 2024-04-17T22:29:28.532 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.544 7f9eac72d700 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ea4104c80 con 0x7f9ea41041c0 2024-04-17T22:29:28.532 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.544 7f9eac72d700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9ea41057a0 con 0x7f9ea40f5640 2024-04-17T22:29:28.532 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.544 7f9ea9528700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f9ea40f5640 0x7f9ea410b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:28.532 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.544 7f9ea9d29700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f9ea41041c0 0x7f9ea41a80d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:28.532 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.544 7f9ea9d29700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f9ea41041c0 0x7f9ea41a80d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:38058/0 (socket says 172.21.15.102:38058) 2024-04-17T22:29:28.532 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.544 7f9ea9528700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f9ea40f5640 0x7f9ea410b5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:41716/0 (socket says 172.21.15.102:41716) 2024-04-17T22:29:28.532 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.544 7f9ea9d29700 1 -- 172.21.15.102:0/2285270179 learned_addr learned my addr 172.21.15.102:0/2285270179 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:29:28.533 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.544 7f9ea8d27700 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f9ea410bc10 0x7f9ea41039c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:49238/0 (socket says 172.21.15.102:49238) 2024-04-17T22:29:28.533 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.544 7f9ea9d29700 1 -- 172.21.15.102:0/2285270179 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f9ea40f5640 msgr2=0x7f9ea410b5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:29:28.533 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.544 7f9ea9d29700 1 --2- 172.21.15.102:0/2285270179 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f9ea40f5640 0x7f9ea410b5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:28.533 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.544 7f9ea9d29700 1 -- 172.21.15.102:0/2285270179 >> v1:172.21.15.102:6789/0 conn(0x7f9ea410bc10 legacy=0x7f9ea41039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:28.533 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.544 7f9ea9d29700 1 -- 172.21.15.102:0/2285270179 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ea4104610 con 0x7f9ea41041c0 2024-04-17T22:29:28.533 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.544 7f9e97fff700 1 -- 172.21.15.102:0/2285270179 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1524328864 0 0) 0x7f9e98001110 con 0x7f9ea410bc10 2024-04-17T22:29:28.533 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.544 7f9ea9d29700 1 --2- 172.21.15.102:0/2285270179 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9ea41041c0 0x7f9ea41a80d0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 rx=0x7f9ea4105950 tx=0x7f9e9c00bec0).ready entity=mon.0 client_cookie=497c05b7b4cb350 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:29:28.533 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.544 7f9e97fff700 1 -- 172.21.15.102:0/2285270179 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9ea4104610 con 0x7f9ea41041c0 2024-04-17T22:29:28.533 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.544 7f9e97fff700 1 -- 172.21.15.102:0/2285270179 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e9c00cce0 con 0x7f9ea41041c0 2024-04-17T22:29:28.533 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.544 7f9e97fff700 1 -- 172.21.15.102:0/2285270179 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9e9c014070 con 0x7f9ea41041c0 2024-04-17T22:29:28.533 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.544 7f9eac72d700 1 -- 172.21.15.102:0/2285270179 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9ea41041c0 msgr2=0x7f9ea41a80d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:28.533 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.544 7f9eac72d700 1 --2- 172.21.15.102:0/2285270179 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9ea41041c0 0x7f9ea41a80d0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 rx=0x7f9ea4105950 tx=0x7f9e9c00bec0).stop 2024-04-17T22:29:28.533 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.545 7f9eac72d700 1 -- 172.21.15.102:0/2285270179 shutdown_connections 2024-04-17T22:29:28.533 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.545 7f9eac72d700 1 --2- 172.21.15.102:0/2285270179 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f9ea40f5640 0x7f9ea410b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:28.533 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.545 7f9eac72d700 1 --2- 172.21.15.102:0/2285270179 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9ea41041c0 0x7f9ea41a80d0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:28.534 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.545 7f9eac72d700 1 -- 172.21.15.102:0/2285270179 >> 172.21.15.102:0/2285270179 conn(0x7f9ea41134c0 msgr2=0x7f9ea4113910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:29:28.534 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.545 7f9eac72d700 1 -- 172.21.15.102:0/2285270179 shutdown_connections 2024-04-17T22:29:28.534 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.545 7f9eac72d700 1 -- 172.21.15.102:0/2285270179 wait complete. 2024-04-17T22:29:28.556 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.567 7f9eac72d700 1 Processor -- start 2024-04-17T22:29:28.556 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.568 7f9eac72d700 1 -- start start 2024-04-17T22:29:28.556 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.568 7f9eac72d700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9ea40d9420 0x7f9ea40d3480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:28.556 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.568 7f9eac72d700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f9ea40d3ab0 0x7f9ea41b01b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:28.556 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.568 7f9eac72d700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f9ea40d3f00 0x7f9ea41b07e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:28.556 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.568 7f9eac72d700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ea41a4b40 con 0x7f9ea40d9420 2024-04-17T22:29:28.556 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.568 7f9eac72d700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ea41a4d90 con 0x7f9ea40d3ab0 2024-04-17T22:29:28.556 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.568 7f9eac72d700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9ea40d9870 con 0x7f9ea40d3f00 2024-04-17T22:29:28.556 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.568 7f9ea9528700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9ea40d9420 0x7f9ea40d3480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:28.556 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.568 7f9ea9d29700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f9ea40d3f00 0x7f9ea41b07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:28.556 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.568 7f9ea9528700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9ea40d9420 0x7f9ea40d3480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:38064/0 (socket says 172.21.15.102:38064) 2024-04-17T22:29:28.557 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.568 7f9ea8d27700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f9ea40d3ab0 0x7f9ea41b01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:28.557 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.568 7f9ea9528700 1 -- 172.21.15.102:0/3375664827 learned_addr learned my addr 172.21.15.102:0/3375664827 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:29:28.557 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.568 7f9ea9528700 1 -- 172.21.15.102:0/3375664827 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f9ea40d3f00 msgr2=0x7f9ea41b07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:28.557 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.568 7f9ea9528700 1 --2- 172.21.15.102:0/3375664827 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f9ea40d3f00 0x7f9ea41b07e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:28.557 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.568 7f9ea9528700 1 -- 172.21.15.102:0/3375664827 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f9ea40d3ab0 msgr2=0x7f9ea41b01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:28.557 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.568 7f9ea9528700 1 --2- 172.21.15.102:0/3375664827 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f9ea40d3ab0 0x7f9ea41b01b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:28.557 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.568 7f9ea9528700 1 -- 172.21.15.102:0/3375664827 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ea41b0e10 con 0x7f9ea40d9420 2024-04-17T22:29:28.557 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.568 7f9ea8d27700 1 --2- 172.21.15.102:0/3375664827 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f9ea40d3ab0 0x7f9ea41b01b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:29:28.557 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.568 7f9ea9d29700 1 --2- 172.21.15.102:0/3375664827 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f9ea40d3f00 0x7f9ea41b07e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:29:28.557 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.568 7f9ea9528700 1 --2- 172.21.15.102:0/3375664827 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9ea40d9420 0x7f9ea40d3480 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 rx=0x7f9ea40d6540 tx=0x7f9ea00103a0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:29:28.557 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.569 7f9e967fc700 1 -- 172.21.15.102:0/3375664827 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9ea41b0e10 con 0x7f9ea40d9420 2024-04-17T22:29:28.557 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.569 7f9e967fc700 1 -- 172.21.15.102:0/3375664827 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ea0014040 con 0x7f9ea40d9420 2024-04-17T22:29:28.557 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.569 7f9eac72d700 1 -- 172.21.15.102:0/3375664827 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ea41b1070 con 0x7f9ea40d9420 2024-04-17T22:29:28.557 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.569 7f9e967fc700 1 -- 172.21.15.102:0/3375664827 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9ea0010b20 con 0x7f9ea40d9420 2024-04-17T22:29:28.558 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.569 7f9eac72d700 1 -- 172.21.15.102:0/3375664827 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ea41b7b90 con 0x7f9ea40d9420 2024-04-17T22:29:28.558 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.569 7f9e967fc700 1 -- 172.21.15.102:0/3375664827 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7f9ea0010d20 con 0x7f9ea40d9420 2024-04-17T22:29:28.558 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.569 7f9e967fc700 1 -- 172.21.15.102:0/3375664827 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3194+0+0 (secure 0 0 0) 0x7f9ea0019d10 con 0x7f9ea40d9420 2024-04-17T22:29:28.558 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.570 7f9eac72d700 1 -- 172.21.15.102:0/3375664827 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ea40f09f0 con 0x7f9ea40d9420 2024-04-17T22:29:28.559 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.573 7f9e967fc700 1 -- 172.21.15.102:0/3375664827 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+110455 (secure 0 0 0) 0x7f9ea0000c30 con 0x7f9ea40d9420 2024-04-17T22:29:28.665 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.678 7f9eac72d700 1 -- 172.21.15.102:0/3375664827 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f9ea41aa8d0 con 0x7f9ea40d9420 2024-04-17T22:29:28.665 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.679 7f9e967fc700 1 -- 172.21.15.102:0/3375664827 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v17) v1 ==== 100+0+11 (secure 0 0 0) 0x7f9ea0012e10 con 0x7f9ea40d9420 2024-04-17T22:29:28.665 INFO:teuthology.orchestra.run.smithi102.stdout:pg_num: 64 2024-04-17T22:29:28.666 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.680 7f9eac72d700 1 -- 172.21.15.102:0/3375664827 >> v1:172.21.15.102:6816/42360 conn(0x7f9e8c0145a0 legacy=0x7f9e8c016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:28.667 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.680 7f9eac72d700 1 -- 172.21.15.102:0/3375664827 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9ea40d9420 msgr2=0x7f9ea40d3480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:28.667 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.680 7f9eac72d700 1 --2- 172.21.15.102:0/3375664827 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9ea40d9420 0x7f9ea40d3480 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 rx=0x7f9ea40d6540 tx=0x7f9ea00103a0).stop 2024-04-17T22:29:28.667 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.680 7f9eac72d700 1 -- 172.21.15.102:0/3375664827 shutdown_connections 2024-04-17T22:29:28.667 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.680 7f9eac72d700 1 --2- 172.21.15.102:0/3375664827 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f9ea40d3f00 0x7f9ea41b07e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:28.667 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.680 7f9eac72d700 1 --2- 172.21.15.102:0/3375664827 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9ea40d9420 0x7f9ea40d3480 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:28.667 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.680 7f9eac72d700 1 --2- 172.21.15.102:0/3375664827 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f9ea40d3ab0 0x7f9ea41b01b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:28.667 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.680 7f9eac72d700 1 -- 172.21.15.102:0/3375664827 >> 172.21.15.102:0/3375664827 conn(0x7f9ea40e04b0 msgr2=0x7f9ea40c23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:29:28.667 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.680 7f9eac72d700 1 -- 172.21.15.102:0/3375664827 shutdown_connections 2024-04-17T22:29:28.667 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.680 7f9eac72d700 1 -- 172.21.15.102:0/3375664827 wait complete. 2024-04-17T22:29:28.677 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-17T22:29:28.806 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.817 7f535b28b700 1 Processor -- start 2024-04-17T22:29:28.806 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.817 7f535b28b700 1 -- start start 2024-04-17T22:29:28.806 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.817 7f535b28b700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f53541a6440 0x7f53541a88a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:28.806 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.817 7f535b28b700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f53541a8ed0 0x7f53541ab330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:28.806 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.817 7f535b28b700 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f535410ee20 con 0x7f535410e1d0 2024-04-17T22:29:28.806 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.817 7f535b28b700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53540f9ea0 con 0x7f53541a8ed0 2024-04-17T22:29:28.806 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.817 7f535b28b700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f53540f8fc0 con 0x7f53541a6440 2024-04-17T22:29:28.806 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.818 7f5358887700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f53541a8ed0 0x7f53541ab330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:28.807 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.818 7f534bfff700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f53541a6440 0x7f53541a88a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:28.807 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.818 7f5358887700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f53541a8ed0 0x7f53541ab330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:56136/0 (socket says 172.21.15.102:56136) 2024-04-17T22:29:28.807 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.818 7f534bfff700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f53541a6440 0x7f53541a88a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:41732/0 (socket says 172.21.15.102:41732) 2024-04-17T22:29:28.807 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.818 7f5358887700 1 -- 172.21.15.102:0/1953596819 learned_addr learned my addr 172.21.15.102:0/1953596819 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:29:28.807 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.818 7f5353fff700 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f535410e1d0 0x7f535410e620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:49248/0 (socket says 172.21.15.102:49248) 2024-04-17T22:29:28.807 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.818 7f53537fe700 1 -- 172.21.15.102:0/1953596819 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3457974568 0 0) 0x7f5344001110 con 0x7f535410e1d0 2024-04-17T22:29:28.807 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.818 7f534bfff700 1 -- 172.21.15.102:0/1953596819 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f53541a8ed0 msgr2=0x7f53541ab330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:29:28.807 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.818 7f534bfff700 1 --2- 172.21.15.102:0/1953596819 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f53541a8ed0 0x7f53541ab330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:28.807 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.818 7f534bfff700 1 -- 172.21.15.102:0/1953596819 >> v1:172.21.15.102:6789/0 conn(0x7f535410e1d0 legacy=0x7f535410e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:28.807 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.818 7f534bfff700 1 -- 172.21.15.102:0/1953596819 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53541ac970 con 0x7f53541a6440 2024-04-17T22:29:28.807 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.818 7f5358887700 1 --2- 172.21.15.102:0/1953596819 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f53541a8ed0 0x7f53541ab330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:29:28.807 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.818 7f534bfff700 1 --2- 172.21.15.102:0/1953596819 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f53541a6440 0x7f53541a88a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7f53540f9560 tx=0x7f533c00b9d0).ready entity=mon.2 client_cookie=6eb46bbcd34a956f server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:29:28.807 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.818 7f53537fe700 1 -- 172.21.15.102:0/1953596819 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f53541ac970 con 0x7f53541a6440 2024-04-17T22:29:28.807 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.818 7f53537fe700 1 -- 172.21.15.102:0/1953596819 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f533c00d570 con 0x7f53541a6440 2024-04-17T22:29:28.808 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.818 7f53537fe700 1 -- 172.21.15.102:0/1953596819 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f533c00d940 con 0x7f53541a6440 2024-04-17T22:29:28.808 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.818 7f535b28b700 1 -- 172.21.15.102:0/1953596819 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f53541a6440 msgr2=0x7f53541a88a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:28.808 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.818 7f535b28b700 1 --2- 172.21.15.102:0/1953596819 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f53541a6440 0x7f53541a88a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7f53540f9560 tx=0x7f533c00b9d0).stop 2024-04-17T22:29:28.808 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.819 7f535b28b700 1 -- 172.21.15.102:0/1953596819 shutdown_connections 2024-04-17T22:29:28.808 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.819 7f535b28b700 1 --2- 172.21.15.102:0/1953596819 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f53541a8ed0 0x7f53541ab330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:28.808 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.819 7f535b28b700 1 --2- 172.21.15.102:0/1953596819 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f53541a6440 0x7f53541a88a0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:28.808 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.819 7f535b28b700 1 -- 172.21.15.102:0/1953596819 >> 172.21.15.102:0/1953596819 conn(0x7f5354113830 msgr2=0x7f53540f4ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:29:28.808 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.819 7f535b28b700 1 -- 172.21.15.102:0/1953596819 shutdown_connections 2024-04-17T22:29:28.808 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.819 7f535b28b700 1 -- 172.21.15.102:0/1953596819 wait complete. 2024-04-17T22:29:28.831 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.842 7f535b28b700 1 Processor -- start 2024-04-17T22:29:28.831 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.842 7f535b28b700 1 -- start start 2024-04-17T22:29:28.831 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.842 7f535b28b700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f53540da620 0x7f53540d52b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:28.831 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.842 7f535b28b700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f53540d58e0 0x7f53541b41d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:28.831 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.842 7f535b28b700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f53540d5d30 0x7f53541b4800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:28.831 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.842 7f535b28b700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53540fa140 con 0x7f53540d5d30 2024-04-17T22:29:28.831 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.842 7f535b28b700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53540d9d60 con 0x7f53540da620 2024-04-17T22:29:28.831 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.842 7f535b28b700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f53540da0b0 con 0x7f53540d58e0 2024-04-17T22:29:28.831 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.842 7f5353fff700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f53540da620 0x7f53540d52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:28.831 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.842 7f534bfff700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f53540d58e0 0x7f53541b41d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:28.831 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.842 7f534bfff700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f53540d58e0 0x7f53541b41d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:41742/0 (socket says 172.21.15.102:41742) 2024-04-17T22:29:28.831 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.842 7f5353fff700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f53540da620 0x7f53540d52b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:56148/0 (socket says 172.21.15.102:56148) 2024-04-17T22:29:28.832 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.842 7f534bfff700 1 -- 172.21.15.102:0/3372515173 learned_addr learned my addr 172.21.15.102:0/3372515173 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:29:28.832 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.842 7f5358887700 1 --2- 172.21.15.102:0/3372515173 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f53540d5d30 0x7f53541b4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:28.832 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.843 7f5353fff700 1 -- 172.21.15.102:0/3372515173 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f53540d58e0 msgr2=0x7f53541b41d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:28.832 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.843 7f5353fff700 1 --2- 172.21.15.102:0/3372515173 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f53540d58e0 0x7f53541b41d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:28.832 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.843 7f5353fff700 1 -- 172.21.15.102:0/3372515173 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f53540d5d30 msgr2=0x7f53541b4800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:28.832 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.843 7f5353fff700 1 --2- 172.21.15.102:0/3372515173 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f53540d5d30 0x7f53541b4800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:28.832 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.843 7f5353fff700 1 -- 172.21.15.102:0/3372515173 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53540d9940 con 0x7f53540da620 2024-04-17T22:29:28.832 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.843 7f534bfff700 1 --2- 172.21.15.102:0/3372515173 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f53540d58e0 0x7f53541b41d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:29:28.832 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.843 7f5358887700 1 --2- 172.21.15.102:0/3372515173 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f53540d5d30 0x7f53541b4800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:29:28.832 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.843 7f5353fff700 1 --2- 172.21.15.102:0/3372515173 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f53540da620 0x7f53540d52b0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7f53540c7060 tx=0x7f534400c860).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:29:28.832 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.843 7f5351ffb700 1 -- 172.21.15.102:0/3372515173 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f53540d9940 con 0x7f53540da620 2024-04-17T22:29:28.832 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.843 7f5351ffb700 1 -- 172.21.15.102:0/3372515173 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5344007e00 con 0x7f53540da620 2024-04-17T22:29:28.832 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.843 7f535b28b700 1 -- 172.21.15.102:0/3372515173 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53541b4e30 con 0x7f53540da620 2024-04-17T22:29:28.833 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.843 7f5351ffb700 1 -- 172.21.15.102:0/3372515173 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f5344005560 con 0x7f53540da620 2024-04-17T22:29:28.833 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.843 7f535b28b700 1 -- 172.21.15.102:0/3372515173 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53541bb9d0 con 0x7f53540da620 2024-04-17T22:29:28.833 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.843 7f5351ffb700 1 -- 172.21.15.102:0/3372515173 <== mon.1 v2:172.21.15.102:3301/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7f534401b070 con 0x7f53540da620 2024-04-17T22:29:28.833 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.843 7f5351ffb700 1 -- 172.21.15.102:0/3372515173 <== mon.1 v2:172.21.15.102:3301/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3194+0+0 (secure 0 0 0) 0x7f5344004a90 con 0x7f53540da620 2024-04-17T22:29:28.833 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.844 7f535b28b700 1 -- 172.21.15.102:0/3372515173 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53540f0790 con 0x7f53540da620 2024-04-17T22:29:28.833 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.847 7f5351ffb700 1 -- 172.21.15.102:0/3372515173 <== mon.1 v2:172.21.15.102:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+110455 (secure 0 0 0) 0x7f53440070d0 con 0x7f53540da620 2024-04-17T22:29:28.978 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.991 7f535b28b700 1 -- 172.21.15.102:0/3372515173 >> v1:172.21.15.102:6816/42360 conn(0x7f53380145a0 legacy=0x7f5338016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:28.978 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.991 7f535b28b700 1 -- 172.21.15.102:0/3372515173 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f53540da620 msgr2=0x7f53540d52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:28.978 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.991 7f535b28b700 1 --2- 172.21.15.102:0/3372515173 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f53540da620 0x7f53540d52b0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7f53540c7060 tx=0x7f534400c860).stop 2024-04-17T22:29:28.978 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.991 7f535b28b700 1 -- 172.21.15.102:0/3372515173 shutdown_connections 2024-04-17T22:29:28.979 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.991 7f535b28b700 1 --2- 172.21.15.102:0/3372515173 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f53540d5d30 0x7f53541b4800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:28.979 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.991 7f535b28b700 1 --2- 172.21.15.102:0/3372515173 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f53540da620 0x7f53540d52b0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:28.979 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.991 7f535b28b700 1 --2- 172.21.15.102:0/3372515173 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f53540d58e0 0x7f53541b41d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:28.979 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.991 7f535b28b700 1 -- 172.21.15.102:0/3372515173 >> 172.21.15.102:0/3372515173 conn(0x7f53540c6af0 msgr2=0x7f53541a8270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:29:28.979 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.991 7f535b28b700 1 -- 172.21.15.102:0/3372515173 shutdown_connections 2024-04-17T22:29:28.979 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:28.991 7f535b28b700 1 -- 172.21.15.102:0/3372515173 wait complete. 2024-04-17T22:29:28.979 INFO:teuthology.orchestra.run.smithi102.stderr:no valid command found; 10 closest matches: 2024-04-17T22:29:28.979 INFO:teuthology.orchestra.run.smithi102.stderr:osd perf 2024-04-17T22:29:28.979 INFO:teuthology.orchestra.run.smithi102.stderr:osd df {plain|tree} {class|name} {} 2024-04-17T22:29:28.980 INFO:teuthology.orchestra.run.smithi102.stderr:osd blocked-by 2024-04-17T22:29:28.980 INFO:teuthology.orchestra.run.smithi102.stderr:osd pool stats {} 2024-04-17T22:29:28.980 INFO:teuthology.orchestra.run.smithi102.stderr:osd pool scrub [...] 2024-04-17T22:29:28.980 INFO:teuthology.orchestra.run.smithi102.stderr:osd pool deep-scrub [...] 2024-04-17T22:29:28.980 INFO:teuthology.orchestra.run.smithi102.stderr:osd pool repair [...] 2024-04-17T22:29:28.980 INFO:teuthology.orchestra.run.smithi102.stderr:osd pool force-recovery [...] 2024-04-17T22:29:28.980 INFO:teuthology.orchestra.run.smithi102.stderr:osd pool force-backfill [...] 2024-04-17T22:29:28.980 INFO:teuthology.orchestra.run.smithi102.stderr:osd pool cancel-force-recovery [...] 2024-04-17T22:29:28.980 INFO:teuthology.orchestra.run.smithi102.stderr:Error EINVAL: invalid command 2024-04-17T22:29:28.988 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-17T22:29:28.989 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blacklist ls 2024-04-17T22:29:29.116 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:29.128 7f74591d9700 1 Processor -- start 2024-04-17T22:29:29.116 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:29.128 7f74591d9700 1 -- start start 2024-04-17T22:29:29.116 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:29.128 7f74591d9700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f74540f5840 0x7f745410b010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:29.117 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:29.128 7f74591d9700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f745410ba90 0x7f74541a02f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:29.117 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:29.128 7f74591d9700 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7454103b40 con 0x7f745410b640 2024-04-17T22:29:29.117 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:29.128 7f74591d9700 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f745410bee0 con 0x7f74540f5840 2024-04-17T22:29:29.117 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:29.128 7f74591d9700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74540fa370 con 0x7f745410ba90 2024-04-17T22:29:29.117 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:29.128 7f7451dfc700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f74540f5840 0x7f745410b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:29.117 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:29.128 7f74525fd700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f745410ba90 0x7f74541a02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:29.117 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:29.128 7f7451dfc700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f74540f5840 0x7f745410b010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:38078/0 (socket says 172.21.15.102:38078) 2024-04-17T22:29:29.117 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:29.128 7f7451dfc700 1 -- 172.21.15.102:0/466051309 learned_addr learned my addr 172.21.15.102:0/466051309 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:29:29.117 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:29.128 7f74515fb700 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f745410b640 0x7f745419faf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:49264/0 (socket says 172.21.15.102:49264) 2024-04-17T22:29:29.117 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:29.129 7f7451dfc700 1 -- 172.21.15.102:0/466051309 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f745410ba90 msgr2=0x7f74541a02f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:29:29.117 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:29.129 7f7451dfc700 1 --2- 172.21.15.102:0/466051309 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f745410ba90 0x7f74541a02f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:29.117 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:29.129 7f7451dfc700 1 -- 172.21.15.102:0/466051309 >> v1:172.21.15.102:6789/0 conn(0x7f745410b640 legacy=0x7f745419faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:29.117 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:29.129 7f7451dfc700 1 -- 172.21.15.102:0/466051309 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74541a0920 con 0x7f74540f5840 2024-04-17T22:29:29.117 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:29.129 7f7451dfc700 1 --2- 172.21.15.102:0/466051309 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f74540f5840 0x7f745410b010 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 rx=0x7f7454103d60 tx=0x7f743c009a80).ready entity=mon.0 client_cookie=d880833d5851d6d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:29:29.118 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:29.129 7f7450dfa700 1 -- 172.21.15.102:0/466051309 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f74541a0920 con 0x7f74540f5840 2024-04-17T22:29:29.118 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:29.129 7f7450dfa700 1 -- 172.21.15.102:0/466051309 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f743c00f510 con 0x7f74540f5840 2024-04-17T22:29:29.118 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:29.129 7f7450dfa700 1 -- 172.21.15.102:0/466051309 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f743c00f6f0 con 0x7f74540f5840 2024-04-17T22:29:29.118 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:29.129 7f74591d9700 1 -- 172.21.15.102:0/466051309 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f74540f5840 msgr2=0x7f745410b010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:29.118 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:29.129 7f74591d9700 1 --2- 172.21.15.102:0/466051309 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f74540f5840 0x7f745410b010 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 rx=0x7f7454103d60 tx=0x7f743c009a80).stop 2024-04-17T22:29:29.118 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:29.129 7f74591d9700 1 -- 172.21.15.102:0/466051309 shutdown_connections 2024-04-17T22:29:29.118 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:29.129 7f74591d9700 1 --2- 172.21.15.102:0/466051309 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f745410ba90 0x7f74541a02f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:29.118 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:29.129 7f74591d9700 1 --2- 172.21.15.102:0/466051309 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f74540f5840 0x7f745410b010 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:29.118 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:29.129 7f74591d9700 1 -- 172.21.15.102:0/466051309 >> 172.21.15.102:0/466051309 conn(0x7f74541138a0 msgr2=0x7f74540f4f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:29:29.118 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:29.129 7f74591d9700 1 -- 172.21.15.102:0/466051309 shutdown_connections 2024-04-17T22:29:29.118 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:29.130 7f74591d9700 1 -- 172.21.15.102:0/466051309 wait complete. 2024-04-17T22:29:29.141 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:29.152 7f74591d9700 1 Processor -- start 2024-04-17T22:29:29.141 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:29.152 7f74591d9700 1 -- start start 2024-04-17T22:29:29.141 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:29.152 7f74591d9700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f74541aa0a0 0x7f74541aa4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:29.141 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:29.152 7f74591d9700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f74541aab20 0x7f74541b0780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:29.141 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:29.152 7f74591d9700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f74541b0db0 0x7f74541b1200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:29:29.141 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:29.152 7f74591d9700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74540cced0 con 0x7f74541aa0a0 2024-04-17T22:29:29.141 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:29.152 7f74591d9700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74540cd220 con 0x7f74541b0db0 2024-04-17T22:29:29.141 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:29.152 7f74591d9700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f74540c4240 con 0x7f74541aab20 2024-04-17T22:29:29.141 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:29.153 7f7451dfc700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f74541aa0a0 0x7f74541aa4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:29.141 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:29.153 7f74525fd700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f74541b0db0 0x7f74541b1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:29.141 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:29.153 7f74515fb700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f74541aab20 0x7f74541b0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:29:29.141 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:29.153 7f7451dfc700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f74541aa0a0 0x7f74541aa4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:38088/0 (socket says 172.21.15.102:38088) 2024-04-17T22:29:29.141 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:29.153 7f7451dfc700 1 -- 172.21.15.102:0/3690097136 learned_addr learned my addr 172.21.15.102:0/3690097136 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:29:29.142 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:29.153 7f7451dfc700 1 -- 172.21.15.102:0/3690097136 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f74541aab20 msgr2=0x7f74541b0780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:29.142 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:29.153 7f7451dfc700 1 --2- 172.21.15.102:0/3690097136 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f74541aab20 0x7f74541b0780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:29.142 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:29.153 7f7451dfc700 1 -- 172.21.15.102:0/3690097136 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f74541b0db0 msgr2=0x7f74541b1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:29.142 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:29.153 7f7451dfc700 1 --2- 172.21.15.102:0/3690097136 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f74541b0db0 0x7f74541b1200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:29.142 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:29.153 7f7451dfc700 1 -- 172.21.15.102:0/3690097136 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74540d6590 con 0x7f74541aa0a0 2024-04-17T22:29:29.142 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:29.153 7f74515fb700 1 --2- 172.21.15.102:0/3690097136 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f74541aab20 0x7f74541b0780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:29:29.142 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:29.153 7f7451dfc700 1 --2- 172.21.15.102:0/3690097136 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f74541aa0a0 0x7f74541aa4f0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 rx=0x7f743c0030b0 tx=0x7f743c0068c0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:29:29.142 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:29.153 7f744affd700 1 -- 172.21.15.102:0/3690097136 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f74540d6590 con 0x7f74541aa0a0 2024-04-17T22:29:29.142 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:29.153 7f744affd700 1 -- 172.21.15.102:0/3690097136 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f743c0116c0 con 0x7f74541aa0a0 2024-04-17T22:29:29.142 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:29.153 7f74591d9700 1 -- 172.21.15.102:0/3690097136 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74540cd970 con 0x7f74541aa0a0 2024-04-17T22:29:29.142 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:29.153 7f744affd700 1 -- 172.21.15.102:0/3690097136 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f743c011a90 con 0x7f74541aa0a0 2024-04-17T22:29:29.142 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:29.153 7f74591d9700 1 -- 172.21.15.102:0/3690097136 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74541b8160 con 0x7f74541aa0a0 2024-04-17T22:29:29.142 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:29.154 7f744affd700 1 -- 172.21.15.102:0/3690097136 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7f743c019030 con 0x7f74541aa0a0 2024-04-17T22:29:29.143 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:29.154 7f744affd700 1 -- 172.21.15.102:0/3690097136 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3194+0+0 (secure 0 0 0) 0x7f743c018040 con 0x7f74541aa0a0 2024-04-17T22:29:29.143 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:29.154 7f74591d9700 1 -- 172.21.15.102:0/3690097136 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74540f0800 con 0x7f74541aa0a0 2024-04-17T22:29:29.144 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:29.157 7f744affd700 1 -- 172.21.15.102:0/3690097136 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+110455 (secure 0 0 0) 0x7f743c0114b0 con 0x7f74541aa0a0 2024-04-17T22:29:29.265 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:29.278 7f74591d9700 1 -- 172.21.15.102:0/3690097136 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd blacklist ls"} v 0) v1 -- 0x7f74540d6b10 con 0x7f74541aa0a0 2024-04-17T22:29:29.265 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:29.279 7f744affd700 1 -- 172.21.15.102:0/3690097136 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd blacklist ls"}]=0 listed 0 entries v17) v1 ==== 80+0+0 (secure 0 0 0) 0x7f743c00f300 con 0x7f74541aa0a0 2024-04-17T22:29:29.267 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:29.280 7f74591d9700 1 -- 172.21.15.102:0/3690097136 >> v1:172.21.15.102:6816/42360 conn(0x7f74380145a0 legacy=0x7f7438016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:29.267 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:29.280 7f74591d9700 1 -- 172.21.15.102:0/3690097136 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f74541aa0a0 msgr2=0x7f74541aa4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:29:29.267 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:29.280 7f74591d9700 1 --2- 172.21.15.102:0/3690097136 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f74541aa0a0 0x7f74541aa4f0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 rx=0x7f743c0030b0 tx=0x7f743c0068c0).stop 2024-04-17T22:29:29.267 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:29.280 7f74591d9700 1 -- 172.21.15.102:0/3690097136 shutdown_connections 2024-04-17T22:29:29.267 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:29.280 7f74591d9700 1 --2- 172.21.15.102:0/3690097136 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f74541b0db0 0x7f74541b1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:29.267 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:29.280 7f74591d9700 1 --2- 172.21.15.102:0/3690097136 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f74541aa0a0 0x7f74541aa4f0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:29.267 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:29.280 7f74591d9700 1 --2- 172.21.15.102:0/3690097136 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f74541aab20 0x7f74541b0780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:29:29.267 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:29.280 7f74591d9700 1 -- 172.21.15.102:0/3690097136 >> 172.21.15.102:0/3690097136 conn(0x7f74540e0e70 msgr2=0x7f74540dab90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:29:29.267 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:29.280 7f74591d9700 1 -- 172.21.15.102:0/3690097136 shutdown_connections 2024-04-17T22:29:29.268 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:29:29.280 7f74591d9700 1 -- 172.21.15.102:0/3690097136 wait complete. 2024-04-17T22:29:29.268 INFO:teuthology.orchestra.run.smithi102.stderr:listed 0 entries 2024-04-17T22:29:29.277 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi190.front.sepia.ceph.com... 2024-04-17T22:29:29.278 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T22:29:29.278 DEBUG:teuthology.orchestra.run.smithi190:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-17T22:29:29.322 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi190.stderr:ceph-fuse[40530]: fuse finished with error 0 and tester_r 0 2024-04-17T22:29:29.324 INFO:teuthology.orchestra.run:waiting for 300 2024-04-17T22:29:35.426 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi190.front.sepia.ceph.com 2024-04-17T22:29:35.426 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T22:29:35.426 DEBUG:teuthology.orchestra.run.smithi190:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-17T22:29:35.457 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T22:29:35.457 DEBUG:teuthology.orchestra.run.smithi190:> ip netns list 2024-04-17T22:29:35.519 INFO:teuthology.orchestra.run.smithi190.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-17T22:29:35.519 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T22:29:35.520 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-17T22:29:35.598 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T22:29:35.598 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip link delete ceph-brx 2024-04-17T22:29:35.696 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-04-17T22:29:35.696 INFO:teuthology.run_tasks:Running task print... 2024-04-17T22:29:35.706 INFO:teuthology.task.print:**** done unmount client.0 2024-04-17T22:29:35.706 INFO:teuthology.run_tasks:Running task install.upgrade... 2024-04-17T22:29:35.752 INFO:teuthology.task.install:project ceph config {'client.0': {'branch': 'pacific'}} overrides {'flavor': 'default', 'sha1': '41c500dd60c93c8d83053b72cffc3dd1a952a336'} 2024-04-17T22:29:35.752 INFO:teuthology.task.install:config contains sha1|tag|branch, removing those keys from override 2024-04-17T22:29:35.752 INFO:teuthology.task.install:remote ubuntu@smithi190.front.sepia.ceph.com config {'branch': 'pacific', 'flavor': 'default'} 2024-04-17T22:29:35.752 INFO:teuthology.task.install:extra packages: [] 2024-04-17T22:29:35.758 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-17T22:29:35.758 INFO:teuthology.task.install:Upgrading ceph rpm packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd 2024-04-17T22:29:35.758 DEBUG:teuthology.orchestra.run.smithi190:> rpm -q ceph-common --qf '%{VERSION}-%{RELEASE}' 2024-04-17T22:29:35.782 INFO:teuthology.orchestra.run.smithi190.stdout:14.2.22-0.el8 2024-04-17T22:29:35.782 INFO:teuthology.packaging:The installed version of ceph-common is 14.2.22-0.el8 2024-04-17T22:29:35.783 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=pacific 2024-04-17T22:29:35.979 INFO:teuthology.task.install:Ceph rpm upgrade from 14.2.22-0.el8 to 16.2.15-0 2024-04-17T22:29:35.979 INFO:teuthology.task.install.rpm:Host smithi190 is: centos 8 x86_64 2024-04-17T22:29:35.979 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=pacific 2024-04-17T22:29:36.176 INFO:teuthology.task.install.rpm:Repo base URL: https://chacra.ceph.com/r/ceph/pacific/618f440892089921c3e944a991122ddc44e60516/centos/8/flavors/default/ 2024-04-17T22:29:36.176 DEBUG:teuthology.orchestra.run.smithi190:> sudo rm -f /etc/yum.repos.d/ceph.repo 2024-04-17T22:29:36.238 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://chacra.ceph.com/r/ceph/pacific/618f440892089921c3e944a991122ddc44e60516/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://chacra.ceph.com/r/ceph/pacific/618f440892089921c3e944a991122ddc44e60516/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://chacra.ceph.com/r/ceph/pacific/618f440892089921c3e944a991122ddc44e60516/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-17T22:29:36.238 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-04-17T22:29:36.239 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-17T22:29:36.274 DEBUG:teuthology.orchestra.run.smithi190:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;ref/pacific/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-17T22:29:36.343 DEBUG:teuthology.orchestra.run.smithi190:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-17T22:29:36.410 DEBUG:teuthology.orchestra.run.smithi190:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-17T22:29:36.467 INFO:teuthology.orchestra.run.smithi190.stdout:check_obsoletes = 1 2024-04-17T22:29:36.479 DEBUG:teuthology.orchestra.run.smithi190:> sudo yum clean all 2024-04-17T22:29:36.737 INFO:teuthology.orchestra.run.smithi190.stdout:71 files removed 2024-04-17T22:29:36.766 DEBUG:teuthology.orchestra.run.smithi190:> sudo yum -y upgrade ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd 2024-04-17T22:29:37.266 INFO:teuthology.orchestra.run.smithi190.stdout:ceph packages for x86_64 886 kB/s | 76 kB 00:00 2024-04-17T22:29:37.391 INFO:teuthology.orchestra.run.smithi190.stdout:ceph noarch packages 176 kB/s | 15 kB 00:00 2024-04-17T22:29:37.492 INFO:teuthology.orchestra.run.smithi190.stdout:ceph source packages 24 kB/s | 1.7 kB 00:00 2024-04-17T22:29:38.984 INFO:teuthology.orchestra.run.smithi190.stdout:CentOS Stream 8 - AppStream 19 MB/s | 28 MB 00:01 2024-04-17T22:29:44.843 INFO:teuthology.orchestra.run.smithi190.stdout:CentOS Stream 8 - BaseOS 13 MB/s | 10 MB 00:00 2024-04-17T22:29:47.035 INFO:teuthology.orchestra.run.smithi190.stdout:CentOS Stream 8 - Extras 33 kB/s | 18 kB 00:00 2024-04-17T22:29:47.744 INFO:teuthology.orchestra.run.smithi190.stdout:CentOS Stream 8 - Extras common packages 13 kB/s | 7.7 kB 00:00 2024-04-17T22:29:48.562 INFO:teuthology.orchestra.run.smithi190.stdout:CentOS Stream 8 - PowerTools 6.8 MB/s | 5.0 MB 00:00 2024-04-17T22:29:49.632 INFO:teuthology.orchestra.run.smithi190.stdout:Copr repo for python3-asyncssh owned by ceph 27 kB/s | 3.5 kB 00:00 2024-04-17T22:29:51.327 INFO:teuthology.orchestra.run.smithi190.stdout:Extra Packages for Enterprise Linux 10 MB/s | 16 MB 00:01 2024-04-17T22:29:55.208 INFO:teuthology.orchestra.run.smithi190.stdout:lab-extras 372 kB/s | 24 kB 00:00 2024-04-17T22:29:58.784 INFO:teuthology.orchestra.run.smithi190.stderr:Package cephadm available, but not installed. 2024-04-17T22:29:58.784 INFO:teuthology.orchestra.run.smithi190.stdout:No match for argument: cephadm 2024-04-17T22:29:58.785 INFO:teuthology.orchestra.run.smithi190.stderr:Package ceph-immutable-object-cache available, but not installed. 2024-04-17T22:29:58.785 INFO:teuthology.orchestra.run.smithi190.stdout:No match for argument: ceph-immutable-object-cache 2024-04-17T22:29:58.795 INFO:teuthology.orchestra.run.smithi190.stderr:Package ceph-mgr-cephadm available, but not installed. 2024-04-17T22:29:58.795 INFO:teuthology.orchestra.run.smithi190.stdout:No match for argument: ceph-mgr-cephadm 2024-04-17T22:29:58.801 INFO:teuthology.orchestra.run.smithi190.stdout:No match for argument: ceph-volume 2024-04-17T22:29:58.904 INFO:teuthology.orchestra.run.smithi190.stdout:Dependencies resolved. 2024-04-17T22:29:58.909 INFO:teuthology.orchestra.run.smithi190.stdout:================================================================================ 2024-04-17T22:29:58.910 INFO:teuthology.orchestra.run.smithi190.stdout: Package Arch Version Repository Size 2024-04-17T22:29:58.910 INFO:teuthology.orchestra.run.smithi190.stdout:================================================================================ 2024-04-17T22:29:58.910 INFO:teuthology.orchestra.run.smithi190.stdout:Upgrading: 2024-04-17T22:29:58.910 INFO:teuthology.orchestra.run.smithi190.stdout: ceph x86_64 2:16.2.15-0.el8 ceph 6.3 k 2024-04-17T22:29:58.910 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-base x86_64 2:16.2.15-0.el8 ceph 6.1 M 2024-04-17T22:29:58.910 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-common x86_64 2:16.2.15-0.el8 ceph 24 M 2024-04-17T22:29:58.910 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-fuse x86_64 2:16.2.15-0.el8 ceph 911 k 2024-04-17T22:29:58.910 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-grafana-dashboards noarch 2:16.2.15-0.el8 ceph-noarch 24 k 2024-04-17T22:29:58.910 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mds x86_64 2:16.2.15-0.el8 ceph 2.4 M 2024-04-17T22:29:58.910 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr x86_64 2:16.2.15-0.el8 ceph 1.6 M 2024-04-17T22:29:58.911 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr-dashboard noarch 2:16.2.15-0.el8 ceph-noarch 3.1 M 2024-04-17T22:29:58.911 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr-diskprediction-local noarch 2:16.2.15-0.el8 ceph-noarch 8.6 M 2024-04-17T22:29:58.911 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr-rook noarch 2:16.2.15-0.el8 ceph-noarch 27 k 2024-04-17T22:29:58.911 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mon x86_64 2:16.2.15-0.el8 ceph 5.2 M 2024-04-17T22:29:58.911 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-osd x86_64 2:16.2.15-0.el8 ceph 19 M 2024-04-17T22:29:58.911 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-radosgw x86_64 2:16.2.15-0.el8 ceph 12 M 2024-04-17T22:29:58.911 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-selinux x86_64 2:16.2.15-0.el8 ceph 26 k 2024-04-17T22:29:58.911 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-test x86_64 2:16.2.15-0.el8 ceph 52 M 2024-04-17T22:29:58.911 INFO:teuthology.orchestra.run.smithi190.stdout: libcephfs-devel x86_64 2:16.2.15-0.el8 ceph 25 k 2024-04-17T22:29:58.911 INFO:teuthology.orchestra.run.smithi190.stdout: libcephfs2 x86_64 2:16.2.15-0.el8 ceph 825 k 2024-04-17T22:29:58.911 INFO:teuthology.orchestra.run.smithi190.stdout: librados-devel x86_64 2:16.2.15-0.el8 ceph 131 k 2024-04-17T22:29:58.911 INFO:teuthology.orchestra.run.smithi190.stdout: librados2 x86_64 2:16.2.15-0.el8 ceph 3.7 M 2024-04-17T22:29:58.911 INFO:teuthology.orchestra.run.smithi190.stdout: libradosstriper1 x86_64 2:16.2.15-0.el8 ceph 494 k 2024-04-17T22:29:58.912 INFO:teuthology.orchestra.run.smithi190.stdout: librbd1 x86_64 2:16.2.15-0.el8 ceph 4.2 M 2024-04-17T22:29:58.912 INFO:teuthology.orchestra.run.smithi190.stdout: librgw2 x86_64 2:16.2.15-0.el8 ceph 3.8 M 2024-04-17T22:29:58.912 INFO:teuthology.orchestra.run.smithi190.stdout: python3-ceph-argparse x86_64 2:16.2.15-0.el8 ceph 45 k 2024-04-17T22:29:58.912 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cephfs x86_64 2:16.2.15-0.el8 ceph 215 k 2024-04-17T22:29:58.912 INFO:teuthology.orchestra.run.smithi190.stdout: python3-rados x86_64 2:16.2.15-0.el8 ceph 389 k 2024-04-17T22:29:58.912 INFO:teuthology.orchestra.run.smithi190.stdout: python3-rbd x86_64 2:16.2.15-0.el8 ceph 369 k 2024-04-17T22:29:58.912 INFO:teuthology.orchestra.run.smithi190.stdout: python3-rgw x86_64 2:16.2.15-0.el8 ceph 114 k 2024-04-17T22:29:58.912 INFO:teuthology.orchestra.run.smithi190.stdout: rbd-fuse x86_64 2:16.2.15-0.el8 ceph 88 k 2024-04-17T22:29:58.912 INFO:teuthology.orchestra.run.smithi190.stdout: rbd-mirror x86_64 2:16.2.15-0.el8 ceph 3.7 M 2024-04-17T22:29:58.912 INFO:teuthology.orchestra.run.smithi190.stdout: rbd-nbd x86_64 2:16.2.15-0.el8 ceph 175 k 2024-04-17T22:29:58.912 INFO:teuthology.orchestra.run.smithi190.stdout:Installing dependencies: 2024-04-17T22:29:58.912 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr-modules-core noarch 2:16.2.15-0.el8 ceph-noarch 245 k 2024-04-17T22:29:58.912 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-prometheus-alerts noarch 2:16.2.15-0.el8 ceph-noarch 15 k 2024-04-17T22:29:58.912 INFO:teuthology.orchestra.run.smithi190.stdout: libcephsqlite x86_64 2:16.2.15-0.el8 ceph 174 k 2024-04-17T22:29:58.913 INFO:teuthology.orchestra.run.smithi190.stdout: python3-ceph-common x86_64 2:16.2.15-0.el8 ceph 99 k 2024-04-17T22:29:58.913 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jsonpatch noarch 1.21-2.el8 appstream 27 k 2024-04-17T22:29:58.913 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jsonpointer noarch 1.10-11.el8 appstream 20 k 2024-04-17T22:29:58.913 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-04-17T22:29:58.913 INFO:teuthology.orchestra.run.smithi190.stdout:Transaction Summary 2024-04-17T22:29:58.913 INFO:teuthology.orchestra.run.smithi190.stdout:================================================================================ 2024-04-17T22:29:58.913 INFO:teuthology.orchestra.run.smithi190.stdout:Install 6 Packages 2024-04-17T22:29:58.913 INFO:teuthology.orchestra.run.smithi190.stdout:Upgrade 30 Packages 2024-04-17T22:29:58.913 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-04-17T22:29:58.914 INFO:teuthology.orchestra.run.smithi190.stdout:Total download size: 154 M 2024-04-17T22:29:58.914 INFO:teuthology.orchestra.run.smithi190.stdout:Downloading Packages: 2024-04-17T22:29:59.110 INFO:teuthology.orchestra.run.smithi190.stdout:(1/36): libcephsqlite-16.2.15-0.el8.x86_64.rpm 2.9 MB/s | 174 kB 00:00 2024-04-17T22:29:59.135 INFO:teuthology.orchestra.run.smithi190.stdout:(2/36): python3-ceph-common-16.2.15-0.el8.x86_6 1.1 MB/s | 99 kB 00:00 2024-04-17T22:29:59.160 INFO:teuthology.orchestra.run.smithi190.stdout:(3/36): ceph-prometheus-alerts-16.2.15-0.el8.no 295 kB/s | 15 kB 00:00 2024-04-17T22:29:59.186 INFO:teuthology.orchestra.run.smithi190.stdout:(4/36): ceph-mgr-modules-core-16.2.15-0.el8.noa 1.8 MB/s | 245 kB 00:00 2024-04-17T22:29:59.211 INFO:teuthology.orchestra.run.smithi190.stdout:(5/36): ceph-16.2.15-0.el8.x86_64.rpm 255 kB/s | 6.3 kB 00:00 2024-04-17T22:29:59.303 INFO:teuthology.orchestra.run.smithi190.stdout:(6/36): ceph-base-16.2.15-0.el8.x86_64.rpm 67 MB/s | 6.1 MB 00:00 2024-04-17T22:29:59.662 INFO:teuthology.orchestra.run.smithi190.stdout:(7/36): ceph-common-16.2.15-0.el8.x86_64.rpm 68 MB/s | 24 MB 00:00 2024-04-17T22:29:59.687 INFO:teuthology.orchestra.run.smithi190.stdout:(8/36): python3-jsonpatch-1.21-2.el8.noarch.rpm 48 kB/s | 27 kB 00:00 2024-04-17T22:29:59.712 INFO:teuthology.orchestra.run.smithi190.stdout:(9/36): python3-jsonpointer-1.10-11.el8.noarch. 36 kB/s | 20 kB 00:00 2024-04-17T22:29:59.745 INFO:teuthology.orchestra.run.smithi190.stdout:(10/36): ceph-fuse-16.2.15-0.el8.x86_64.rpm 11 MB/s | 911 kB 00:00 2024-04-17T22:29:59.796 INFO:teuthology.orchestra.run.smithi190.stdout:(11/36): ceph-mgr-16.2.15-0.el8.x86_64.rpm 19 MB/s | 1.6 MB 00:00 2024-04-17T22:29:59.846 INFO:teuthology.orchestra.run.smithi190.stdout:(12/36): ceph-mds-16.2.15-0.el8.x86_64.rpm 18 MB/s | 2.4 MB 00:00 2024-04-17T22:29:59.939 INFO:teuthology.orchestra.run.smithi190.stdout:(13/36): ceph-mon-16.2.15-0.el8.x86_64.rpm 27 MB/s | 5.2 MB 00:00 2024-04-17T22:29:59.964 INFO:teuthology.orchestra.run.smithi190.stdout:(14/36): ceph-selinux-16.2.15-0.el8.x86_64.rpm 1.0 MB/s | 26 kB 00:00 2024-04-17T22:30:00.172 INFO:teuthology.orchestra.run.smithi190.stdout:(15/36): ceph-radosgw-16.2.15-0.el8.x86_64.rpm 36 MB/s | 12 MB 00:00 2024-04-17T22:30:00.197 INFO:teuthology.orchestra.run.smithi190.stdout:(16/36): libcephfs-devel-16.2.15-0.el8.x86_64.r 1.0 MB/s | 25 kB 00:00 2024-04-17T22:30:00.259 INFO:teuthology.orchestra.run.smithi190.stdout:(17/36): libcephfs2-16.2.15-0.el8.x86_64.rpm 13 MB/s | 825 kB 00:00 2024-04-17T22:30:00.290 INFO:teuthology.orchestra.run.smithi190.stdout:(18/36): librados-devel-16.2.15-0.el8.x86_64.rp 4.2 MB/s | 131 kB 00:00 2024-04-17T22:30:00.490 INFO:teuthology.orchestra.run.smithi190.stdout:(19/36): ceph-osd-16.2.15-0.el8.x86_64.rpm 27 MB/s | 19 MB 00:00 2024-04-17T22:30:00.524 INFO:teuthology.orchestra.run.smithi190.stdout:(20/36): libradosstriper1-16.2.15-0.el8.x86_64. 15 MB/s | 494 kB 00:00 2024-04-17T22:30:00.591 INFO:teuthology.orchestra.run.smithi190.stdout:(21/36): librados2-16.2.15-0.el8.x86_64.rpm 12 MB/s | 3.7 MB 00:00 2024-04-17T22:30:00.683 INFO:teuthology.orchestra.run.smithi190.stdout:(22/36): librbd1-16.2.15-0.el8.x86_64.rpm 26 MB/s | 4.2 MB 00:00 2024-04-17T22:30:00.708 INFO:teuthology.orchestra.run.smithi190.stdout:(23/36): python3-ceph-argparse-16.2.15-0.el8.x8 1.8 MB/s | 45 kB 00:00 2024-04-17T22:30:00.733 INFO:teuthology.orchestra.run.smithi190.stdout:(24/36): python3-cephfs-16.2.15-0.el8.x86_64.rp 8.4 MB/s | 215 kB 00:00 2024-04-17T22:30:00.775 INFO:teuthology.orchestra.run.smithi190.stdout:(25/36): python3-rados-16.2.15-0.el8.x86_64.rpm 9.1 MB/s | 389 kB 00:00 2024-04-17T22:30:00.834 INFO:teuthology.orchestra.run.smithi190.stdout:(26/36): librgw2-16.2.15-0.el8.x86_64.rpm 16 MB/s | 3.8 MB 00:00 2024-04-17T22:30:00.859 INFO:teuthology.orchestra.run.smithi190.stdout:(27/36): python3-rgw-16.2.15-0.el8.x86_64.rpm 4.5 MB/s | 114 kB 00:00 2024-04-17T22:30:00.910 INFO:teuthology.orchestra.run.smithi190.stdout:(28/36): python3-rbd-16.2.15-0.el8.x86_64.rpm 2.7 MB/s | 369 kB 00:00 2024-04-17T22:30:00.935 INFO:teuthology.orchestra.run.smithi190.stdout:(29/36): rbd-fuse-16.2.15-0.el8.x86_64.rpm 1.1 MB/s | 88 kB 00:00 2024-04-17T22:30:00.952 INFO:teuthology.orchestra.run.smithi190.stdout:(30/36): rbd-nbd-16.2.15-0.el8.x86_64.rpm 10 MB/s | 175 kB 00:00 2024-04-17T22:30:00.977 INFO:teuthology.orchestra.run.smithi190.stdout:(31/36): ceph-grafana-dashboards-16.2.15-0.el8. 979 kB/s | 24 kB 00:00 2024-04-17T22:30:01.061 INFO:teuthology.orchestra.run.smithi190.stdout:(32/36): rbd-mirror-16.2.15-0.el8.x86_64.rpm 24 MB/s | 3.7 MB 00:00 2024-04-17T22:30:01.119 INFO:teuthology.orchestra.run.smithi190.stdout:(33/36): ceph-mgr-dashboard-16.2.15-0.el8.noarc 22 MB/s | 3.1 MB 00:00 2024-04-17T22:30:01.153 INFO:teuthology.orchestra.run.smithi190.stdout:(34/36): ceph-mgr-rook-16.2.15-0.el8.noarch.rpm 822 kB/s | 27 kB 00:00 2024-04-17T22:30:01.287 INFO:teuthology.orchestra.run.smithi190.stdout:(35/36): ceph-mgr-diskprediction-local-16.2.15- 38 MB/s | 8.6 MB 00:00 2024-04-17T22:30:01.795 INFO:teuthology.orchestra.run.smithi190.stdout:(36/36): ceph-test-16.2.15-0.el8.x86_64.rpm 28 MB/s | 52 MB 00:01 2024-04-17T22:30:01.799 INFO:teuthology.orchestra.run.smithi190.stdout:-------------------------------------------------------------------------------- 2024-04-17T22:30:01.799 INFO:teuthology.orchestra.run.smithi190.stdout:Total 53 MB/s | 154 MB 00:02 2024-04-17T22:30:01.819 INFO:teuthology.orchestra.run.smithi190.stdout:Running transaction check 2024-04-17T22:30:01.893 INFO:teuthology.orchestra.run.smithi190.stdout:Transaction check succeeded. 2024-04-17T22:30:01.893 INFO:teuthology.orchestra.run.smithi190.stdout:Running transaction test 2024-04-17T22:30:02.597 INFO:teuthology.orchestra.run.smithi190.stdout:Transaction test succeeded. 2024-04-17T22:30:02.600 INFO:teuthology.orchestra.run.smithi190.stdout:Running transaction 2024-04-17T22:30:04.787 INFO:teuthology.orchestra.run.smithi190.stdout: Preparing : 1/1 2024-04-17T22:30:05.128 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: librados2-2:16.2.15-0.el8.x86_64 1/1 2024-04-17T22:30:05.309 INFO:teuthology.orchestra.run.smithi190.stdout: Upgrading : librados2-2:16.2.15-0.el8.x86_64 1/66 2024-04-17T22:30:05.714 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: librados2-2:16.2.15-0.el8.x86_64 1/66 2024-04-17T22:30:05.892 INFO:teuthology.orchestra.run.smithi190.stdout: Upgrading : librbd1-2:16.2.15-0.el8.x86_64 2/66 2024-04-17T22:30:06.015 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: librbd1-2:16.2.15-0.el8.x86_64 2/66 2024-04-17T22:30:06.217 INFO:teuthology.orchestra.run.smithi190.stdout: Upgrading : libcephfs2-2:16.2.15-0.el8.x86_64 3/66 2024-04-17T22:30:06.634 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: libcephfs2-2:16.2.15-0.el8.x86_64 3/66 2024-04-17T22:30:06.858 INFO:teuthology.orchestra.run.smithi190.stdout: Upgrading : librgw2-2:16.2.15-0.el8.x86_64 4/66 2024-04-17T22:30:06.955 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: librgw2-2:16.2.15-0.el8.x86_64 4/66 2024-04-17T22:30:07.156 INFO:teuthology.orchestra.run.smithi190.stdout: Upgrading : python3-rados-2:16.2.15-0.el8.x86_64 5/66 2024-04-17T22:30:07.433 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libcephsqlite-2:16.2.15-0.el8.x86_64 6/66 2024-04-17T22:30:07.538 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: libcephsqlite-2:16.2.15-0.el8.x86_64 6/66 2024-04-17T22:30:07.708 INFO:teuthology.orchestra.run.smithi190.stdout: Upgrading : libradosstriper1-2:16.2.15-0.el8.x86_64 7/66 2024-04-17T22:30:07.778 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: libradosstriper1-2:16.2.15-0.el8.x86_64 7/66 2024-04-17T22:30:07.936 INFO:teuthology.orchestra.run.smithi190.stdout: Upgrading : python3-ceph-argparse-2:16.2.15-0.el8.x86_64 8/66 2024-04-17T22:30:08.180 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-ceph-common-2:16.2.15-0.el8.x86_64 9/66 2024-04-17T22:30:08.393 INFO:teuthology.orchestra.run.smithi190.stdout: Upgrading : python3-cephfs-2:16.2.15-0.el8.x86_64 10/66 2024-04-17T22:30:08.595 INFO:teuthology.orchestra.run.smithi190.stdout: Upgrading : python3-rbd-2:16.2.15-0.el8.x86_64 11/66 2024-04-17T22:30:08.792 INFO:teuthology.orchestra.run.smithi190.stdout: Upgrading : python3-rgw-2:16.2.15-0.el8.x86_64 12/66 2024-04-17T22:30:10.836 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-common-2:16.2.15-0.el8.x86_64 13/66 2024-04-17T22:30:11.083 INFO:teuthology.orchestra.run.smithi190.stdout: Upgrading : ceph-common-2:16.2.15-0.el8.x86_64 13/66 2024-04-17T22:30:11.872 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-common-2:16.2.15-0.el8.x86_64 13/66 2024-04-17T22:30:12.120 INFO:teuthology.orchestra.run.smithi190.stdout: Upgrading : ceph-base-2:16.2.15-0.el8.x86_64 14/66 2024-04-17T22:30:12.170 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-base-2:16.2.15-0.el8.x86_64 14/66 2024-04-17T22:30:12.299 INFO:teuthology.orchestra.run.smithi190.stdout: Upgrading : ceph-selinux-2:16.2.15-0.el8.x86_64 15/66 2024-04-17T22:30:30.771 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-selinux-2:16.2.15-0.el8.x86_64 15/66 2024-04-17T22:30:30.964 INFO:teuthology.orchestra.run.smithi190.stdout: Upgrading : ceph-mds-2:16.2.15-0.el8.x86_64 16/66 2024-04-17T22:30:31.448 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-mds-2:16.2.15-0.el8.x86_64 16/66 2024-04-17T22:30:31.655 INFO:teuthology.orchestra.run.smithi190.stdout: Upgrading : ceph-mon-2:16.2.15-0.el8.x86_64 17/66 2024-04-17T22:30:33.345 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-mon-2:16.2.15-0.el8.x86_64 17/66 2024-04-17T22:30:33.529 INFO:teuthology.orchestra.run.smithi190.stdout: Upgrading : ceph-osd-2:16.2.15-0.el8.x86_64 18/66 2024-04-17T22:30:33.554 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-osd-2:16.2.15-0.el8.x86_64 18/66 2024-04-17T22:30:33.762 INFO:teuthology.orchestra.run.smithi190.stdout: Upgrading : librados-devel-2:16.2.15-0.el8.x86_64 19/66 2024-04-17T22:30:33.911 INFO:teuthology.orchestra.run.smithi190.stdout: Upgrading : ceph-grafana-dashboards-2:16.2.15-0.el8.noarch 20/66 2024-04-17T22:30:34.120 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 21/66 2024-04-17T22:30:34.314 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 22/66 2024-04-17T22:30:35.380 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-prometheus-alerts-2:16.2.15-0.el8.noarch 23/66 2024-04-17T22:30:35.749 INFO:teuthology.orchestra.run.smithi190.stdout: Upgrading : ceph-mgr-dashboard-2:16.2.15-0.el8.noarch 24/66 2024-04-17T22:30:36.777 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-mgr-dashboard-2:16.2.15-0.el8.noarch 24/66 2024-04-17T22:30:36.970 INFO:teuthology.orchestra.run.smithi190.stdout: Upgrading : ceph-mgr-diskprediction-local-2:16.2.15-0.el8.noar 25/66 2024-04-17T22:30:37.056 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:16.2.15-0.el8.noar 25/66 2024-04-17T22:30:37.450 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-mgr-modules-core-2:16.2.15-0.el8.noarch 26/66 2024-04-17T22:30:37.683 INFO:teuthology.orchestra.run.smithi190.stdout: Upgrading : ceph-mgr-2:16.2.15-0.el8.x86_64 27/66 2024-04-17T22:30:37.696 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-mgr-2:16.2.15-0.el8.x86_64 27/66 2024-04-17T22:30:37.870 INFO:teuthology.orchestra.run.smithi190.stdout: Upgrading : ceph-mgr-rook-2:16.2.15-0.el8.noarch 28/66 2024-04-17T22:30:37.895 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-mgr-rook-2:16.2.15-0.el8.noarch 28/66 2024-04-17T22:30:37.982 INFO:teuthology.orchestra.run.smithi190.stdout: Upgrading : ceph-2:16.2.15-0.el8.x86_64 29/66 2024-04-17T22:30:39.158 INFO:teuthology.orchestra.run.smithi190.stdout: Upgrading : libcephfs-devel-2:16.2.15-0.el8.x86_64 30/66 2024-04-17T22:30:39.390 INFO:teuthology.orchestra.run.smithi190.stdout: Upgrading : ceph-radosgw-2:16.2.15-0.el8.x86_64 31/66 2024-04-17T22:30:39.845 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-radosgw-2:16.2.15-0.el8.x86_64 31/66 2024-04-17T22:30:40.057 INFO:teuthology.orchestra.run.smithi190.stdout: Upgrading : rbd-mirror-2:16.2.15-0.el8.x86_64 32/66 2024-04-17T22:30:44.875 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: rbd-mirror-2:16.2.15-0.el8.x86_64 32/66 2024-04-17T22:30:45.127 INFO:teuthology.orchestra.run.smithi190.stdout: Upgrading : ceph-test-2:16.2.15-0.el8.x86_64 33/66 2024-04-17T22:30:45.360 INFO:teuthology.orchestra.run.smithi190.stdout: Upgrading : rbd-fuse-2:16.2.15-0.el8.x86_64 34/66 2024-04-17T22:30:45.611 INFO:teuthology.orchestra.run.smithi190.stdout: Upgrading : rbd-nbd-2:16.2.15-0.el8.x86_64 35/66 2024-04-17T22:30:45.887 INFO:teuthology.orchestra.run.smithi190.stdout: Upgrading : ceph-fuse-2:16.2.15-0.el8.x86_64 36/66 2024-04-17T22:30:45.983 INFO:teuthology.orchestra.run.smithi190.stdout: Cleanup : ceph-2:14.2.22-0.el8.x86_64 37/66 2024-04-17T22:30:46.195 INFO:teuthology.orchestra.run.smithi190.stdout: Cleanup : ceph-test-2:14.2.22-0.el8.x86_64 38/66 2024-04-17T22:30:46.201 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-radosgw-2:14.2.22-0.el8.x86_64 39/66 2024-04-17T22:30:46.201 INFO:teuthology.orchestra.run.smithi190.stdout: Cleanup : ceph-radosgw-2:14.2.22-0.el8.x86_64 39/66 2024-04-17T22:30:46.399 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-radosgw-2:14.2.22-0.el8.x86_64 39/66 2024-04-17T22:30:46.545 INFO:teuthology.orchestra.run.smithi190.stdout: Cleanup : libcephfs-devel-2:14.2.22-0.el8.x86_64 40/66 2024-04-17T22:30:46.711 INFO:teuthology.orchestra.run.smithi190.stdout: Cleanup : rbd-nbd-2:14.2.22-0.el8.x86_64 41/66 2024-04-17T22:30:46.718 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: rbd-mirror-2:14.2.22-0.el8.x86_64 42/66 2024-04-17T22:30:46.718 INFO:teuthology.orchestra.run.smithi190.stdout: Cleanup : rbd-mirror-2:14.2.22-0.el8.x86_64 42/66 2024-04-17T22:30:46.871 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: rbd-mirror-2:14.2.22-0.el8.x86_64 42/66 2024-04-17T22:30:46.881 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-mgr-2:14.2.22-0.el8.x86_64 43/66 2024-04-17T22:30:46.882 INFO:teuthology.orchestra.run.smithi190.stdout: Cleanup : ceph-mgr-2:14.2.22-0.el8.x86_64 43/66 2024-04-17T22:30:47.107 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-mgr-2:14.2.22-0.el8.x86_64 43/66 2024-04-17T22:30:47.287 INFO:teuthology.orchestra.run.smithi190.stdout: Cleanup : rbd-fuse-2:14.2.22-0.el8.x86_64 44/66 2024-04-17T22:30:47.445 INFO:teuthology.orchestra.run.smithi190.stdout: Cleanup : librados-devel-2:14.2.22-0.el8.x86_64 45/66 2024-04-17T22:30:47.451 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-mds-2:14.2.22-0.el8.x86_64 46/66 2024-04-17T22:30:47.452 INFO:teuthology.orchestra.run.smithi190.stdout: Cleanup : ceph-mds-2:14.2.22-0.el8.x86_64 46/66 2024-04-17T22:30:47.604 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-mds-2:14.2.22-0.el8.x86_64 46/66 2024-04-17T22:30:47.610 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-mon-2:14.2.22-0.el8.x86_64 47/66 2024-04-17T22:30:47.611 INFO:teuthology.orchestra.run.smithi190.stdout: Cleanup : ceph-mon-2:14.2.22-0.el8.x86_64 47/66 2024-04-17T22:30:47.754 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-mon-2:14.2.22-0.el8.x86_64 47/66 2024-04-17T22:30:47.761 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-osd-2:14.2.22-0.el8.x86_64 48/66 2024-04-17T22:30:47.761 INFO:teuthology.orchestra.run.smithi190.stdout: Cleanup : ceph-osd-2:14.2.22-0.el8.x86_64 48/66 2024-04-17T22:30:48.000 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-osd-2:14.2.22-0.el8.x86_64 48/66 2024-04-17T22:30:48.000 INFO:teuthology.orchestra.run.smithi190.stdout: Cleanup : ceph-mgr-dashboard-2:14.2.22-0.el8.noarch 49/66 2024-04-17T22:30:48.215 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-mgr-dashboard-2:14.2.22-0.el8.noarch 49/66 2024-04-17T22:30:48.216 INFO:teuthology.orchestra.run.smithi190.stdout: Cleanup : ceph-selinux-2:14.2.22-0.el8.x86_64 50/66 2024-04-17T22:30:48.389 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-selinux-2:14.2.22-0.el8.x86_64 50/66 2024-04-17T22:30:48.406 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-base-2:14.2.22-0.el8.x86_64 51/66 2024-04-17T22:30:48.407 INFO:teuthology.orchestra.run.smithi190.stdout: Cleanup : ceph-base-2:14.2.22-0.el8.x86_64 51/66 2024-04-17T22:30:48.702 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-base-2:14.2.22-0.el8.x86_64 51/66 2024-04-17T22:30:48.702 INFO:teuthology.orchestra.run.smithi190.stdout: Cleanup : ceph-common-2:14.2.22-0.el8.x86_64 52/66 2024-04-17T22:30:48.887 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-common-2:14.2.22-0.el8.x86_64 52/66 2024-04-17T22:30:48.887 INFO:teuthology.orchestra.run.smithi190.stdout: Cleanup : libradosstriper1-2:14.2.22-0.el8.x86_64 53/66 2024-04-17T22:30:49.128 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: libradosstriper1-2:14.2.22-0.el8.x86_64 53/66 2024-04-17T22:30:49.295 INFO:teuthology.orchestra.run.smithi190.stdout: Cleanup : python3-cephfs-2:14.2.22-0.el8.x86_64 54/66 2024-04-17T22:30:49.470 INFO:teuthology.orchestra.run.smithi190.stdout: Cleanup : python3-rbd-2:14.2.22-0.el8.x86_64 55/66 2024-04-17T22:30:49.470 INFO:teuthology.orchestra.run.smithi190.stdout: Cleanup : librbd1-2:14.2.22-0.el8.x86_64 56/66 2024-04-17T22:30:49.678 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: librbd1-2:14.2.22-0.el8.x86_64 56/66 2024-04-17T22:30:49.896 INFO:teuthology.orchestra.run.smithi190.stdout: Cleanup : python3-rgw-2:14.2.22-0.el8.x86_64 57/66 2024-04-17T22:30:49.896 INFO:teuthology.orchestra.run.smithi190.stdout: Cleanup : librgw2-2:14.2.22-0.el8.x86_64 58/66 2024-04-17T22:30:50.104 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: librgw2-2:14.2.22-0.el8.x86_64 58/66 2024-04-17T22:30:50.278 INFO:teuthology.orchestra.run.smithi190.stdout: Cleanup : python3-rados-2:14.2.22-0.el8.x86_64 59/66 2024-04-17T22:30:50.278 INFO:teuthology.orchestra.run.smithi190.stdout: Cleanup : libcephfs2-2:14.2.22-0.el8.x86_64 60/66 2024-04-17T22:30:50.545 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: libcephfs2-2:14.2.22-0.el8.x86_64 60/66 2024-04-17T22:30:50.694 INFO:teuthology.orchestra.run.smithi190.stdout: Cleanup : ceph-fuse-2:14.2.22-0.el8.x86_64 61/66 2024-04-17T22:30:50.811 INFO:teuthology.orchestra.run.smithi190.stdout: Cleanup : python3-ceph-argparse-2:14.2.22-0.el8.x86_64 62/66 2024-04-17T22:30:50.968 INFO:teuthology.orchestra.run.smithi190.stdout: Cleanup : ceph-grafana-dashboards-2:14.2.22-0.el8.noarch 63/66 2024-04-17T22:30:50.968 INFO:teuthology.orchestra.run.smithi190.stdout: Cleanup : ceph-mgr-diskprediction-local-2:14.2.22-0.el8.noar 64/66 2024-04-17T22:30:51.153 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:14.2.22-0.el8.noar 64/66 2024-04-17T22:30:51.153 INFO:teuthology.orchestra.run.smithi190.stdout: Cleanup : ceph-mgr-rook-2:14.2.22-0.el8.noarch 65/66 2024-04-17T22:30:51.296 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-mgr-rook-2:14.2.22-0.el8.noarch 65/66 2024-04-17T22:30:51.297 INFO:teuthology.orchestra.run.smithi190.stdout: Cleanup : librados2-2:14.2.22-0.el8.x86_64 66/66 2024-04-17T22:30:51.985 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: librados2-2:14.2.22-0.el8.x86_64 66/66 2024-04-17T22:30:51.985 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libcephsqlite-2:16.2.15-0.el8.x86_64 1/66 2024-04-17T22:30:51.985 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-ceph-common-2:16.2.15-0.el8.x86_64 2/66 2024-04-17T22:30:51.985 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-mgr-modules-core-2:16.2.15-0.el8.noarch 3/66 2024-04-17T22:30:51.985 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-prometheus-alerts-2:16.2.15-0.el8.noarch 4/66 2024-04-17T22:30:51.985 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 5/66 2024-04-17T22:30:51.985 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 6/66 2024-04-17T22:30:51.985 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-2:16.2.15-0.el8.x86_64 7/66 2024-04-17T22:30:51.985 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-2:14.2.22-0.el8.x86_64 8/66 2024-04-17T22:30:51.985 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-base-2:16.2.15-0.el8.x86_64 9/66 2024-04-17T22:30:51.985 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-base-2:14.2.22-0.el8.x86_64 10/66 2024-04-17T22:30:51.986 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-common-2:16.2.15-0.el8.x86_64 11/66 2024-04-17T22:30:51.986 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-common-2:14.2.22-0.el8.x86_64 12/66 2024-04-17T22:30:51.986 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-fuse-2:16.2.15-0.el8.x86_64 13/66 2024-04-17T22:30:51.986 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-fuse-2:14.2.22-0.el8.x86_64 14/66 2024-04-17T22:30:51.986 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-mds-2:16.2.15-0.el8.x86_64 15/66 2024-04-17T22:30:51.986 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-mds-2:14.2.22-0.el8.x86_64 16/66 2024-04-17T22:30:51.986 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-mgr-2:16.2.15-0.el8.x86_64 17/66 2024-04-17T22:30:51.986 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-mgr-2:14.2.22-0.el8.x86_64 18/66 2024-04-17T22:30:51.986 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-mon-2:16.2.15-0.el8.x86_64 19/66 2024-04-17T22:30:51.986 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-mon-2:14.2.22-0.el8.x86_64 20/66 2024-04-17T22:30:51.986 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-osd-2:16.2.15-0.el8.x86_64 21/66 2024-04-17T22:30:51.986 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-osd-2:14.2.22-0.el8.x86_64 22/66 2024-04-17T22:30:51.987 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-radosgw-2:16.2.15-0.el8.x86_64 23/66 2024-04-17T22:30:51.987 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-radosgw-2:14.2.22-0.el8.x86_64 24/66 2024-04-17T22:30:51.987 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-selinux-2:16.2.15-0.el8.x86_64 25/66 2024-04-17T22:30:51.987 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-selinux-2:14.2.22-0.el8.x86_64 26/66 2024-04-17T22:30:51.987 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-test-2:16.2.15-0.el8.x86_64 27/66 2024-04-17T22:30:51.987 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-test-2:14.2.22-0.el8.x86_64 28/66 2024-04-17T22:30:51.991 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libcephfs-devel-2:16.2.15-0.el8.x86_64 29/66 2024-04-17T22:30:51.991 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libcephfs-devel-2:14.2.22-0.el8.x86_64 30/66 2024-04-17T22:30:51.991 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libcephfs2-2:16.2.15-0.el8.x86_64 31/66 2024-04-17T22:30:51.991 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libcephfs2-2:14.2.22-0.el8.x86_64 32/66 2024-04-17T22:30:51.991 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : librados-devel-2:16.2.15-0.el8.x86_64 33/66 2024-04-17T22:30:51.991 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : librados-devel-2:14.2.22-0.el8.x86_64 34/66 2024-04-17T22:30:51.991 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : librados2-2:16.2.15-0.el8.x86_64 35/66 2024-04-17T22:30:51.991 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : librados2-2:14.2.22-0.el8.x86_64 36/66 2024-04-17T22:30:51.991 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libradosstriper1-2:16.2.15-0.el8.x86_64 37/66 2024-04-17T22:30:51.991 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libradosstriper1-2:14.2.22-0.el8.x86_64 38/66 2024-04-17T22:30:51.991 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : librbd1-2:16.2.15-0.el8.x86_64 39/66 2024-04-17T22:30:51.991 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : librbd1-2:14.2.22-0.el8.x86_64 40/66 2024-04-17T22:30:51.992 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : librgw2-2:16.2.15-0.el8.x86_64 41/66 2024-04-17T22:30:51.992 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : librgw2-2:14.2.22-0.el8.x86_64 42/66 2024-04-17T22:30:51.992 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-ceph-argparse-2:16.2.15-0.el8.x86_64 43/66 2024-04-17T22:30:51.992 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-ceph-argparse-2:14.2.22-0.el8.x86_64 44/66 2024-04-17T22:30:51.992 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-cephfs-2:16.2.15-0.el8.x86_64 45/66 2024-04-17T22:30:51.992 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-cephfs-2:14.2.22-0.el8.x86_64 46/66 2024-04-17T22:30:51.992 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-rados-2:16.2.15-0.el8.x86_64 47/66 2024-04-17T22:30:51.992 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-rados-2:14.2.22-0.el8.x86_64 48/66 2024-04-17T22:30:51.992 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-rbd-2:16.2.15-0.el8.x86_64 49/66 2024-04-17T22:30:51.992 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-rbd-2:14.2.22-0.el8.x86_64 50/66 2024-04-17T22:30:51.992 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-rgw-2:16.2.15-0.el8.x86_64 51/66 2024-04-17T22:30:51.992 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-rgw-2:14.2.22-0.el8.x86_64 52/66 2024-04-17T22:30:51.992 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : rbd-fuse-2:16.2.15-0.el8.x86_64 53/66 2024-04-17T22:30:51.992 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : rbd-fuse-2:14.2.22-0.el8.x86_64 54/66 2024-04-17T22:30:51.992 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : rbd-mirror-2:16.2.15-0.el8.x86_64 55/66 2024-04-17T22:30:51.993 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : rbd-mirror-2:14.2.22-0.el8.x86_64 56/66 2024-04-17T22:30:51.993 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : rbd-nbd-2:16.2.15-0.el8.x86_64 57/66 2024-04-17T22:30:51.993 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : rbd-nbd-2:14.2.22-0.el8.x86_64 58/66 2024-04-17T22:30:51.993 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-grafana-dashboards-2:16.2.15-0.el8.noarch 59/66 2024-04-17T22:30:51.993 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-grafana-dashboards-2:14.2.22-0.el8.noarch 60/66 2024-04-17T22:30:51.993 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-mgr-dashboard-2:16.2.15-0.el8.noarch 61/66 2024-04-17T22:30:51.993 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-mgr-dashboard-2:14.2.22-0.el8.noarch 62/66 2024-04-17T22:30:51.993 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-mgr-diskprediction-local-2:16.2.15-0.el8.noar 63/66 2024-04-17T22:30:51.993 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-mgr-diskprediction-local-2:14.2.22-0.el8.noar 64/66 2024-04-17T22:30:51.993 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-mgr-rook-2:16.2.15-0.el8.noarch 65/66 2024-04-17T22:30:53.555 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-mgr-rook-2:14.2.22-0.el8.noarch 66/66 2024-04-17T22:30:53.555 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-04-17T22:30:53.555 INFO:teuthology.orchestra.run.smithi190.stdout:Upgraded: 2024-04-17T22:30:53.555 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-2:16.2.15-0.el8.x86_64 2024-04-17T22:30:53.555 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-base-2:16.2.15-0.el8.x86_64 2024-04-17T22:30:53.555 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-common-2:16.2.15-0.el8.x86_64 2024-04-17T22:30:53.555 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-fuse-2:16.2.15-0.el8.x86_64 2024-04-17T22:30:53.555 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-grafana-dashboards-2:16.2.15-0.el8.noarch 2024-04-17T22:30:53.555 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mds-2:16.2.15-0.el8.x86_64 2024-04-17T22:30:53.555 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr-2:16.2.15-0.el8.x86_64 2024-04-17T22:30:53.556 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr-dashboard-2:16.2.15-0.el8.noarch 2024-04-17T22:30:53.556 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr-diskprediction-local-2:16.2.15-0.el8.noarch 2024-04-17T22:30:53.556 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr-rook-2:16.2.15-0.el8.noarch 2024-04-17T22:30:53.556 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mon-2:16.2.15-0.el8.x86_64 2024-04-17T22:30:53.556 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-osd-2:16.2.15-0.el8.x86_64 2024-04-17T22:30:53.556 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-radosgw-2:16.2.15-0.el8.x86_64 2024-04-17T22:30:53.556 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-selinux-2:16.2.15-0.el8.x86_64 2024-04-17T22:30:53.556 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-test-2:16.2.15-0.el8.x86_64 2024-04-17T22:30:53.556 INFO:teuthology.orchestra.run.smithi190.stdout: libcephfs-devel-2:16.2.15-0.el8.x86_64 2024-04-17T22:30:53.556 INFO:teuthology.orchestra.run.smithi190.stdout: libcephfs2-2:16.2.15-0.el8.x86_64 2024-04-17T22:30:53.556 INFO:teuthology.orchestra.run.smithi190.stdout: librados-devel-2:16.2.15-0.el8.x86_64 2024-04-17T22:30:53.556 INFO:teuthology.orchestra.run.smithi190.stdout: librados2-2:16.2.15-0.el8.x86_64 2024-04-17T22:30:53.557 INFO:teuthology.orchestra.run.smithi190.stdout: libradosstriper1-2:16.2.15-0.el8.x86_64 2024-04-17T22:30:53.557 INFO:teuthology.orchestra.run.smithi190.stdout: librbd1-2:16.2.15-0.el8.x86_64 2024-04-17T22:30:53.557 INFO:teuthology.orchestra.run.smithi190.stdout: librgw2-2:16.2.15-0.el8.x86_64 2024-04-17T22:30:53.557 INFO:teuthology.orchestra.run.smithi190.stdout: python3-ceph-argparse-2:16.2.15-0.el8.x86_64 2024-04-17T22:30:53.557 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cephfs-2:16.2.15-0.el8.x86_64 2024-04-17T22:30:53.557 INFO:teuthology.orchestra.run.smithi190.stdout: python3-rados-2:16.2.15-0.el8.x86_64 2024-04-17T22:30:53.557 INFO:teuthology.orchestra.run.smithi190.stdout: python3-rbd-2:16.2.15-0.el8.x86_64 2024-04-17T22:30:53.557 INFO:teuthology.orchestra.run.smithi190.stdout: python3-rgw-2:16.2.15-0.el8.x86_64 2024-04-17T22:30:53.557 INFO:teuthology.orchestra.run.smithi190.stdout: rbd-fuse-2:16.2.15-0.el8.x86_64 2024-04-17T22:30:53.557 INFO:teuthology.orchestra.run.smithi190.stdout: rbd-mirror-2:16.2.15-0.el8.x86_64 2024-04-17T22:30:53.557 INFO:teuthology.orchestra.run.smithi190.stdout: rbd-nbd-2:16.2.15-0.el8.x86_64 2024-04-17T22:30:53.557 INFO:teuthology.orchestra.run.smithi190.stdout:Installed: 2024-04-17T22:30:53.557 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr-modules-core-2:16.2.15-0.el8.noarch 2024-04-17T22:30:53.558 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-prometheus-alerts-2:16.2.15-0.el8.noarch 2024-04-17T22:30:53.558 INFO:teuthology.orchestra.run.smithi190.stdout: libcephsqlite-2:16.2.15-0.el8.x86_64 2024-04-17T22:30:53.558 INFO:teuthology.orchestra.run.smithi190.stdout: python3-ceph-common-2:16.2.15-0.el8.x86_64 2024-04-17T22:30:53.558 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jsonpatch-1.21-2.el8.noarch 2024-04-17T22:30:53.558 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jsonpointer-1.10-11.el8.noarch 2024-04-17T22:30:53.558 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-04-17T22:30:53.558 INFO:teuthology.orchestra.run.smithi190.stdout:Complete! 2024-04-17T22:30:53.709 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=pacific 2024-04-17T22:30:53.907 DEBUG:teuthology.orchestra.run.smithi190:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-17T22:30:53.930 INFO:teuthology.orchestra.run.smithi190.stdout:16.2.15-0.el8 2024-04-17T22:30:53.930 INFO:teuthology.packaging:The installed version of ceph is 16.2.15-0.el8 2024-04-17T22:30:53.930 INFO:teuthology.task.install:The correct ceph version 16.2.15-0 is installed. 2024-04-17T22:30:53.932 INFO:teuthology.run_tasks:Running task print... 2024-04-17T22:30:53.945 INFO:teuthology.task.print:**** done install.upgrade on client.0 2024-04-17T22:30:53.945 INFO:teuthology.run_tasks:Running task ceph.healthy... 2024-04-17T22:30:53.992 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-17T22:30:53.992 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-04-17T22:30:53.993 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-04-17T22:30:54.138 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.150 7faa4103d700 1 Processor -- start 2024-04-17T22:30:54.138 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.150 7faa4103d700 1 -- start start 2024-04-17T22:30:54.138 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.150 7faa4103d700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7faa3c0f5770 0x7faa3c10c110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:30:54.138 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.150 7faa4103d700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7faa3c10c740 0x7faa3c1a0c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:30:54.139 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.150 7faa4103d700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7faa3c10cb90 0x7faa3c1a1230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:30:54.139 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.150 7faa4103d700 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa3c0f93e0 con 0x7faa3c10c740 2024-04-17T22:30:54.139 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.150 7faa4103d700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa3c1a1860 con 0x7faa3c10cb90 2024-04-17T22:30:54.139 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.150 7faa4103d700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7faa3c1a1a90 con 0x7faa3c0f5770 2024-04-17T22:30:54.139 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.151 7faa39dfc700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7faa3c0f5770 0x7faa3c10c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:30:54.139 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.151 7faa3a5fd700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7faa3c10cb90 0x7faa3c1a1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:30:54.139 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.151 7faa33fff700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7faa3c10c740 0x7faa3c1a0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:30:54.139 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.151 7faa33fff700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7faa3c10c740 0x7faa3c1a0c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:40152/0 (socket says 172.21.15.102:40152) 2024-04-17T22:30:54.139 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.151 7faa3a5fd700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7faa3c10cb90 0x7faa3c1a1230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:48578/0 (socket says 172.21.15.102:48578) 2024-04-17T22:30:54.139 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.151 7faa39dfc700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7faa3c0f5770 0x7faa3c10c110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:58942/0 (socket says 172.21.15.102:58942) 2024-04-17T22:30:54.139 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.151 7faa33fff700 1 -- 172.21.15.102:0/448306958 learned_addr learned my addr 172.21.15.102:0/448306958 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:30:54.139 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.151 7faa3a5fd700 1 -- 172.21.15.102:0/448306958 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7faa3c0f5770 msgr2=0x7faa3c10c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:30:54.139 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.151 7faa3a5fd700 1 --2- 172.21.15.102:0/448306958 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7faa3c0f5770 0x7faa3c10c110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:30:54.139 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.151 7faa3a5fd700 1 -- 172.21.15.102:0/448306958 >> v2:172.21.15.102:3300/0 conn(0x7faa3c10c740 msgr2=0x7faa3c1a0c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:30:54.140 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.151 7faa3a5fd700 1 --2- 172.21.15.102:0/448306958 >> v2:172.21.15.102:3300/0 conn(0x7faa3c10c740 0x7faa3c1a0c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:30:54.140 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.151 7faa3a5fd700 1 -- 172.21.15.102:0/448306958 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa3c1a8310 con 0x7faa3c10cb90 2024-04-17T22:30:54.140 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.151 7faa39dfc700 1 --2- 172.21.15.102:0/448306958 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7faa3c0f5770 0x7faa3c10c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:30:54.140 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.151 7faa33fff700 1 --2- 172.21.15.102:0/448306958 >> v2:172.21.15.102:3300/0 conn(0x7faa3c10c740 0x7faa3c1a0c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:30:54.140 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.151 7faa3a5fd700 1 --2- 172.21.15.102:0/448306958 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7faa3c10cb90 0x7faa3c1a1230 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0x7faa3c089f10 tx=0x7faa34009d40).ready entity=mon.1 client_cookie=79aa7323e4327973 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:30:54.140 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.151 7faa395fb700 1 -- 172.21.15.102:0/448306958 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7faa3c1a8310 con 0x7faa3c10cb90 2024-04-17T22:30:54.140 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.151 7faa395fb700 1 -- 172.21.15.102:0/448306958 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa3400b7c0 con 0x7faa3c10cb90 2024-04-17T22:30:54.140 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.151 7faa395fb700 1 -- 172.21.15.102:0/448306958 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7faa3400b9a0 con 0x7faa3c10cb90 2024-04-17T22:30:54.140 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.151 7faa4103d700 1 -- 172.21.15.102:0/448306958 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7faa3c10cb90 msgr2=0x7faa3c1a1230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:30:54.140 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.151 7faa4103d700 1 --2- 172.21.15.102:0/448306958 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7faa3c10cb90 0x7faa3c1a1230 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0x7faa3c089f10 tx=0x7faa34009d40).stop 2024-04-17T22:30:54.140 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.152 7faa4103d700 1 -- 172.21.15.102:0/448306958 shutdown_connections 2024-04-17T22:30:54.140 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.152 7faa4103d700 1 --2- 172.21.15.102:0/448306958 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7faa3c10cb90 0x7faa3c1a1230 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:30:54.140 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.152 7faa4103d700 1 --2- 172.21.15.102:0/448306958 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7faa3c0f5770 0x7faa3c10c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:30:54.140 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.152 7faa4103d700 1 --2- 172.21.15.102:0/448306958 >> v2:172.21.15.102:3300/0 conn(0x7faa3c10c740 0x7faa3c1a0c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:30:54.141 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.152 7faa4103d700 1 -- 172.21.15.102:0/448306958 >> 172.21.15.102:0/448306958 conn(0x7faa3c1138c0 msgr2=0x7faa3c0f4f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:30:54.141 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.152 7faa4103d700 1 -- 172.21.15.102:0/448306958 shutdown_connections 2024-04-17T22:30:54.141 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.152 7faa4103d700 1 -- 172.21.15.102:0/448306958 wait complete. 2024-04-17T22:30:54.162 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.175 7faa4103d700 1 Processor -- start 2024-04-17T22:30:54.162 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.175 7faa4103d700 1 -- start start 2024-04-17T22:30:54.162 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.175 7faa4103d700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7faa3c1ad650 0x7faa3c1adaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:30:54.163 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.175 7faa4103d700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faa3c1ae0d0 0x7faa3c1b40a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:30:54.163 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.175 7faa4103d700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7faa3c1b46d0 0x7faa3c1b4b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:30:54.163 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.175 7faa4103d700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa3c0c46f0 con 0x7faa3c1ae0d0 2024-04-17T22:30:54.163 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.175 7faa4103d700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa3c0da150 con 0x7faa3c1ad650 2024-04-17T22:30:54.163 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.175 7faa4103d700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7faa3c1b5150 con 0x7faa3c1b46d0 2024-04-17T22:30:54.163 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.175 7faa33fff700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faa3c1ae0d0 0x7faa3c1b40a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:30:54.163 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.175 7faa3a5fd700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7faa3c1b46d0 0x7faa3c1b4b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:30:54.163 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.175 7faa39dfc700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7faa3c1ad650 0x7faa3c1adaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:30:54.163 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.175 7faa33fff700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faa3c1ae0d0 0x7faa3c1b40a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:40168/0 (socket says 172.21.15.102:40168) 2024-04-17T22:30:54.163 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.175 7faa3a5fd700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7faa3c1b46d0 0x7faa3c1b4b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:58946/0 (socket says 172.21.15.102:58946) 2024-04-17T22:30:54.163 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.175 7faa33fff700 1 -- 172.21.15.102:0/708046743 learned_addr learned my addr 172.21.15.102:0/708046743 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:30:54.163 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.175 7faa39dfc700 1 -- 172.21.15.102:0/708046743 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7faa3c1b46d0 msgr2=0x7faa3c1b4b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:30:54.163 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.175 7faa39dfc700 1 --2- 172.21.15.102:0/708046743 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7faa3c1b46d0 0x7faa3c1b4b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:30:54.163 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.175 7faa39dfc700 1 -- 172.21.15.102:0/708046743 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faa3c1ae0d0 msgr2=0x7faa3c1b40a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:30:54.164 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.175 7faa39dfc700 1 --2- 172.21.15.102:0/708046743 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faa3c1ae0d0 0x7faa3c1b40a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:30:54.164 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.175 7faa39dfc700 1 -- 172.21.15.102:0/708046743 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa3c1a6c20 con 0x7faa3c1ad650 2024-04-17T22:30:54.164 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.175 7faa33fff700 1 --2- 172.21.15.102:0/708046743 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faa3c1ae0d0 0x7faa3c1b40a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:30:54.164 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.175 7faa39dfc700 1 --2- 172.21.15.102:0/708046743 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7faa3c1ad650 0x7faa3c1adaa0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0x7faa3c0d7740 tx=0x7faa2c00ecd0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:30:54.164 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.176 7faa32ffd700 1 -- 172.21.15.102:0/708046743 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7faa3c1a6c20 con 0x7faa3c1ad650 2024-04-17T22:30:54.164 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.176 7faa32ffd700 1 -- 172.21.15.102:0/708046743 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa2c00c9a0 con 0x7faa3c1ad650 2024-04-17T22:30:54.164 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.176 7faa32ffd700 1 -- 172.21.15.102:0/708046743 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7faa2c00f040 con 0x7faa3c1ad650 2024-04-17T22:30:54.164 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.176 7faa4103d700 1 -- 172.21.15.102:0/708046743 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa3c1bb790 con 0x7faa3c1ad650 2024-04-17T22:30:54.164 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.176 7faa4103d700 1 -- 172.21.15.102:0/708046743 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa3c1bbb70 con 0x7faa3c1ad650 2024-04-17T22:30:54.164 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.176 7faa32ffd700 1 -- 172.21.15.102:0/708046743 <== mon.1 v2:172.21.15.102:3301/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7faa2c011070 con 0x7faa3c1ad650 2024-04-17T22:30:54.164 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.176 7faa32ffd700 1 -- 172.21.15.102:0/708046743 <== mon.1 v2:172.21.15.102:3301/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3194+0+0 (secure 0 0 0) 0x7faa2c010040 con 0x7faa3c1ad650 2024-04-17T22:30:54.165 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.177 7faa4103d700 1 -- 172.21.15.102:0/708046743 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa3c1ae7e0 con 0x7faa3c1ad650 2024-04-17T22:30:54.166 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.180 7faa32ffd700 1 -- 172.21.15.102:0/708046743 <== mon.1 v2:172.21.15.102:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+110455 (secure 0 0 0) 0x7faa2c00c5a0 con 0x7faa3c1ad650 2024-04-17T22:30:54.292 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.305 7faa4103d700 1 -- 172.21.15.102:0/708046743 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7faa3c0abcc0 con 0x7faa3c1ad650 2024-04-17T22:30:54.292 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.306 7faa32ffd700 1 -- 172.21.15.102:0/708046743 <== mon.1 v2:172.21.15.102:3301/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v3) v1 ==== 74+0+32134 (secure 0 0 0) 0x7faa2c000c30 con 0x7faa3c1ad650 2024-04-17T22:30:54.293 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-17T22:30:54.294 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.308 7faa4103d700 1 -- 172.21.15.102:0/708046743 >> v1:172.21.15.102:6816/42360 conn(0x7faa200145a0 legacy=0x7faa20016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:30:54.294 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.308 7faa4103d700 1 -- 172.21.15.102:0/708046743 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7faa3c1ad650 msgr2=0x7faa3c1adaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:30:54.294 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.308 7faa4103d700 1 --2- 172.21.15.102:0/708046743 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7faa3c1ad650 0x7faa3c1adaa0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0x7faa3c0d7740 tx=0x7faa2c00ecd0).stop 2024-04-17T22:30:54.294 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.308 7faa4103d700 1 -- 172.21.15.102:0/708046743 shutdown_connections 2024-04-17T22:30:54.295 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.308 7faa4103d700 1 --2- 172.21.15.102:0/708046743 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7faa3c1b46d0 0x7faa3c1b4b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:30:54.295 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.308 7faa4103d700 1 --2- 172.21.15.102:0/708046743 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7faa3c1ad650 0x7faa3c1adaa0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:30:54.295 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.308 7faa4103d700 1 --2- 172.21.15.102:0/708046743 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faa3c1ae0d0 0x7faa3c1b40a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:30:54.295 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.308 7faa4103d700 1 -- 172.21.15.102:0/708046743 >> 172.21.15.102:0/708046743 conn(0x7faa3c0c6af0 msgr2=0x7faa3c0c31c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:30:54.295 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.308 7faa4103d700 1 -- 172.21.15.102:0/708046743 shutdown_connections 2024-04-17T22:30:54.295 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.308 7faa4103d700 1 -- 172.21.15.102:0/708046743 wait complete. 2024-04-17T22:30:54.304 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":3,"active_gid":4104,"active_name":"x","active_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6816","nonce":42360}]},"active_addr":"172.21.15.102:6816/42360","active_change":"2024-04-17 22:28:51.181757","available":true,"standbys":[],"modules":["iostat","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"ansible","can_run":true,"error_string":"","module_options":{"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_url":{"name":"server_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"verify_server":{"name":"verify_server","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"none","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_iterations":{"name":"upmap_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimization iterations","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_cephfs":{"name":"FEATURE_TOGGLE_cephfs","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_iscsi":{"name":"FEATURE_TOGGLE_iscsi","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_mirroring":{"name":"FEATURE_TOGGLE_mirroring","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_rbd":{"name":"FEATURE_TOGGLE_rbd","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_rgw":{"name":"FEATURE_TOGGLE_rgw","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_HOST":{"name":"RGW_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_PORT":{"name":"RGW_API_PORT","type":"int","level":"advanced","flags":0,"default_value":"80","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SCHEME":{"name":"RGW_API_SCHEME","type":"str","level":"advanced","flags":0,"default_value":"http","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_USER_ID":{"name":"RGW_API_USER_ID","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"deepsea","can_run":true,"error_string":"","module_options":{"salt_api_eauth":{"name":"salt_api_eauth","type":"str","level":"advanced","flags":0,"default_value":"sharedsecret","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"salt_api_password":{"name":"salt_api_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"salt_api_url":{"name":"salt_api_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"salt_api_username":{"name":"salt_api_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"str","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"str","level":"advanced","flags":0,"default_value":"30","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"str","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"str","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{}},{"name":"iostat","can_run":true,"error_string":"","module_options":{}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator_cli","can_run":true,"error_string":"","module_options":{"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"persist_interval":{"name":"persist_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how frequently to persist completed events","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{}},{"name":"restful","can_run":true,"error_string":"","module_options":{"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"nautilus":["balancer","crash","devicehealth","orchestrator_cli","progress","rbd_support","status","volumes"]}} 2024-04-17T22:30:54.305 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-04-17T22:30:54.305 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-04-17T22:30:54.306 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T22:30:54.432 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.444 7fbafdd25700 1 Processor -- start 2024-04-17T22:30:54.432 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.444 7fbafdd25700 1 -- start start 2024-04-17T22:30:54.432 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.444 7fbafdd25700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fbaf810e260 0x7fbaf810e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:30:54.432 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.444 7fbafdd25700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fbaf80f1f50 0x7fbaf81ab500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:30:54.432 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.444 7fbafdd25700 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbaf80f23a0 con 0x7fbaf810ece0 2024-04-17T22:30:54.433 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.444 7fbafdd25700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbaf80f9f30 con 0x7fbaf80f1f50 2024-04-17T22:30:54.433 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.444 7fbafdd25700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbaf80f9050 con 0x7fbaf810e260 2024-04-17T22:30:54.433 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.444 7fbaf72b3700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fbaf80f1f50 0x7fbaf81ab500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:30:54.433 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.444 7fbaeffff700 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7fbaf810ece0 0x7fbaf80f1750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:40858/0 (socket says 172.21.15.102:40858) 2024-04-17T22:30:54.433 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.444 7fbaf6ab2700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fbaf810e260 0x7fbaf810e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:30:54.433 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.444 7fbaeffff700 1 -- 172.21.15.102:0/212996934 learned_addr learned my addr 172.21.15.102:0/212996934 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-17T22:30:54.433 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.444 7fbaf62b1700 1 -- 172.21.15.102:0/212996934 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 83734731 0 0) 0x7fbae0001110 con 0x7fbaf810ece0 2024-04-17T22:30:54.433 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.444 7fbaf72b3700 1 -- 172.21.15.102:0/212996934 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fbaf810e260 msgr2=0x7fbaf810e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:30:54.433 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.444 7fbaf72b3700 1 --2- 172.21.15.102:0/212996934 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fbaf810e260 0x7fbaf810e6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:30:54.433 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.444 7fbaf72b3700 1 -- 172.21.15.102:0/212996934 >> v1:172.21.15.102:6789/0 conn(0x7fbaf810ece0 legacy=0x7fbaf80f1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:30:54.433 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.445 7fbaf72b3700 1 -- 172.21.15.102:0/212996934 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbaf80f25c0 con 0x7fbaf80f1f50 2024-04-17T22:30:54.433 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.445 7fbaf6ab2700 1 --2- 172.21.15.102:0/212996934 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fbaf810e260 0x7fbaf810e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:30:54.433 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.445 7fbaf72b3700 1 --2- 172.21.15.102:0/212996934 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fbaf80f1f50 0x7fbaf81ab500 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0x7fbaf80f46c0 tx=0x7fbaf0009d70).ready entity=mon.1 client_cookie=a0d2ee3eece58bb8 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:30:54.433 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.445 7fbaf62b1700 1 -- 172.21.15.102:0/212996934 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fbaf80f25c0 con 0x7fbaf80f1f50 2024-04-17T22:30:54.434 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.445 7fbaf62b1700 1 -- 172.21.15.102:0/212996934 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbaf000b7c0 con 0x7fbaf80f1f50 2024-04-17T22:30:54.434 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.445 7fbaf62b1700 1 -- 172.21.15.102:0/212996934 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fbaf000b9a0 con 0x7fbaf80f1f50 2024-04-17T22:30:54.434 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.445 7fbafdd25700 1 -- 172.21.15.102:0/212996934 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fbaf80f1f50 msgr2=0x7fbaf81ab500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:30:54.434 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.445 7fbafdd25700 1 --2- 172.21.15.102:0/212996934 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fbaf80f1f50 0x7fbaf81ab500 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0x7fbaf80f46c0 tx=0x7fbaf0009d70).stop 2024-04-17T22:30:54.434 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.445 7fbafdd25700 1 -- 172.21.15.102:0/212996934 shutdown_connections 2024-04-17T22:30:54.434 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.445 7fbafdd25700 1 --2- 172.21.15.102:0/212996934 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fbaf80f1f50 0x7fbaf81ab500 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:30:54.434 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.445 7fbafdd25700 1 --2- 172.21.15.102:0/212996934 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fbaf810e260 0x7fbaf810e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:30:54.434 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.445 7fbafdd25700 1 -- 172.21.15.102:0/212996934 >> 172.21.15.102:0/212996934 conn(0x7fbaf81138c0 msgr2=0x7fbaf80f4f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:30:54.434 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.445 7fbafdd25700 1 -- 172.21.15.102:0/212996934 shutdown_connections 2024-04-17T22:30:54.434 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.445 7fbafdd25700 1 -- 172.21.15.102:0/212996934 wait complete. 2024-04-17T22:30:54.457 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.468 7fbafdd25700 1 Processor -- start 2024-04-17T22:30:54.457 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.468 7fbafdd25700 1 -- start start 2024-04-17T22:30:54.457 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.468 7fbafdd25700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fbaf80e04b0 0x7fbaf81ad990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:30:54.457 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.468 7fbafdd25700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fbaf81adfc0 0x7fbaf81b42c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:30:54.457 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.468 7fbafdd25700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbaf81ae410 0x7fbaf81b48f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:30:54.457 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.468 7fbafdd25700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbaf8113b20 con 0x7fbaf81ae410 2024-04-17T22:30:54.457 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.468 7fbafdd25700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbaf80fa1d0 con 0x7fbaf81adfc0 2024-04-17T22:30:54.457 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.468 7fbafdd25700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbaf80e0900 con 0x7fbaf80e04b0 2024-04-17T22:30:54.457 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.468 7fbaf6ab2700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fbaf80e04b0 0x7fbaf81ad990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:30:54.457 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.469 7fbaf6ab2700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fbaf80e04b0 0x7fbaf81ad990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:58968/0 (socket says 172.21.15.102:58968) 2024-04-17T22:30:54.458 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.469 7fbaeffff700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fbaf81adfc0 0x7fbaf81b42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:30:54.458 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.469 7fbaf72b3700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbaf81ae410 0x7fbaf81b48f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:30:54.458 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.469 7fbaf6ab2700 1 -- 172.21.15.102:0/3023640431 learned_addr learned my addr 172.21.15.102:0/3023640431 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:30:54.458 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.469 7fbaf6ab2700 1 -- 172.21.15.102:0/3023640431 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fbaf81adfc0 msgr2=0x7fbaf81b42c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:30:54.458 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.469 7fbaf6ab2700 1 --2- 172.21.15.102:0/3023640431 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fbaf81adfc0 0x7fbaf81b42c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:30:54.458 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.469 7fbaf6ab2700 1 -- 172.21.15.102:0/3023640431 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbaf81ae410 msgr2=0x7fbaf81b48f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:30:54.458 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.469 7fbaf6ab2700 1 --2- 172.21.15.102:0/3023640431 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbaf81ae410 0x7fbaf81b48f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:30:54.458 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.469 7fbaf6ab2700 1 -- 172.21.15.102:0/3023640431 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbaf80dad20 con 0x7fbaf80e04b0 2024-04-17T22:30:54.458 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.469 7fbaf72b3700 1 --2- 172.21.15.102:0/3023640431 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbaf81ae410 0x7fbaf81b48f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:30:54.458 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.469 7fbaf6ab2700 1 --2- 172.21.15.102:0/3023640431 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fbaf80e04b0 0x7fbaf81ad990 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0x7fbaf80d6c90 tx=0x7fbae800bf70).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:30:54.458 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.469 7fbaf4aae700 1 -- 172.21.15.102:0/3023640431 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fbaf80dad20 con 0x7fbaf80e04b0 2024-04-17T22:30:54.458 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.469 7fbaf4aae700 1 -- 172.21.15.102:0/3023640431 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbae800ee60 con 0x7fbaf80e04b0 2024-04-17T22:30:54.458 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.469 7fbaf4aae700 1 -- 172.21.15.102:0/3023640431 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fbae800f040 con 0x7fbaf80e04b0 2024-04-17T22:30:54.458 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.469 7fbafdd25700 1 -- 172.21.15.102:0/3023640431 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbaf81b4f20 con 0x7fbaf80e04b0 2024-04-17T22:30:54.459 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.469 7fbafdd25700 1 -- 172.21.15.102:0/3023640431 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbaf81bba40 con 0x7fbaf80e04b0 2024-04-17T22:30:54.459 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.470 7fbaf4aae700 1 -- 172.21.15.102:0/3023640431 <== mon.2 v2:172.21.15.102:3302/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7fbae8011070 con 0x7fbaf80e04b0 2024-04-17T22:30:54.459 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.470 7fbaf4aae700 1 -- 172.21.15.102:0/3023640431 <== mon.2 v2:172.21.15.102:3302/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3194+0+0 (secure 0 0 0) 0x7fbae8010040 con 0x7fbaf80e04b0 2024-04-17T22:30:54.459 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.470 7fbafdd25700 1 -- 172.21.15.102:0/3023640431 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbaf80c4a10 con 0x7fbaf80e04b0 2024-04-17T22:30:54.459 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.473 7fbaf4aae700 1 -- 172.21.15.102:0/3023640431 <== mon.2 v2:172.21.15.102:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+110455 (secure 0 0 0) 0x7fbae800c620 con 0x7fbaf80e04b0 2024-04-17T22:30:54.563 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.577 7fbafdd25700 1 -- 172.21.15.102:0/3023640431 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbaf80f0820 con 0x7fbaf80e04b0 2024-04-17T22:30:54.564 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.578 7fbaf4aae700 1 -- 172.21.15.102:0/3023640431 <== mon.2 v2:172.21.15.102:3302/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) v1 ==== 74+0+8764 (secure 0 0 0) 0x7fbae8000c30 con 0x7fbaf80e04b0 2024-04-17T22:30:54.564 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-17T22:30:54.564 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":17,"fsid":"cb63e789-091d-49b1-b120-55f9db526871","created":"2024-04-17 22:28:48.209635","modified":"2024-04-17 22:29:23.257387","last_up_change":"2024-04-17 22:28:54.236215","last_in_change":"2024-04-17 22:28:54.236215","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":4,"require_min_compat_client":"nautilus","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-04-17 22:28:57.478545","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-17 22:29:06.144708","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-17 22:29:06.748924","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3351ef85-ade4-4b90-a8f4-213f4e00ba01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6812","nonce":42955}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6813","nonce":42955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6815","nonce":42955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6814","nonce":42955}]},"public_addr":"172.21.15.102:6812/42955","cluster_addr":"172.21.15.102:6813/42955","heartbeat_back_addr":"172.21.15.102:6815/42955","heartbeat_front_addr":"172.21.15.102:6814/42955","state":["exists","up"]},{"osd":1,"uuid":"04f67eac-f36b-48f5-982d-4cfc776596d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6808","nonce":42958}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6809","nonce":42958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6811","nonce":42958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6810","nonce":42958}]},"public_addr":"172.21.15.102:6808/42958","cluster_addr":"172.21.15.102:6809/42958","heartbeat_back_addr":"172.21.15.102:6811/42958","heartbeat_front_addr":"172.21.15.102:6810/42958","state":["exists","up"]},{"osd":2,"uuid":"d524204b-70b2-40a8-8f73-f9e7d44318af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6804","nonce":42954}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6805","nonce":42954}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6807","nonce":42954}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6806","nonce":42954}]},"public_addr":"172.21.15.102:6804/42954","cluster_addr":"172.21.15.102:6805/42954","heartbeat_back_addr":"172.21.15.102:6807/42954","heartbeat_front_addr":"172.21.15.102:6806/42954","state":["exists","up"]},{"osd":3,"uuid":"4fc9d20c-1ad0-484f-aafa-2c9a7574245b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6800","nonce":42953}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6801","nonce":42953}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6803","nonce":42953}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6802","nonce":42953}]},"public_addr":"172.21.15.102:6800/42953","cluster_addr":"172.21.15.102:6801/42953","heartbeat_back_addr":"172.21.15.102:6803/42953","heartbeat_front_addr":"172.21.15.102:6802/42953","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2024-04-17T22:30:54.566 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.579 7fbafdd25700 1 -- 172.21.15.102:0/3023640431 >> v1:172.21.15.102:6816/42360 conn(0x7fbadc0145a0 legacy=0x7fbadc016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:30:54.566 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.579 7fbafdd25700 1 -- 172.21.15.102:0/3023640431 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fbaf80e04b0 msgr2=0x7fbaf81ad990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:30:54.566 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.579 7fbafdd25700 1 --2- 172.21.15.102:0/3023640431 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fbaf80e04b0 0x7fbaf81ad990 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0x7fbaf80d6c90 tx=0x7fbae800bf70).stop 2024-04-17T22:30:54.566 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.579 7fbafdd25700 1 -- 172.21.15.102:0/3023640431 shutdown_connections 2024-04-17T22:30:54.566 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.579 7fbafdd25700 1 --2- 172.21.15.102:0/3023640431 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbaf81ae410 0x7fbaf81b48f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:30:54.566 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.579 7fbafdd25700 1 --2- 172.21.15.102:0/3023640431 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fbaf80e04b0 0x7fbaf81ad990 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:30:54.566 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.579 7fbafdd25700 1 --2- 172.21.15.102:0/3023640431 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fbaf81adfc0 0x7fbaf81b42c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:30:54.566 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.579 7fbafdd25700 1 -- 172.21.15.102:0/3023640431 >> 172.21.15.102:0/3023640431 conn(0x7fbaf80c5930 msgr2=0x7fbaf80d70e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:30:54.566 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.579 7fbafdd25700 1 -- 172.21.15.102:0/3023640431 shutdown_connections 2024-04-17T22:30:54.567 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.579 7fbafdd25700 1 -- 172.21.15.102:0/3023640431 wait complete. 2024-04-17T22:30:54.576 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-04-17T22:30:54.577 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T22:30:54.703 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.715 7ff0fb24b700 1 Processor -- start 2024-04-17T22:30:54.703 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.715 7ff0fb24b700 1 -- start start 2024-04-17T22:30:54.703 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.715 7ff0fb24b700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7ff0f410c910 0x7ff0f41a6300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:30:54.704 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.715 7ff0fb24b700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7ff0f41a6930 0x7ff0f41a8d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:30:54.704 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.715 7ff0fb24b700 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0f41a1c10 con 0x7ff0f40f5770 2024-04-17T22:30:54.704 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.715 7ff0fb24b700 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0f40f93e0 con 0x7ff0f410c910 2024-04-17T22:30:54.704 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.715 7ff0fb24b700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff0f410cd60 con 0x7ff0f41a6930 2024-04-17T22:30:54.704 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.716 7ff0f8847700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7ff0f41a6930 0x7ff0f41a8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:30:54.704 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.716 7ff0ebfff700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7ff0f410c910 0x7ff0f41a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:30:54.704 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.716 7ff0f8847700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7ff0f41a6930 0x7ff0f41a8d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:58978/0 (socket says 172.21.15.102:58978) 2024-04-17T22:30:54.704 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.716 7ff0f8847700 1 -- 172.21.15.102:0/2905387475 learned_addr learned my addr 172.21.15.102:0/2905387475 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:30:54.704 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.716 7ff0f8847700 1 -- 172.21.15.102:0/2905387475 >> v2:172.21.15.102:3300/0 conn(0x7ff0f410c910 msgr2=0x7ff0f41a6300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:30:54.704 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.716 7ff0f8847700 1 --2- 172.21.15.102:0/2905387475 >> v2:172.21.15.102:3300/0 conn(0x7ff0f410c910 0x7ff0f41a6300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:30:54.704 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.716 7ff0f8847700 1 -- 172.21.15.102:0/2905387475 >> v1:172.21.15.102:6789/0 conn(0x7ff0f40f5770 legacy=0x7ff0f410c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:30:54.704 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.716 7ff0f8847700 1 -- 172.21.15.102:0/2905387475 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0f41aa3d0 con 0x7ff0f41a6930 2024-04-17T22:30:54.704 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.716 7ff0f37fe700 1 -- 172.21.15.102:0/2905387475 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 585217385 0 0) 0x7ff0f41a1c10 con 0x7ff0f40f5770 2024-04-17T22:30:54.704 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.716 7ff0ebfff700 1 --2- 172.21.15.102:0/2905387475 >> v2:172.21.15.102:3300/0 conn(0x7ff0f410c910 0x7ff0f41a6300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:30:54.705 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.716 7ff0f8847700 1 --2- 172.21.15.102:0/2905387475 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7ff0f41a6930 0x7ff0f41a8d90 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 rx=0x7ff0f40f4610 tx=0x7ff0ec009a00).ready entity=mon.2 client_cookie=a1f13cf6c1f0760e server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:30:54.705 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.716 7ff0f37fe700 1 -- 172.21.15.102:0/2905387475 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff0f41aa3d0 con 0x7ff0f41a6930 2024-04-17T22:30:54.705 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.716 7ff0f37fe700 1 -- 172.21.15.102:0/2905387475 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0ec01a610 con 0x7ff0f41a6930 2024-04-17T22:30:54.705 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.716 7ff0f37fe700 1 -- 172.21.15.102:0/2905387475 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff0ec01a9e0 con 0x7ff0f41a6930 2024-04-17T22:30:54.705 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.716 7ff0fb24b700 1 -- 172.21.15.102:0/2905387475 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7ff0f41a6930 msgr2=0x7ff0f41a8d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:30:54.705 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.716 7ff0fb24b700 1 --2- 172.21.15.102:0/2905387475 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7ff0f41a6930 0x7ff0f41a8d90 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 rx=0x7ff0f40f4610 tx=0x7ff0ec009a00).stop 2024-04-17T22:30:54.705 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.717 7ff0fb24b700 1 -- 172.21.15.102:0/2905387475 shutdown_connections 2024-04-17T22:30:54.705 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.717 7ff0fb24b700 1 --2- 172.21.15.102:0/2905387475 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7ff0f41a6930 0x7ff0f41a8d90 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:30:54.705 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.717 7ff0fb24b700 1 --2- 172.21.15.102:0/2905387475 >> v2:172.21.15.102:3300/0 conn(0x7ff0f410c910 0x7ff0f41a6300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:30:54.705 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.717 7ff0fb24b700 1 -- 172.21.15.102:0/2905387475 >> 172.21.15.102:0/2905387475 conn(0x7ff0f41138c0 msgr2=0x7ff0f40f4f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:30:54.705 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.717 7ff0fb24b700 1 -- 172.21.15.102:0/2905387475 shutdown_connections 2024-04-17T22:30:54.705 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.717 7ff0fb24b700 1 -- 172.21.15.102:0/2905387475 wait complete. 2024-04-17T22:30:54.727 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.740 7ff0fb24b700 1 Processor -- start 2024-04-17T22:30:54.728 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.740 7ff0fb24b700 1 -- start start 2024-04-17T22:30:54.728 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.740 7ff0fb24b700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7ff0f40c8740 0x7ff0f40d2f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:30:54.728 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.740 7ff0fb24b700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff0f40d3540 0x7ff0f41afd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:30:54.728 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.740 7ff0fb24b700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ff0f40d3990 0x7ff0f41b03b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:30:54.728 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.740 7ff0fb24b700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0f40c6580 con 0x7ff0f40d3540 2024-04-17T22:30:54.728 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.740 7ff0fb24b700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0f40c6840 con 0x7ff0f40d3990 2024-04-17T22:30:54.728 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.740 7ff0fb24b700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff0f40c8b90 con 0x7ff0f40c8740 2024-04-17T22:30:54.728 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.740 7ff0f3fff700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7ff0f40c8740 0x7ff0f40d2f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:30:54.728 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.740 7ff0f8847700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ff0f40d3990 0x7ff0f41b03b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:30:54.728 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.740 7ff0ebfff700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff0f40d3540 0x7ff0f41afd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:30:54.728 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.740 7ff0f3fff700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7ff0f40c8740 0x7ff0f40d2f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:58988/0 (socket says 172.21.15.102:58988) 2024-04-17T22:30:54.728 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.740 7ff0ebfff700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff0f40d3540 0x7ff0f41afd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:40206/0 (socket says 172.21.15.102:40206) 2024-04-17T22:30:54.728 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.740 7ff0f3fff700 1 -- 172.21.15.102:0/1224741140 learned_addr learned my addr 172.21.15.102:0/1224741140 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:30:54.729 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.740 7ff0f3fff700 1 -- 172.21.15.102:0/1224741140 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ff0f40d3990 msgr2=0x7ff0f41b03b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:30:54.729 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.740 7ff0f3fff700 1 --2- 172.21.15.102:0/1224741140 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ff0f40d3990 0x7ff0f41b03b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:30:54.729 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.740 7ff0f3fff700 1 -- 172.21.15.102:0/1224741140 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff0f40d3540 msgr2=0x7ff0f41afd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:30:54.729 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.740 7ff0f3fff700 1 --2- 172.21.15.102:0/1224741140 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff0f40d3540 0x7ff0f41afd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:30:54.729 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.740 7ff0f3fff700 1 -- 172.21.15.102:0/1224741140 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0f41b09e0 con 0x7ff0f40c8740 2024-04-17T22:30:54.729 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.740 7ff0ebfff700 1 --2- 172.21.15.102:0/1224741140 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff0f40d3540 0x7ff0f41afd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:30:54.729 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.740 7ff0f3fff700 1 --2- 172.21.15.102:0/1224741140 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7ff0f40c8740 0x7ff0f40d2f10 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7ff0e4000bc0 tx=0x7ff0e400c790).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:30:54.729 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.741 7ff0f1ffb700 1 -- 172.21.15.102:0/1224741140 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff0f41b09e0 con 0x7ff0f40c8740 2024-04-17T22:30:54.729 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.741 7ff0f1ffb700 1 -- 172.21.15.102:0/1224741140 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0e4007240 con 0x7ff0f40c8740 2024-04-17T22:30:54.729 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.741 7ff0fb24b700 1 -- 172.21.15.102:0/1224741140 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0f41b0d00 con 0x7ff0f40c8740 2024-04-17T22:30:54.729 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.741 7ff0f1ffb700 1 -- 172.21.15.102:0/1224741140 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff0e4006710 con 0x7ff0f40c8740 2024-04-17T22:30:54.729 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.741 7ff0fb24b700 1 -- 172.21.15.102:0/1224741140 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0f41b7650 con 0x7ff0f40c8740 2024-04-17T22:30:54.729 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.741 7ff0f1ffb700 1 -- 172.21.15.102:0/1224741140 <== mon.2 v2:172.21.15.102:3302/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7ff0e401a020 con 0x7ff0f40c8740 2024-04-17T22:30:54.729 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.741 7ff0f1ffb700 1 -- 172.21.15.102:0/1224741140 <== mon.2 v2:172.21.15.102:3302/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3194+0+0 (secure 0 0 0) 0x7ff0e4003c20 con 0x7ff0f40c8740 2024-04-17T22:30:54.729 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.742 7ff0fb24b700 1 -- 172.21.15.102:0/1224741140 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0f40f0820 con 0x7ff0f40c8740 2024-04-17T22:30:54.731 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.745 7ff0f1ffb700 1 -- 172.21.15.102:0/1224741140 <== mon.2 v2:172.21.15.102:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+110455 (secure 0 0 0) 0x7ff0e4007030 con 0x7ff0f40c8740 2024-04-17T22:30:54.836 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.850 7ff0fb24b700 1 -- 172.21.15.102:0/1224741140 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff0f41aa3f0 con 0x7ff0f40c8740 2024-04-17T22:30:54.837 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.851 7ff0f1ffb700 1 -- 172.21.15.102:0/1224741140 <== mon.2 v2:172.21.15.102:3302/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) v1 ==== 74+0+8764 (secure 0 0 0) 0x7ff0e4000c30 con 0x7ff0f40c8740 2024-04-17T22:30:54.837 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-17T22:30:54.837 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":17,"fsid":"cb63e789-091d-49b1-b120-55f9db526871","created":"2024-04-17 22:28:48.209635","modified":"2024-04-17 22:29:23.257387","last_up_change":"2024-04-17 22:28:54.236215","last_in_change":"2024-04-17 22:28:54.236215","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":4,"require_min_compat_client":"nautilus","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-04-17 22:28:57.478545","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-17 22:29:06.144708","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-17 22:29:06.748924","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3351ef85-ade4-4b90-a8f4-213f4e00ba01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6812","nonce":42955}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6813","nonce":42955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6815","nonce":42955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6814","nonce":42955}]},"public_addr":"172.21.15.102:6812/42955","cluster_addr":"172.21.15.102:6813/42955","heartbeat_back_addr":"172.21.15.102:6815/42955","heartbeat_front_addr":"172.21.15.102:6814/42955","state":["exists","up"]},{"osd":1,"uuid":"04f67eac-f36b-48f5-982d-4cfc776596d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6808","nonce":42958}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6809","nonce":42958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6811","nonce":42958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6810","nonce":42958}]},"public_addr":"172.21.15.102:6808/42958","cluster_addr":"172.21.15.102:6809/42958","heartbeat_back_addr":"172.21.15.102:6811/42958","heartbeat_front_addr":"172.21.15.102:6810/42958","state":["exists","up"]},{"osd":2,"uuid":"d524204b-70b2-40a8-8f73-f9e7d44318af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6804","nonce":42954}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6805","nonce":42954}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6807","nonce":42954}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6806","nonce":42954}]},"public_addr":"172.21.15.102:6804/42954","cluster_addr":"172.21.15.102:6805/42954","heartbeat_back_addr":"172.21.15.102:6807/42954","heartbeat_front_addr":"172.21.15.102:6806/42954","state":["exists","up"]},{"osd":3,"uuid":"4fc9d20c-1ad0-484f-aafa-2c9a7574245b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6800","nonce":42953}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6801","nonce":42953}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6803","nonce":42953}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6802","nonce":42953}]},"public_addr":"172.21.15.102:6800/42953","cluster_addr":"172.21.15.102:6801/42953","heartbeat_back_addr":"172.21.15.102:6803/42953","heartbeat_front_addr":"172.21.15.102:6802/42953","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2024-04-17T22:30:54.839 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.852 7ff0fb24b700 1 -- 172.21.15.102:0/1224741140 >> v1:172.21.15.102:6816/42360 conn(0x7ff0d80145a0 legacy=0x7ff0d8016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:30:54.839 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.852 7ff0fb24b700 1 -- 172.21.15.102:0/1224741140 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7ff0f40c8740 msgr2=0x7ff0f40d2f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:30:54.839 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.852 7ff0fb24b700 1 --2- 172.21.15.102:0/1224741140 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7ff0f40c8740 0x7ff0f40d2f10 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7ff0e4000bc0 tx=0x7ff0e400c790).stop 2024-04-17T22:30:54.839 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.852 7ff0fb24b700 1 -- 172.21.15.102:0/1224741140 shutdown_connections 2024-04-17T22:30:54.839 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.852 7ff0fb24b700 1 --2- 172.21.15.102:0/1224741140 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ff0f40d3990 0x7ff0f41b03b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:30:54.839 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.852 7ff0fb24b700 1 --2- 172.21.15.102:0/1224741140 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7ff0f40c8740 0x7ff0f40d2f10 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:30:54.839 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.852 7ff0fb24b700 1 --2- 172.21.15.102:0/1224741140 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff0f40d3540 0x7ff0f41afd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:30:54.839 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.852 7ff0fb24b700 1 -- 172.21.15.102:0/1224741140 >> 172.21.15.102:0/1224741140 conn(0x7ff0f40e04d0 msgr2=0x7ff0f40c4e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:30:54.840 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.852 7ff0fb24b700 1 -- 172.21.15.102:0/1224741140 shutdown_connections 2024-04-17T22:30:54.840 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.852 7ff0fb24b700 1 -- 172.21.15.102:0/1224741140 wait complete. 2024-04-17T22:30:54.850 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-04-17T22:30:54.850 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-04-17T22:30:54.851 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-04-17T22:30:54.851 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-04-17T22:30:54.983 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.997 7febd463e700 1 Processor -- start 2024-04-17T22:30:54.983 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.997 7f11c5b02700 1 Processor -- start 2024-04-17T22:30:54.985 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.997 7febd463e700 1 -- start start 2024-04-17T22:30:54.985 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.998 7febd463e700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7febcc10e1d0 0x7febcc10e620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:30:54.985 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.998 7febd463e700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7febcc10ec50 0x7febcc0f16c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:30:54.985 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.998 7febd463e700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7febcc0f1cf0 0x7febcc1aa460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:30:54.985 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.998 7febd463e700 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7febcc0f9ea0 con 0x7febcc10e1d0 2024-04-17T22:30:54.985 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.998 7febd463e700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febcc0f8fc0 con 0x7febcc10ec50 2024-04-17T22:30:54.985 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.998 7febd463e700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7febcc0fa140 con 0x7febcc0f1cf0 2024-04-17T22:30:54.985 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.998 7febd1c3a700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7febcc0f1cf0 0x7febcc1aa460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:30:54.985 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.998 7febd0c38700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7febcc10ec50 0x7febcc0f16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:30:54.986 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.998 7febd1c3a700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7febcc0f1cf0 0x7febcc1aa460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:59000/0 (socket says 172.21.15.102:59000) 2024-04-17T22:30:54.986 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.998 7febd1c3a700 1 -- 172.21.15.102:0/606060533 learned_addr learned my addr 172.21.15.102:0/606060533 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:30:54.986 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.998 7febd0c38700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7febcc10ec50 0x7febcc0f16c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:48606/0 (socket says 172.21.15.102:48606) 2024-04-17T22:30:54.986 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.998 7febd1c3a700 1 -- 172.21.15.102:0/606060533 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7febcc10ec50 msgr2=0x7febcc0f16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:30:54.986 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.998 7febd1c3a700 1 --2- 172.21.15.102:0/606060533 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7febcc10ec50 0x7febcc0f16c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:30:54.986 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.998 7febd1c3a700 1 -- 172.21.15.102:0/606060533 >> v2:172.21.15.102:3300/0 conn(0x7febcc10e1d0 msgr2=0x7febcc10e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:30:54.986 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.998 7febd1c3a700 1 --2- 172.21.15.102:0/606060533 >> v2:172.21.15.102:3300/0 conn(0x7febcc10e1d0 0x7febcc10e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2024-04-17T22:30:54.986 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.998 7febd1c3a700 1 -- 172.21.15.102:0/606060533 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febcc0f2140 con 0x7febcc0f1cf0 2024-04-17T22:30:54.986 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.998 7febd0c38700 1 --2- 172.21.15.102:0/606060533 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7febcc10ec50 0x7febcc0f16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:30:54.987 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.998 7f11c5b02700 1 -- start start 2024-04-17T22:30:54.987 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.998 7f11c5b02700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f11c00f5870 0x7f11c01a08b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:30:54.987 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.998 7f11c5b02700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f11c01a0ee0 0x7f11c00f1730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:30:54.987 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.998 7f11c5b02700 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11c00f2560 con 0x7f11c01a1330 2024-04-17T22:30:54.987 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.998 7f11c5b02700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11c00fa1d0 con 0x7f11c00f5870 2024-04-17T22:30:54.987 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.998 7f11c5b02700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f11c00f9500 con 0x7f11c01a0ee0 2024-04-17T22:30:54.987 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.998 7f11be05c700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f11c01a0ee0 0x7f11c00f1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:30:54.987 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.998 7f11be05c700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f11c01a0ee0 0x7f11c00f1730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:59016/0 (socket says 172.21.15.102:59016) 2024-04-17T22:30:54.987 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.998 7f11be05c700 1 -- 172.21.15.102:0/392770232 learned_addr learned my addr 172.21.15.102:0/392770232 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:30:54.987 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.998 7f11be85d700 1 --2- 172.21.15.102:0/392770232 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f11c00f5870 0x7f11c01a08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:30:54.987 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.998 7febd1c3a700 1 --2- 172.21.15.102:0/606060533 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7febcc0f1cf0 0x7febcc1aa460 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0x7febcc0f4630 tx=0x7febc4009dc0).ready entity=mon.2 client_cookie=3031438723cc4ed server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:30:54.988 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.999 7f11be85d700 1 -- 172.21.15.102:0/392770232 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f11c01a0ee0 msgr2=0x7f11c00f1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:30:54.988 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.999 7f11be85d700 1 --2- 172.21.15.102:0/392770232 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f11c01a0ee0 0x7f11c00f1730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:30:54.988 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.999 7f11be85d700 1 -- 172.21.15.102:0/392770232 >> v1:172.21.15.102:6789/0 conn(0x7f11c01a1330 legacy=0x7f11c00f1d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:30:54.988 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.999 7f11be85d700 1 -- 172.21.15.102:0/392770232 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11c01b0940 con 0x7f11c00f5870 2024-04-17T22:30:54.988 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.999 7f11be05c700 1 --2- 172.21.15.102:0/392770232 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f11c01a0ee0 0x7f11c00f1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:30:54.989 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.999 7f11be85d700 1 --2- 172.21.15.102:0/392770232 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f11c00f5870 0x7f11c01a08b0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 rx=0x7f11c00f2780 tx=0x7f11b000ba50).ready entity=mon.1 client_cookie=a561482b160f9442 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:30:54.989 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.999 7febbbfff700 1 -- 172.21.15.102:0/606060533 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7febcc0f2140 con 0x7febcc0f1cf0 2024-04-17T22:30:54.989 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.999 7febbbfff700 1 -- 172.21.15.102:0/606060533 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febc400b8d0 con 0x7febcc0f1cf0 2024-04-17T22:30:54.989 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.999 7febbbfff700 1 -- 172.21.15.102:0/606060533 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7febc400bab0 con 0x7febcc0f1cf0 2024-04-17T22:30:54.990 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.999 7f11bd85b700 1 -- 172.21.15.102:0/392770232 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f11c01b0940 con 0x7f11c00f5870 2024-04-17T22:30:54.990 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.999 7f11bd85b700 1 -- 172.21.15.102:0/392770232 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11b000d570 con 0x7f11c00f5870 2024-04-17T22:30:54.990 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.999 7f11bd85b700 1 -- 172.21.15.102:0/392770232 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f11b000d940 con 0x7f11c00f5870 2024-04-17T22:30:54.990 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.999 7f11c5b02700 1 -- 172.21.15.102:0/392770232 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f11c00f5870 msgr2=0x7f11c01a08b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:30:54.990 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.999 7f11c5b02700 1 --2- 172.21.15.102:0/392770232 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f11c00f5870 0x7f11c01a08b0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 rx=0x7f11c00f2780 tx=0x7f11b000ba50).stop 2024-04-17T22:30:54.990 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.999 7febd463e700 1 -- 172.21.15.102:0/606060533 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7febcc0f1cf0 msgr2=0x7febcc1aa460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:30:54.990 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:54.999 7febd463e700 1 --2- 172.21.15.102:0/606060533 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7febcc0f1cf0 0x7febcc1aa460 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0x7febcc0f4630 tx=0x7febc4009dc0).stop 2024-04-17T22:30:54.991 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.000 7f11c5b02700 1 -- 172.21.15.102:0/392770232 shutdown_connections 2024-04-17T22:30:54.991 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.000 7f11c5b02700 1 --2- 172.21.15.102:0/392770232 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f11c00f5870 0x7f11c01a08b0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:30:54.991 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.000 7f11c5b02700 1 --2- 172.21.15.102:0/392770232 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f11c01a0ee0 0x7f11c00f1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:30:54.991 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.000 7f11c5b02700 1 -- 172.21.15.102:0/392770232 >> 172.21.15.102:0/392770232 conn(0x7f11c01138a0 msgr2=0x7f11c00f4f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:30:54.991 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.000 7f11c5b02700 1 -- 172.21.15.102:0/392770232 shutdown_connections 2024-04-17T22:30:54.991 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.000 7f11c5b02700 1 -- 172.21.15.102:0/392770232 wait complete. 2024-04-17T22:30:54.992 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.000 7febd463e700 1 -- 172.21.15.102:0/606060533 shutdown_connections 2024-04-17T22:30:54.992 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.000 7febd463e700 1 --2- 172.21.15.102:0/606060533 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7febcc0f1cf0 0x7febcc1aa460 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:30:54.992 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.000 7febd463e700 1 --2- 172.21.15.102:0/606060533 >> v2:172.21.15.102:3300/0 conn(0x7febcc10e1d0 0x7febcc10e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2024-04-17T22:30:54.992 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.000 7febd463e700 1 --2- 172.21.15.102:0/606060533 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7febcc10ec50 0x7febcc0f16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:30:54.992 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.000 7febd463e700 1 -- 172.21.15.102:0/606060533 >> 172.21.15.102:0/606060533 conn(0x7febcc113830 msgr2=0x7febcc0f4ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:30:54.992 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.001 7febd463e700 1 -- 172.21.15.102:0/606060533 shutdown_connections 2024-04-17T22:30:54.992 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.001 7febd463e700 1 -- 172.21.15.102:0/606060533 wait complete. 2024-04-17T22:30:54.993 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.003 7f4e769ef700 1 Processor -- start 2024-04-17T22:30:54.993 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.003 7f4e769ef700 1 -- start start 2024-04-17T22:30:54.993 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.003 7f4e769ef700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4e700f5750 0x7f4e7010c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:30:54.993 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.003 7f4e769ef700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f4e7010c720 0x7f4e701a0be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:30:54.993 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.003 7f4e769ef700 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e701a1a10 con 0x7f4e7010cb70 2024-04-17T22:30:54.993 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.003 7f4e769ef700 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e700f93c0 con 0x7f4e7010c720 2024-04-17T22:30:54.993 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.003 7f4e769ef700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e701a1c30 con 0x7f4e700f5750 2024-04-17T22:30:54.994 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.004 7f4e6f5eb700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4e700f5750 0x7f4e7010c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:30:54.994 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.004 7f4e6f5eb700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4e700f5750 0x7f4e7010c0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:48630/0 (socket says 172.21.15.102:48630) 2024-04-17T22:30:54.994 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.004 7f4e6f5eb700 1 -- 172.21.15.102:0/1815752276 learned_addr learned my addr 172.21.15.102:0/1815752276 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:30:54.994 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.004 7f4e6edea700 1 --2- 172.21.15.102:0/1815752276 >> v2:172.21.15.102:3300/0 conn(0x7f4e7010c720 0x7f4e701a0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:30:54.994 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.004 7f53e473e700 1 Processor -- start 2024-04-17T22:30:54.995 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.004 7f4e6f5eb700 1 -- 172.21.15.102:0/1815752276 >> v2:172.21.15.102:3300/0 conn(0x7f4e7010c720 msgr2=0x7f4e701a0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:30:54.995 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.004 7f4e6f5eb700 1 --2- 172.21.15.102:0/1815752276 >> v2:172.21.15.102:3300/0 conn(0x7f4e7010c720 0x7f4e701a0be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:30:54.996 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.004 7f4e6f5eb700 1 -- 172.21.15.102:0/1815752276 >> v1:172.21.15.102:6789/0 conn(0x7f4e7010cb70 legacy=0x7f4e701a1210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:30:54.996 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.004 7f4e6f5eb700 1 -- 172.21.15.102:0/1815752276 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e701aa310 con 0x7f4e700f5750 2024-04-17T22:30:54.996 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.004 7f4e6f5eb700 1 --2- 172.21.15.102:0/1815752276 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4e700f5750 0x7f4e7010c0f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 rx=0x7f4e60002f70 tx=0x7f4e600099d0).ready entity=mon.1 client_cookie=c9fa0b355609cfd server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:30:54.996 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.004 7f4e6e5e9700 1 -- 172.21.15.102:0/1815752276 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f4e701aa310 con 0x7f4e700f5750 2024-04-17T22:30:54.996 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.004 7f4e6e5e9700 1 -- 172.21.15.102:0/1815752276 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e6000f510 con 0x7f4e700f5750 2024-04-17T22:30:54.996 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.005 7f4e6e5e9700 1 -- 172.21.15.102:0/1815752276 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f4e6000f6f0 con 0x7f4e700f5750 2024-04-17T22:30:54.996 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.005 7f4e769ef700 1 -- 172.21.15.102:0/1815752276 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4e700f5750 msgr2=0x7f4e7010c0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:30:54.996 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.005 7f4e769ef700 1 --2- 172.21.15.102:0/1815752276 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4e700f5750 0x7f4e7010c0f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 rx=0x7f4e60002f70 tx=0x7f4e600099d0).stop 2024-04-17T22:30:54.997 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.004 7f53e473e700 1 -- start start 2024-04-17T22:30:54.997 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.005 7f53e473e700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f53dc0f5870 0x7f53dc1a08b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:30:54.997 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.005 7f53e473e700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f53dc1a0ee0 0x7f53dc0f1730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:30:54.997 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.005 7f53e473e700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f53dc1a1330 0x7f53dc0f1d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:30:54.997 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.005 7f53e473e700 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53dc0fa1d0 con 0x7f53dc0f5870 2024-04-17T22:30:54.997 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.005 7f53e473e700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53dc0f9500 con 0x7f53dc1a0ee0 2024-04-17T22:30:54.997 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.005 7f53e473e700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f53dc0f2390 con 0x7f53dc1a1330 2024-04-17T22:30:54.997 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.005 7f53e0d38700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f53dc1a0ee0 0x7f53dc0f1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:30:54.997 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.005 7f53e0d38700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f53dc1a0ee0 0x7f53dc0f1730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:48632/0 (socket says 172.21.15.102:48632) 2024-04-17T22:30:54.997 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.005 7f53e0d38700 1 -- 172.21.15.102:0/574303294 learned_addr learned my addr 172.21.15.102:0/574303294 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:30:54.998 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.005 7f53e0d38700 1 -- 172.21.15.102:0/574303294 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f53dc1a1330 msgr2=0x7f53dc0f1d60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-17T22:30:54.998 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.005 7f53e0d38700 1 --2- 172.21.15.102:0/574303294 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f53dc1a1330 0x7f53dc0f1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2024-04-17T22:30:54.998 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.005 7f53e0d38700 1 -- 172.21.15.102:0/574303294 >> v2:172.21.15.102:3300/0 conn(0x7f53dc0f5870 msgr2=0x7f53dc1a08b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:30:54.998 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.005 7f53e0d38700 1 --2- 172.21.15.102:0/574303294 >> v2:172.21.15.102:3300/0 conn(0x7f53dc0f5870 0x7f53dc1a08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2024-04-17T22:30:54.998 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.005 7f53e0d38700 1 -- 172.21.15.102:0/574303294 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53dc0f25c0 con 0x7f53dc1a0ee0 2024-04-17T22:30:54.998 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.005 7f53e0d38700 1 --2- 172.21.15.102:0/574303294 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f53dc1a0ee0 0x7f53dc0f1730 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 rx=0x7f53dc106480 tx=0x7f53d4009e10).ready entity=mon.1 client_cookie=b1f7347971521b5c server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:30:54.998 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.005 7f53cbfff700 1 -- 172.21.15.102:0/574303294 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f53dc0f25c0 con 0x7f53dc1a0ee0 2024-04-17T22:30:54.998 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.005 7f4e769ef700 1 -- 172.21.15.102:0/1815752276 shutdown_connections 2024-04-17T22:30:54.999 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.006 7f4e769ef700 1 --2- 172.21.15.102:0/1815752276 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4e700f5750 0x7f4e7010c0f0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:30:54.999 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.006 7f4e769ef700 1 --2- 172.21.15.102:0/1815752276 >> v2:172.21.15.102:3300/0 conn(0x7f4e7010c720 0x7f4e701a0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:30:54.999 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.006 7f4e769ef700 1 -- 172.21.15.102:0/1815752276 >> 172.21.15.102:0/1815752276 conn(0x7f4e701138a0 msgr2=0x7f4e700f4f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:30:54.999 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.006 7f53cbfff700 1 -- 172.21.15.102:0/574303294 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53d400b890 con 0x7f53dc1a0ee0 2024-04-17T22:30:54.999 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.006 7f53cbfff700 1 -- 172.21.15.102:0/574303294 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f53d400ba70 con 0x7f53dc1a0ee0 2024-04-17T22:30:54.999 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.006 7f4e769ef700 1 -- 172.21.15.102:0/1815752276 shutdown_connections 2024-04-17T22:30:55.000 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.006 7f53e473e700 1 -- 172.21.15.102:0/574303294 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f53dc1a0ee0 msgr2=0x7f53dc0f1730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:30:55.000 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.006 7f53e473e700 1 --2- 172.21.15.102:0/574303294 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f53dc1a0ee0 0x7f53dc0f1730 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 rx=0x7f53dc106480 tx=0x7f53d4009e10).stop 2024-04-17T22:30:55.000 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.006 7f4e769ef700 1 -- 172.21.15.102:0/1815752276 wait complete. 2024-04-17T22:30:55.001 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.006 7f53e473e700 1 -- 172.21.15.102:0/574303294 shutdown_connections 2024-04-17T22:30:55.001 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.006 7f53e473e700 1 --2- 172.21.15.102:0/574303294 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f53dc1a1330 0x7f53dc0f1d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2024-04-17T22:30:55.001 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.006 7f53e473e700 1 --2- 172.21.15.102:0/574303294 >> v2:172.21.15.102:3300/0 conn(0x7f53dc0f5870 0x7f53dc1a08b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2024-04-17T22:30:55.001 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.006 7f53e473e700 1 --2- 172.21.15.102:0/574303294 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f53dc1a0ee0 0x7f53dc0f1730 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:30:55.001 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.006 7f53e473e700 1 -- 172.21.15.102:0/574303294 >> 172.21.15.102:0/574303294 conn(0x7f53dc1138a0 msgr2=0x7f53dc0f4f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:30:55.001 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.007 7f53e473e700 1 -- 172.21.15.102:0/574303294 shutdown_connections 2024-04-17T22:30:55.001 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.007 7f53e473e700 1 -- 172.21.15.102:0/574303294 wait complete. 2024-04-17T22:30:55.011 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.024 7f11c5b02700 1 Processor -- start 2024-04-17T22:30:55.011 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.024 7f11c5b02700 1 -- start start 2024-04-17T22:30:55.011 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.024 7f11c5b02700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f11c00d5ff0 0x7f11c00d6440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:30:55.011 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.024 7f11c5b02700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f11c00d6a70 0x7f11c00dae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:30:55.011 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.024 7f11c5b02700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f11c00db480 0x7f11c00db8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:30:55.011 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.024 7f11c5b02700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11c008a9b0 con 0x7f11c00d6a70 2024-04-17T22:30:55.011 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.024 7f11c5b02700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11c00c4530 con 0x7f11c00d5ff0 2024-04-17T22:30:55.011 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.024 7f11c5b02700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f11c00cb570 con 0x7f11c00db480 2024-04-17T22:30:55.011 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.024 7f11be05c700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f11c00d6a70 0x7f11c00dae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:30:55.011 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.024 7f11be05c700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f11c00d6a70 0x7f11c00dae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:40236/0 (socket says 172.21.15.102:40236) 2024-04-17T22:30:55.011 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.024 7f11be05c700 1 -- 172.21.15.102:0/403227232 learned_addr learned my addr 172.21.15.102:0/403227232 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:30:55.011 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.024 7f11be85d700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f11c00d5ff0 0x7f11c00d6440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:30:55.012 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.025 7febd463e700 1 Processor -- start 2024-04-17T22:30:55.012 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.025 7febd463e700 1 -- start start 2024-04-17T22:30:55.012 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.025 7f11be05c700 1 -- 172.21.15.102:0/403227232 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f11c00db480 msgr2=0x7f11c00db8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:30:55.013 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.025 7f11be05c700 1 --2- 172.21.15.102:0/403227232 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f11c00db480 0x7f11c00db8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-04-17T22:30:55.013 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.025 7f11be05c700 1 -- 172.21.15.102:0/403227232 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f11c00d5ff0 msgr2=0x7f11c00d6440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:30:55.013 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.025 7f11be05c700 1 --2- 172.21.15.102:0/403227232 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f11c00d5ff0 0x7f11c00d6440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:30:55.013 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.025 7f11be05c700 1 -- 172.21.15.102:0/403227232 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11c00cb1c0 con 0x7f11c00d6a70 2024-04-17T22:30:55.013 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.025 7f11be05c700 1 --2- 172.21.15.102:0/403227232 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f11c00d6a70 0x7f11c00dae50 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 rx=0x7f11c00cac30 tx=0x7f11b400ec80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:30:55.013 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.025 7f11be85d700 1 --2- 172.21.15.102:0/403227232 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f11c00d5ff0 0x7f11c00d6440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T22:30:55.013 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.025 7febd463e700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7febcc1a5330 0x7febcc1a5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:30:55.013 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.025 7febd463e700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7febcc1a5f50 0x7febcc1b8da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:30:55.013 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.025 7febd463e700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7febcc1a63a0 0x7febcc1b93d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:30:55.014 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.025 7febd463e700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febcc0d3aa0 con 0x7febcc1a5f50 2024-04-17T22:30:55.014 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.025 7febd463e700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febcc1b9a00 con 0x7febcc1a5330 2024-04-17T22:30:55.014 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.025 7febd463e700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7febcc1c0420 con 0x7febcc1a63a0 2024-04-17T22:30:55.014 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.025 7febd1c3a700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7febcc1a63a0 0x7febcc1b93d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:30:55.014 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.025 7febd1439700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7febcc1a5330 0x7febcc1a5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:30:55.014 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.025 7febd1c3a700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7febcc1a63a0 0x7febcc1b93d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:59042/0 (socket says 172.21.15.102:59042) 2024-04-17T22:30:55.014 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.025 7febd1c3a700 1 -- 172.21.15.102:0/3314178118 learned_addr learned my addr 172.21.15.102:0/3314178118 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:30:55.014 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.025 7febd0c38700 1 --2- 172.21.15.102:0/3314178118 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7febcc1a5f50 0x7febcc1b8da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:30:55.014 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.025 7f11affff700 1 -- 172.21.15.102:0/403227232 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f11c00cb1c0 con 0x7f11c00d6a70 2024-04-17T22:30:55.015 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.025 7f11affff700 1 -- 172.21.15.102:0/403227232 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11b400c5a0 con 0x7f11c00d6a70 2024-04-17T22:30:55.015 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.025 7f11affff700 1 -- 172.21.15.102:0/403227232 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f11b400f040 con 0x7f11c00d6a70 2024-04-17T22:30:55.015 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.026 7febd1c3a700 1 -- 172.21.15.102:0/3314178118 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7febcc1a5330 msgr2=0x7febcc1a5920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:30:55.015 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.026 7febd1c3a700 1 --2- 172.21.15.102:0/3314178118 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7febcc1a5330 0x7febcc1a5920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:30:55.015 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.026 7febd1c3a700 1 -- 172.21.15.102:0/3314178118 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7febcc1a5f50 msgr2=0x7febcc1b8da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:30:55.015 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.026 7febd1c3a700 1 --2- 172.21.15.102:0/3314178118 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7febcc1a5f50 0x7febcc1b8da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:30:55.015 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.026 7febd1c3a700 1 -- 172.21.15.102:0/3314178118 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febcc1c05d0 con 0x7febcc1a63a0 2024-04-17T22:30:55.015 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.026 7f11c5b02700 1 -- 172.21.15.102:0/403227232 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11c00ca240 con 0x7f11c00d6a70 2024-04-17T22:30:55.016 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.026 7f11c5b02700 1 -- 172.21.15.102:0/403227232 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11c01bfa10 con 0x7f11c00d6a70 2024-04-17T22:30:55.016 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.026 7febd1c3a700 1 --2- 172.21.15.102:0/3314178118 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7febcc1a63a0 0x7febcc1b93d0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7febcc0f4630 tx=0x7febc4006c40).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:30:55.016 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.026 7febba7fc700 1 -- 172.21.15.102:0/3314178118 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7febcc1c05d0 con 0x7febcc1a63a0 2024-04-17T22:30:55.016 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.026 7febba7fc700 1 -- 172.21.15.102:0/3314178118 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febc4011a00 con 0x7febcc1a63a0 2024-04-17T22:30:55.016 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.026 7febba7fc700 1 -- 172.21.15.102:0/3314178118 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7febc4011dd0 con 0x7febcc1a63a0 2024-04-17T22:30:55.017 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.026 7f11affff700 1 -- 172.21.15.102:0/403227232 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7f11b4011070 con 0x7f11c00d6a70 2024-04-17T22:30:55.017 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.026 7f11affff700 1 -- 172.21.15.102:0/403227232 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3194+0+0 (secure 0 0 0) 0x7f11b4010040 con 0x7f11c00d6a70 2024-04-17T22:30:55.017 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.027 7f11c5b02700 1 -- 172.21.15.102:0/403227232 --> v1:172.21.15.102:6804/42954 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f11c00f0800 con 0x7f11c00c3800 2024-04-17T22:30:55.018 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.026 7febd463e700 1 -- 172.21.15.102:0/3314178118 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febcc1c07d0 con 0x7febcc1a63a0 2024-04-17T22:30:55.018 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.026 7febd0c38700 1 --2- 172.21.15.102:0/3314178118 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7febcc1a5f50 0x7febcc1b8da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T22:30:55.018 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.026 7febd1439700 1 --2- 172.21.15.102:0/3314178118 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7febcc1a5330 0x7febcc1a5920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T22:30:55.018 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.026 7febd463e700 1 -- 172.21.15.102:0/3314178118 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febcc1c0bb0 con 0x7febcc1a63a0 2024-04-17T22:30:55.018 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.027 7febba7fc700 1 -- 172.21.15.102:0/3314178118 <== mon.2 v2:172.21.15.102:3302/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7febc401c3e0 con 0x7febcc1a63a0 2024-04-17T22:30:55.018 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.027 7febba7fc700 1 -- 172.21.15.102:0/3314178118 <== mon.2 v2:172.21.15.102:3302/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3194+0+0 (secure 0 0 0) 0x7febc40290d0 con 0x7febcc1a63a0 2024-04-17T22:30:55.018 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.027 7febaa7fc700 1 -- 172.21.15.102:0/3314178118 --> v1:172.21.15.102:6812/42955 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7feba40055f0 con 0x7feba4001650 2024-04-17T22:30:55.018 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.028 7f11affff700 1 -- 172.21.15.102:0/403227232 <== osd.2 v1:172.21.15.102:6804/42954 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+5151 (unknown 0 0 4057655604) 0x7f11c00f0800 con 0x7f11c00c3800 2024-04-17T22:30:55.019 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.028 7febba7fc700 1 -- 172.21.15.102:0/3314178118 <== osd.0 v1:172.21.15.102:6812/42955 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+5151 (unknown 0 0 4057655604) 0x7feba40055f0 con 0x7feba4001650 2024-04-17T22:30:55.019 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.030 7f4e769ef700 1 Processor -- start 2024-04-17T22:30:55.019 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.030 7f4e769ef700 1 -- start start 2024-04-17T22:30:55.019 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.031 7f4e769ef700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4e700f61c0 0x7f4e700f6610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:30:55.019 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.031 7f4e769ef700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4e700f6c40 0x7f4e701ad2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:30:55.019 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.031 7f4e769ef700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f4e701ad920 0x7f4e701bb4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:30:55.020 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.031 7f4e769ef700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e700a8890 con 0x7f4e700f61c0 2024-04-17T22:30:55.020 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.031 7f4e769ef700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e700c40f0 con 0x7f4e700f6c40 2024-04-17T22:30:55.020 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.031 7f4e769ef700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4e700f8100 con 0x7f4e701ad920 2024-04-17T22:30:55.020 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.031 7f4e6edea700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4e700f6c40 0x7f4e701ad2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:30:55.020 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.031 7f4e6f5eb700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4e700f61c0 0x7f4e700f6610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:30:55.020 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.031 7f4e6edea700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4e700f6c40 0x7f4e701ad2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:48660/0 (socket says 172.21.15.102:48660) 2024-04-17T22:30:55.020 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.031 7f4e6f5eb700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4e700f61c0 0x7f4e700f6610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:40246/0 (socket says 172.21.15.102:40246) 2024-04-17T22:30:55.020 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.031 7f4e6edea700 1 -- 172.21.15.102:0/1670183031 learned_addr learned my addr 172.21.15.102:0/1670183031 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:30:55.020 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.031 7f53e473e700 1 Processor -- start 2024-04-17T22:30:55.021 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.031 7f4e6edea700 1 -- 172.21.15.102:0/1670183031 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f4e701ad920 msgr2=0x7f4e701bb4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:30:55.021 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.031 7f4e6edea700 1 --2- 172.21.15.102:0/1670183031 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f4e701ad920 0x7f4e701bb4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-04-17T22:30:55.021 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.031 7f4e6edea700 1 -- 172.21.15.102:0/1670183031 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4e700f61c0 msgr2=0x7f4e700f6610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:30:55.021 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.031 7f4e6edea700 1 --2- 172.21.15.102:0/1670183031 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4e700f61c0 0x7f4e700f6610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:30:55.021 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.031 7f4e6edea700 1 -- 172.21.15.102:0/1670183031 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e700cb010 con 0x7f4e700f6c40 2024-04-17T22:30:55.022 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.031 7f4e6f5eb700 1 --2- 172.21.15.102:0/1670183031 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4e700f61c0 0x7f4e700f6610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:30:55.022 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.032 7f4e6edea700 1 --2- 172.21.15.102:0/1670183031 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4e700f6c40 0x7f4e701ad2f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 rx=0x7f4e700e1350 tx=0x7f4e6400ecd0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:30:55.022 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.031 7f53e473e700 1 -- start start 2024-04-17T22:30:55.022 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.032 7f53e473e700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f53dc0cc1c0 0x7f53dc0cc610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:30:55.022 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.032 7f53e473e700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f53dc0ccc40 0x7f53dc0e0240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:30:55.022 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.032 7f53e473e700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f53dc0e0870 0x7f53dc0e0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:30:55.023 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.032 7f53e473e700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53dc0f4f10 con 0x7f53dc0cc1c0 2024-04-17T22:30:55.023 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.032 7f53e473e700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53dc0b45a0 con 0x7f53dc0e0870 2024-04-17T22:30:55.023 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.032 7f53e473e700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f53dc0b54e0 con 0x7f53dc0ccc40 2024-04-17T22:30:55.023 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.032 7f4e6cde6700 1 -- 172.21.15.102:0/1670183031 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f4e700cb010 con 0x7f4e700f6c40 2024-04-17T22:30:55.024 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.032 7f4e769ef700 1 -- 172.21.15.102:0/1670183031 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e700cac10 con 0x7f4e700f6c40 2024-04-17T22:30:55.024 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.032 7f4e769ef700 1 -- 172.21.15.102:0/1670183031 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e701adf50 con 0x7f4e700f6c40 2024-04-17T22:30:55.024 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.032 7f53e1d3a700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f53dc0e0870 0x7f53dc0e0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:30:55.024 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.032 7f53e1d3a700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f53dc0e0870 0x7f53dc0e0cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:48672/0 (socket says 172.21.15.102:48672) 2024-04-17T22:30:55.024 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.032 7f53e1d3a700 1 -- 172.21.15.102:0/1455284617 learned_addr learned my addr 172.21.15.102:0/1455284617 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:30:55.024 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.032 7f53e0d38700 1 --2- 172.21.15.102:0/1455284617 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f53dc0ccc40 0x7f53dc0e0240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:30:55.024 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.032 7f4e6cde6700 1 -- 172.21.15.102:0/1670183031 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e6400c9a0 con 0x7f4e700f6c40 2024-04-17T22:30:55.025 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.032 7f4e6cde6700 1 -- 172.21.15.102:0/1670183031 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f4e6400f040 con 0x7f4e700f6c40 2024-04-17T22:30:55.025 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.033 7f53e0d38700 1 -- 172.21.15.102:0/1455284617 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f53dc0e0870 msgr2=0x7f53dc0e0cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:30:55.025 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.033 7f53e0d38700 1 --2- 172.21.15.102:0/1455284617 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f53dc0e0870 0x7f53dc0e0cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:30:55.025 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.033 7f53e0d38700 1 -- 172.21.15.102:0/1455284617 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f53dc0cc1c0 msgr2=0x7f53dc0cc610 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-17T22:30:55.025 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.033 7f53e0d38700 1 --2- 172.21.15.102:0/1455284617 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f53dc0cc1c0 0x7f53dc0cc610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-04-17T22:30:55.025 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.033 7f53e0d38700 1 -- 172.21.15.102:0/1455284617 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53dc0d1470 con 0x7f53dc0ccc40 2024-04-17T22:30:55.025 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.033 7f4e6cde6700 1 -- 172.21.15.102:0/1670183031 <== mon.1 v2:172.21.15.102:3301/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7f4e64011070 con 0x7f4e700f6c40 2024-04-17T22:30:55.026 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.033 7f4e6cde6700 1 -- 172.21.15.102:0/1670183031 <== mon.1 v2:172.21.15.102:3301/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3194+0+0 (secure 0 0 0) 0x7f4e64010040 con 0x7f4e700f6c40 2024-04-17T22:30:55.026 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.033 7f53e0d38700 1 --2- 172.21.15.102:0/1455284617 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f53dc0ccc40 0x7f53dc0e0240 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7f53dc0d2800 tx=0x7f53d4006c40).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:30:55.026 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.033 7f4e769ef700 1 -- 172.21.15.102:0/1670183031 --> v1:172.21.15.102:6808/42958 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4e4c0055f0 con 0x7f4e4c001650 2024-04-17T22:30:55.027 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.034 7f53ca7fc700 1 -- 172.21.15.102:0/1455284617 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f53dc0d1470 con 0x7f53dc0ccc40 2024-04-17T22:30:55.027 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.034 7f53ca7fc700 1 -- 172.21.15.102:0/1455284617 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53d4011d30 con 0x7f53dc0ccc40 2024-04-17T22:30:55.027 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.034 7f53ca7fc700 1 -- 172.21.15.102:0/1455284617 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f53d4017040 con 0x7f53dc0ccc40 2024-04-17T22:30:55.027 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.034 7f53e473e700 1 -- 172.21.15.102:0/1455284617 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53dc0b7bc0 con 0x7f53dc0ccc40 2024-04-17T22:30:55.027 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.034 7f53e473e700 1 -- 172.21.15.102:0/1455284617 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53dc1c80a0 con 0x7f53dc0ccc40 2024-04-17T22:30:55.028 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.034 7f4e6cde6700 1 -- 172.21.15.102:0/1670183031 <== osd.1 v1:172.21.15.102:6808/42958 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+5151 (unknown 0 0 4057655604) 0x7f4e4c0055f0 con 0x7f4e4c001650 2024-04-17T22:30:55.028 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.035 7f53ca7fc700 1 -- 172.21.15.102:0/1455284617 <== mon.2 v2:172.21.15.102:3302/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7f53d401c660 con 0x7f53dc0ccc40 2024-04-17T22:30:55.028 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.035 7f53ca7fc700 1 -- 172.21.15.102:0/1455284617 <== mon.2 v2:172.21.15.102:3302/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3194+0+0 (secure 0 0 0) 0x7f53d4018030 con 0x7f53dc0ccc40 2024-04-17T22:30:55.028 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.035 7f53e473e700 1 -- 172.21.15.102:0/1455284617 --> v1:172.21.15.102:6800/42953 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f53dc08a3b0 con 0x7f53dc0a7bd0 2024-04-17T22:30:55.029 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.035 7f11c5b02700 1 -- 172.21.15.102:0/403227232 --> v1:172.21.15.102:6804/42954 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f11c01b2900 con 0x7f11c00c3800 2024-04-17T22:30:55.029 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.036 7f11affff700 1 -- 172.21.15.102:0/403227232 <== osd.2 v1:172.21.15.102:6804/42954 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (unknown 0 0 2009191733) 0x7f11c01b2900 con 0x7f11c00c3800 2024-04-17T22:30:55.029 INFO:teuthology.orchestra.run.smithi102.stdout:38654705691 2024-04-17T22:30:55.029 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.036 7f11c5b02700 1 -- 172.21.15.102:0/403227232 >> v1:172.21.15.102:6804/42954 conn(0x7f11c00c3800 legacy=0x7f11c00a6f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:30:55.030 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.036 7febd463e700 1 -- 172.21.15.102:0/3314178118 --> v1:172.21.15.102:6812/42955 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7febcc0d33c0 con 0x7feba4001650 2024-04-17T22:30:55.030 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.036 7f11c5b02700 1 -- 172.21.15.102:0/403227232 >> v1:172.21.15.102:6816/42360 conn(0x7f119c0145a0 legacy=0x7f119c016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:30:55.030 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.036 7f11c5b02700 1 -- 172.21.15.102:0/403227232 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f11c00d6a70 msgr2=0x7f11c00dae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:30:55.030 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.036 7f11c5b02700 1 --2- 172.21.15.102:0/403227232 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f11c00d6a70 0x7f11c00dae50 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 rx=0x7f11c00cac30 tx=0x7f11b400ec80).stop 2024-04-17T22:30:55.031 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.036 7f11bf05e700 1 -- 172.21.15.102:0/403227232 reap_dead start 2024-04-17T22:30:55.031 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.036 7f11c5b02700 1 -- 172.21.15.102:0/403227232 shutdown_connections 2024-04-17T22:30:55.031 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.036 7f11c5b02700 1 -- 172.21.15.102:0/403227232 >> 172.21.15.102:0/403227232 conn(0x7f11c00cf4e0 msgr2=0x7f11c00c3000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:30:55.031 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.036 7f11c5b02700 1 -- 172.21.15.102:0/403227232 shutdown_connections 2024-04-17T22:30:55.031 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.036 7f11c5b02700 1 -- 172.21.15.102:0/403227232 wait complete. 2024-04-17T22:30:55.031 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.037 7febba7fc700 1 -- 172.21.15.102:0/3314178118 <== osd.0 v1:172.21.15.102:6812/42955 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (unknown 0 0 2009191733) 0x7febbc0094a0 con 0x7feba4001650 2024-04-17T22:30:55.032 INFO:teuthology.orchestra.run.smithi102.stdout:38654705691 2024-04-17T22:30:55.032 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.037 7febd463e700 1 -- 172.21.15.102:0/3314178118 >> v1:172.21.15.102:6812/42955 conn(0x7feba4001650 legacy=0x7feba4003ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:30:55.032 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.037 7febd463e700 1 -- 172.21.15.102:0/3314178118 >> v1:172.21.15.102:6816/42360 conn(0x7febac0145a0 legacy=0x7febac016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:30:55.032 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.037 7febd463e700 1 -- 172.21.15.102:0/3314178118 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7febcc1a63a0 msgr2=0x7febcc1b93d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:30:55.032 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.037 7febd463e700 1 --2- 172.21.15.102:0/3314178118 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7febcc1a63a0 0x7febcc1b93d0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7febcc0f4630 tx=0x7febc4006c40).stop 2024-04-17T22:30:55.033 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.037 7febd1c3a700 1 -- 172.21.15.102:0/3314178118 reap_dead start 2024-04-17T22:30:55.033 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.037 7febd463e700 1 -- 172.21.15.102:0/3314178118 shutdown_connections 2024-04-17T22:30:55.033 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.037 7febd463e700 1 -- 172.21.15.102:0/3314178118 >> 172.21.15.102:0/3314178118 conn(0x7febcc1a4720 msgr2=0x7febcc0c6840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:30:55.033 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.037 7febd463e700 1 -- 172.21.15.102:0/3314178118 shutdown_connections 2024-04-17T22:30:55.033 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.037 7febd463e700 1 -- 172.21.15.102:0/3314178118 wait complete. 2024-04-17T22:30:55.033 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.040 7f53ca7fc700 1 -- 172.21.15.102:0/1455284617 <== osd.3 v1:172.21.15.102:6800/42953 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+5151 (unknown 0 0 4057655604) 0x7f53d80094a0 con 0x7f53dc0a7bd0 2024-04-17T22:30:55.034 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.042 7f4e769ef700 1 -- 172.21.15.102:0/1670183031 --> v1:172.21.15.102:6808/42958 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f4e4c007040 con 0x7f4e4c001650 2024-04-17T22:30:55.034 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.042 7f4e6cde6700 1 -- 172.21.15.102:0/1670183031 <== osd.1 v1:172.21.15.102:6808/42958 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (unknown 0 0 1684229442) 0x7f4e4c007040 con 0x7f4e4c001650 2024-04-17T22:30:55.034 INFO:teuthology.orchestra.run.smithi102.stdout:38654705690 2024-04-17T22:30:55.035 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.043 7f4e769ef700 1 -- 172.21.15.102:0/1670183031 >> v1:172.21.15.102:6808/42958 conn(0x7f4e4c001650 legacy=0x7f4e4c003ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:30:55.035 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.043 7f4e769ef700 1 -- 172.21.15.102:0/1670183031 >> v1:172.21.15.102:6816/42360 conn(0x7f4e500145a0 legacy=0x7f4e50016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:30:55.035 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.043 7f4e769ef700 1 -- 172.21.15.102:0/1670183031 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4e700f6c40 msgr2=0x7f4e701ad2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:30:55.035 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.043 7f4e769ef700 1 --2- 172.21.15.102:0/1670183031 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4e700f6c40 0x7f4e701ad2f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 rx=0x7f4e700e1350 tx=0x7f4e6400ecd0).stop 2024-04-17T22:30:55.035 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.043 7f4e6fdec700 1 -- 172.21.15.102:0/1670183031 reap_dead start 2024-04-17T22:30:55.035 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.043 7f4e769ef700 1 -- 172.21.15.102:0/1670183031 shutdown_connections 2024-04-17T22:30:55.035 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.043 7f4e769ef700 1 -- 172.21.15.102:0/1670183031 >> 172.21.15.102:0/1670183031 conn(0x7f4e700e0d50 msgr2=0x7f4e700daa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:30:55.035 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.043 7f4e769ef700 1 -- 172.21.15.102:0/1670183031 shutdown_connections 2024-04-17T22:30:55.035 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.043 7f4e769ef700 1 -- 172.21.15.102:0/1670183031 wait complete. 2024-04-17T22:30:55.036 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.047 7f53e473e700 1 -- 172.21.15.102:0/1455284617 --> v1:172.21.15.102:6800/42953 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f53dc0b4820 con 0x7f53dc0a7bd0 2024-04-17T22:30:55.036 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.048 7f53ca7fc700 1 -- 172.21.15.102:0/1455284617 <== osd.3 v1:172.21.15.102:6800/42953 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (unknown 0 0 2009191733) 0x7f53dc0b4820 con 0x7f53dc0a7bd0 2024-04-17T22:30:55.036 INFO:teuthology.orchestra.run.smithi102.stdout:38654705691 2024-04-17T22:30:55.037 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.048 7f53e473e700 1 -- 172.21.15.102:0/1455284617 >> v1:172.21.15.102:6800/42953 conn(0x7f53dc0a7bd0 legacy=0x7f53dc0a6f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:30:55.037 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.048 7f53e473e700 1 -- 172.21.15.102:0/1455284617 >> v1:172.21.15.102:6816/42360 conn(0x7f53bc0145a0 legacy=0x7f53bc016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:30:55.037 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.048 7f53e473e700 1 -- 172.21.15.102:0/1455284617 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f53dc0ccc40 msgr2=0x7f53dc0e0240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:30:55.037 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.048 7f53e473e700 1 --2- 172.21.15.102:0/1455284617 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f53dc0ccc40 0x7f53dc0e0240 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7f53dc0d2800 tx=0x7f53d4006c40).stop 2024-04-17T22:30:55.037 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.048 7f53e1d3a700 1 -- 172.21.15.102:0/1455284617 reap_dead start 2024-04-17T22:30:55.037 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.048 7f53e473e700 1 -- 172.21.15.102:0/1455284617 shutdown_connections 2024-04-17T22:30:55.037 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.048 7f53e473e700 1 -- 172.21.15.102:0/1455284617 >> 172.21.15.102:0/1455284617 conn(0x7f53dc0c54f0 msgr2=0x7f53dc0ce820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:30:55.037 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.048 7f53e473e700 1 -- 172.21.15.102:0/1455284617 shutdown_connections 2024-04-17T22:30:55.037 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.048 7f53e473e700 1 -- 172.21.15.102:0/1455284617 wait complete. 2024-04-17T22:30:55.038 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-17T22:30:55.039 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-17T22:30:55.043 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-17T22:30:55.046 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-17T22:30:55.175 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.188 7fd6a0a99700 1 Processor -- start 2024-04-17T22:30:55.175 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.189 7fd6a0a99700 1 -- start start 2024-04-17T22:30:55.175 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.189 7fd6a0a99700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd69c10e240 0x7fd69c10e690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:30:55.175 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.189 7fd6a0a99700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd69c0f1f30 0x7fd69c1ab4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:30:55.175 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.189 7fd6a0a99700 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd69c0f2380 con 0x7fd69c10ecc0 2024-04-17T22:30:55.176 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.189 7fd6a0a99700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd69c0f9f10 con 0x7fd69c0f1f30 2024-04-17T22:30:55.176 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.189 7fd6a0a99700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd69c0f9030 con 0x7fd69c10e240 2024-04-17T22:30:55.177 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.189 7fd69a051700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd69c0f1f30 0x7fd69c1ab4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:30:55.177 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.189 7fd69a051700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd69c0f1f30 0x7fd69c1ab4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:48686/0 (socket says 172.21.15.102:48686) 2024-04-17T22:30:55.177 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.189 7fd69a051700 1 -- 172.21.15.102:0/2746996375 learned_addr learned my addr 172.21.15.102:0/2746996375 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:30:55.177 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.190 7fd699850700 1 --2- 172.21.15.102:0/2746996375 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd69c10e240 0x7fd69c10e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:30:55.178 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.190 7fd69884e700 1 -- 172.21.15.102:0/2746996375 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2085711195 0 0) 0x7fd690001110 con 0x7fd69c10ecc0 2024-04-17T22:30:55.178 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.190 7fd69a051700 1 -- 172.21.15.102:0/2746996375 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd69c10e240 msgr2=0x7fd69c10e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:30:55.178 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.190 7fd69a051700 1 --2- 172.21.15.102:0/2746996375 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd69c10e240 0x7fd69c10e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:30:55.178 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.190 7fd69a051700 1 -- 172.21.15.102:0/2746996375 >> v1:172.21.15.102:6789/0 conn(0x7fd69c10ecc0 legacy=0x7fd69c0f1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:30:55.178 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.190 7fd69a051700 1 -- 172.21.15.102:0/2746996375 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd69c0f25a0 con 0x7fd69c0f1f30 2024-04-17T22:30:55.178 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.190 7fd69a051700 1 --2- 172.21.15.102:0/2746996375 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd69c0f1f30 0x7fd69c1ab4e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 rx=0x7fd69c0f46a0 tx=0x7fd694009e40).ready entity=mon.1 client_cookie=50a88d4a11070eb9 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:30:55.178 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.190 7fd69884e700 1 -- 172.21.15.102:0/2746996375 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd69c0f25a0 con 0x7fd69c0f1f30 2024-04-17T22:30:55.178 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.190 7fd69884e700 1 -- 172.21.15.102:0/2746996375 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd69400b870 con 0x7fd69c0f1f30 2024-04-17T22:30:55.178 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.190 7fd69884e700 1 -- 172.21.15.102:0/2746996375 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd69400ba50 con 0x7fd69c0f1f30 2024-04-17T22:30:55.178 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.190 7fd6a0a99700 1 -- 172.21.15.102:0/2746996375 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd69c0f1f30 msgr2=0x7fd69c1ab4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:30:55.178 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.190 7fd6a0a99700 1 --2- 172.21.15.102:0/2746996375 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd69c0f1f30 0x7fd69c1ab4e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 rx=0x7fd69c0f46a0 tx=0x7fd694009e40).stop 2024-04-17T22:30:55.179 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.191 7fd6a0a99700 1 -- 172.21.15.102:0/2746996375 shutdown_connections 2024-04-17T22:30:55.179 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.191 7fd6a0a99700 1 --2- 172.21.15.102:0/2746996375 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd69c0f1f30 0x7fd69c1ab4e0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:30:55.179 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.191 7fd6a0a99700 1 --2- 172.21.15.102:0/2746996375 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd69c10e240 0x7fd69c10e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:30:55.179 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.191 7fd6a0a99700 1 -- 172.21.15.102:0/2746996375 >> 172.21.15.102:0/2746996375 conn(0x7fd69c1138a0 msgr2=0x7fd69c0f4f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:30:55.179 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.191 7fd6a0a99700 1 -- 172.21.15.102:0/2746996375 shutdown_connections 2024-04-17T22:30:55.179 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.191 7fd6a0a99700 1 -- 172.21.15.102:0/2746996375 wait complete. 2024-04-17T22:30:55.181 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.192 7fbf2545b700 1 Processor -- start 2024-04-17T22:30:55.181 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.192 7fbf2545b700 1 -- start start 2024-04-17T22:30:55.181 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.192 7fbf2545b700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fbf200f5840 0x7fbf2010b010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:30:55.181 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.192 7fbf2545b700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fbf2010ba90 0x7fbf201a02f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:30:55.181 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.192 7fbf2545b700 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf20103b40 con 0x7fbf2010b640 2024-04-17T22:30:55.181 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.192 7fbf2545b700 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf2010bee0 con 0x7fbf200f5840 2024-04-17T22:30:55.181 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.192 7fbf2545b700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbf200fa370 con 0x7fbf2010ba90 2024-04-17T22:30:55.181 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.193 7fbf1e05c700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fbf200f5840 0x7fbf2010b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:30:55.181 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.193 7fbf1d85b700 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7fbf2010b640 0x7fbf2019faf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:40904/0 (socket says 172.21.15.102:40904) 2024-04-17T22:30:55.181 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.193 7fbf1d85b700 1 -- 172.21.15.102:0/403500122 learned_addr learned my addr 172.21.15.102:0/403500122 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-17T22:30:55.182 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.193 7fbf1e85d700 1 --2- 172.21.15.102:0/403500122 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fbf2010ba90 0x7fbf201a02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:30:55.182 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.193 7fbf1e05c700 1 -- 172.21.15.102:0/403500122 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fbf2010ba90 msgr2=0x7fbf201a02f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:30:55.182 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.193 7fbf1e05c700 1 --2- 172.21.15.102:0/403500122 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fbf2010ba90 0x7fbf201a02f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:30:55.182 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.193 7fbf1e05c700 1 -- 172.21.15.102:0/403500122 >> v1:172.21.15.102:6789/0 conn(0x7fbf2010b640 legacy=0x7fbf2019faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:30:55.182 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.193 7fbf1e05c700 1 -- 172.21.15.102:0/403500122 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf201a0920 con 0x7fbf200f5840 2024-04-17T22:30:55.182 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.193 7fbf1e85d700 1 --2- 172.21.15.102:0/403500122 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fbf2010ba90 0x7fbf201a02f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:30:55.182 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.193 7fbf1e05c700 1 --2- 172.21.15.102:0/403500122 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbf200f5840 0x7fbf2010b010 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 rx=0x7fbf20103d60 tx=0x7fbf1000ba20).ready entity=mon.0 client_cookie=93df4c13027c19b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:30:55.182 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.193 7fbf1d05a700 1 -- 172.21.15.102:0/403500122 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fbf201a0920 con 0x7fbf200f5840 2024-04-17T22:30:55.182 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.193 7fbf1d05a700 1 -- 172.21.15.102:0/403500122 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf10012aa0 con 0x7fbf200f5840 2024-04-17T22:30:55.182 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.193 7fbf1d05a700 1 -- 172.21.15.102:0/403500122 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fbf10017070 con 0x7fbf200f5840 2024-04-17T22:30:55.182 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.194 7fbf2545b700 1 -- 172.21.15.102:0/403500122 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbf200f5840 msgr2=0x7fbf2010b010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:30:55.182 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.194 7fbf2545b700 1 --2- 172.21.15.102:0/403500122 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbf200f5840 0x7fbf2010b010 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 rx=0x7fbf20103d60 tx=0x7fbf1000ba20).stop 2024-04-17T22:30:55.183 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.194 7fbf2545b700 1 -- 172.21.15.102:0/403500122 shutdown_connections 2024-04-17T22:30:55.183 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.194 7fbf2545b700 1 --2- 172.21.15.102:0/403500122 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fbf2010ba90 0x7fbf201a02f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:30:55.183 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.194 7fbf2545b700 1 --2- 172.21.15.102:0/403500122 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbf200f5840 0x7fbf2010b010 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:30:55.183 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.194 7fbf2545b700 1 -- 172.21.15.102:0/403500122 >> 172.21.15.102:0/403500122 conn(0x7fbf201138a0 msgr2=0x7fbf200f4f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:30:55.183 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.194 7fbf2545b700 1 -- 172.21.15.102:0/403500122 shutdown_connections 2024-04-17T22:30:55.183 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.194 7fbf2545b700 1 -- 172.21.15.102:0/403500122 wait complete. 2024-04-17T22:30:55.203 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.217 7fd6a0a99700 1 Processor -- start 2024-04-17T22:30:55.203 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.217 7fd6a0a99700 1 -- start start 2024-04-17T22:30:55.204 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.217 7fd6a0a99700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd69c0d5eb0 0x7fd69c1ad1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:30:55.204 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.217 7fd6a0a99700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd69c1ad7d0 0x7fd69c1b3c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:30:55.204 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.217 7fd6a0a99700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd69c1adc20 0x7fd69c1b4260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:30:55.204 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.217 7fd6a0a99700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd69c08ac90 con 0x7fd69c1adc20 2024-04-17T22:30:55.204 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.217 7fd6a0a99700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd69c0fa1b0 con 0x7fd69c1ad7d0 2024-04-17T22:30:55.204 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.217 7fd6a0a99700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd69c0d6f40 con 0x7fd69c0d5eb0 2024-04-17T22:30:55.204 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.217 7fd699850700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd69c0d5eb0 0x7fd69c1ad1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:30:55.205 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.217 7fd699850700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd69c0d5eb0 0x7fd69c1ad1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:59086/0 (socket says 172.21.15.102:59086) 2024-04-17T22:30:55.205 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.217 7fd699850700 1 -- 172.21.15.102:0/1575495704 learned_addr learned my addr 172.21.15.102:0/1575495704 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:30:55.205 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.217 7fd69a051700 1 --2- 172.21.15.102:0/1575495704 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd69c1adc20 0x7fd69c1b4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:30:55.207 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.218 7fd699850700 1 -- 172.21.15.102:0/1575495704 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd69c1ad7d0 msgr2=0x7fd69c1b3c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:30:55.207 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.218 7fd699850700 1 --2- 172.21.15.102:0/1575495704 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd69c1ad7d0 0x7fd69c1b3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-04-17T22:30:55.207 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.218 7fd699850700 1 -- 172.21.15.102:0/1575495704 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd69c1adc20 msgr2=0x7fd69c1b4260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:30:55.208 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.218 7fd699850700 1 --2- 172.21.15.102:0/1575495704 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd69c1adc20 0x7fd69c1b4260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:30:55.208 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.218 7fd699850700 1 -- 172.21.15.102:0/1575495704 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd69c0d6560 con 0x7fd69c0d5eb0 2024-04-17T22:30:55.208 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.218 7fd69a051700 1 --2- 172.21.15.102:0/1575495704 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd69c1adc20 0x7fd69c1b4260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:30:55.208 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.218 7fd699850700 1 --2- 172.21.15.102:0/1575495704 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd69c0d5eb0 0x7fd69c1ad1a0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0x7fd69c0e66c0 tx=0x7fd68c00dc10).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:30:55.208 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.218 7fd68affd700 1 -- 172.21.15.102:0/1575495704 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd69c0d6560 con 0x7fd69c0d5eb0 2024-04-17T22:30:55.208 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.218 7fd68affd700 1 -- 172.21.15.102:0/1575495704 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd68c009780 con 0x7fd69c0d5eb0 2024-04-17T22:30:55.208 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.218 7fd6a0a99700 1 -- 172.21.15.102:0/1575495704 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd69c1b4890 con 0x7fd69c0d5eb0 2024-04-17T22:30:55.208 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.218 7fd68affd700 1 -- 172.21.15.102:0/1575495704 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd68c00f040 con 0x7fd69c0d5eb0 2024-04-17T22:30:55.208 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.218 7fd6a0a99700 1 -- 172.21.15.102:0/1575495704 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd69c1bb500 con 0x7fd69c0d5eb0 2024-04-17T22:30:55.208 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.219 7fd68affd700 1 -- 172.21.15.102:0/1575495704 <== mon.2 v2:172.21.15.102:3302/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7fd68c009b50 con 0x7fd69c0d5eb0 2024-04-17T22:30:55.208 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.220 7fd6a0a99700 1 -- 172.21.15.102:0/1575495704 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd69c0f0800 con 0x7fd69c0d5eb0 2024-04-17T22:30:55.208 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.220 7fd68affd700 1 -- 172.21.15.102:0/1575495704 <== mon.2 v2:172.21.15.102:3302/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3194+0+0 (secure 0 0 0) 0x7fd68c010070 con 0x7fd69c0d5eb0 2024-04-17T22:30:55.209 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.223 7fd68affd700 1 -- 172.21.15.102:0/1575495704 <== mon.2 v2:172.21.15.102:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+110455 (secure 0 0 0) 0x7fd68c00dd50 con 0x7fd69c0d5eb0 2024-04-17T22:30:55.214 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.226 7fbf2545b700 1 Processor -- start 2024-04-17T22:30:55.214 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.226 7fbf2545b700 1 -- start start 2024-04-17T22:30:55.214 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.226 7fbf2545b700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fbf201a9000 0x7fbf201a9450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:30:55.214 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.226 7fbf2545b700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fbf201a9a80 0x7fbf201afa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:30:55.214 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.226 7fbf2545b700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbf201b0070 0x7fbf201b04c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:30:55.215 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.226 7fbf2545b700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf200d9200 con 0x7fbf201b0070 2024-04-17T22:30:55.215 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.226 7fbf2545b700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf200d9850 con 0x7fbf201a9000 2024-04-17T22:30:55.215 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.226 7fbf2545b700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbf200d9500 con 0x7fbf201a9a80 2024-04-17T22:30:55.215 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.227 7fbf1d85b700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fbf201a9a80 0x7fbf201afa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:30:55.215 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.227 7fbf1e05c700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fbf201a9000 0x7fbf201a9450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:30:55.215 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.227 7fbf1d85b700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fbf201a9a80 0x7fbf201afa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:59096/0 (socket says 172.21.15.102:59096) 2024-04-17T22:30:55.215 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.227 7fbf1d85b700 1 -- 172.21.15.102:0/382970588 learned_addr learned my addr 172.21.15.102:0/382970588 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:30:55.215 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.227 7fbf1e05c700 1 --2- 172.21.15.102:0/382970588 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fbf201a9000 0x7fbf201a9450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:48694/0 (socket says 172.21.15.102:48694) 2024-04-17T22:30:55.215 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.227 7fbf1d85b700 1 -- 172.21.15.102:0/382970588 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fbf201a9000 msgr2=0x7fbf201a9450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:30:55.215 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.227 7fbf1d85b700 1 --2- 172.21.15.102:0/382970588 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fbf201a9000 0x7fbf201a9450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:30:55.215 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.227 7fbf1d85b700 1 -- 172.21.15.102:0/382970588 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbf201b0070 msgr2=0x7fbf201b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:30:55.215 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.227 7fbf1d85b700 1 --2- 172.21.15.102:0/382970588 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbf201b0070 0x7fbf201b04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-04-17T22:30:55.215 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.227 7fbf1d85b700 1 -- 172.21.15.102:0/382970588 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf200cac20 con 0x7fbf201a9a80 2024-04-17T22:30:55.215 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.227 7fbf1e05c700 1 --2- 172.21.15.102:0/382970588 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fbf201a9000 0x7fbf201a9450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:30:55.216 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.227 7fbf1d85b700 1 --2- 172.21.15.102:0/382970588 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fbf201a9a80 0x7fbf201afa40 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0x7fbf200c4380 tx=0x7fbf14007f70).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:30:55.216 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.227 7fbf0f7fe700 1 -- 172.21.15.102:0/382970588 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fbf200cac20 con 0x7fbf201a9a80 2024-04-17T22:30:55.216 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.227 7fbf0f7fe700 1 -- 172.21.15.102:0/382970588 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf140072e0 con 0x7fbf201a9a80 2024-04-17T22:30:55.216 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.227 7fbf2545b700 1 -- 172.21.15.102:0/382970588 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf201b7130 con 0x7fbf201a9a80 2024-04-17T22:30:55.216 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.227 7fbf0f7fe700 1 -- 172.21.15.102:0/382970588 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fbf14005810 con 0x7fbf201a9a80 2024-04-17T22:30:55.216 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.227 7fbf2545b700 1 -- 172.21.15.102:0/382970588 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf201b7510 con 0x7fbf201a9a80 2024-04-17T22:30:55.216 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.228 7fbf0f7fe700 1 -- 172.21.15.102:0/382970588 <== mon.2 v2:172.21.15.102:3302/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7fbf14005a10 con 0x7fbf201a9a80 2024-04-17T22:30:55.216 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.228 7fbf0f7fe700 1 -- 172.21.15.102:0/382970588 <== mon.2 v2:172.21.15.102:3302/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3194+0+0 (secure 0 0 0) 0x7fbf14014800 con 0x7fbf201a9a80 2024-04-17T22:30:55.216 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.229 7fbf2545b700 1 -- 172.21.15.102:0/382970588 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf200f0800 con 0x7fbf201a9a80 2024-04-17T22:30:55.218 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.232 7fbf0f7fe700 1 -- 172.21.15.102:0/382970588 <== mon.2 v2:172.21.15.102:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+110455 (secure 0 0 0) 0x7fbf140070d0 con 0x7fbf201a9a80 2024-04-17T22:30:55.221 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.235 7f60c3cbb700 1 Processor -- start 2024-04-17T22:30:55.221 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.235 7f60c3cbb700 1 -- start start 2024-04-17T22:30:55.222 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.236 7f60c3cbb700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f60bc0f5840 0x7f60bc10b010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:30:55.222 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.236 7f60c3cbb700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f60bc10ba90 0x7f60bc1a02f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:30:55.222 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.236 7f60c3cbb700 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60bc103b40 con 0x7f60bc10b640 2024-04-17T22:30:55.222 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.236 7f60c3cbb700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60bc10bee0 con 0x7f60bc0f5840 2024-04-17T22:30:55.222 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.236 7f60c3cbb700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f60bc0fa370 con 0x7f60bc10ba90 2024-04-17T22:30:55.223 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.236 7f60c12b7700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f60bc10ba90 0x7f60bc1a02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:30:55.223 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.236 7f60c12b7700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f60bc10ba90 0x7f60bc1a02f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:59112/0 (socket says 172.21.15.102:59112) 2024-04-17T22:30:55.223 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.236 7f60c12b7700 1 -- 172.21.15.102:0/126878641 learned_addr learned my addr 172.21.15.102:0/126878641 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:30:55.223 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.236 7f60bbfff700 1 --1- 172.21.15.102:0/126878641 >> v1:172.21.15.102:6789/0 conn(0x7f60bc10b640 0x7f60bc19faf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:40914/0 (socket says 172.21.15.102:40914) 2024-04-17T22:30:55.223 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.236 7f60c0ab6700 1 --2- 172.21.15.102:0/126878641 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f60bc0f5840 0x7f60bc10b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:30:55.224 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.236 7f60c12b7700 1 -- 172.21.15.102:0/126878641 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f60bc0f5840 msgr2=0x7f60bc10b010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:30:55.224 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.236 7f60c12b7700 1 --2- 172.21.15.102:0/126878641 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f60bc0f5840 0x7f60bc10b010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:30:55.224 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.236 7f60c12b7700 1 -- 172.21.15.102:0/126878641 >> v1:172.21.15.102:6789/0 conn(0x7f60bc10b640 legacy=0x7f60bc19faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:30:55.224 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.236 7f60c12b7700 1 -- 172.21.15.102:0/126878641 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60bc1a0920 con 0x7f60bc10ba90 2024-04-17T22:30:55.224 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.236 7f60c0ab6700 1 --2- 172.21.15.102:0/126878641 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f60bc0f5840 0x7f60bc10b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T22:30:55.224 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.237 7f60c12b7700 1 --2- 172.21.15.102:0/126878641 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f60bc10ba90 0x7f60bc1a02f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7f60bc0f46a0 tx=0x7f60b400d5d0).ready entity=mon.2 client_cookie=a060d9bdeb6d9af9 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:30:55.224 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.237 7f60bb7fe700 1 -- 172.21.15.102:0/126878641 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f60bc1a0920 con 0x7f60bc10ba90 2024-04-17T22:30:55.225 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.237 7f60bb7fe700 1 -- 172.21.15.102:0/126878641 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60b4010660 con 0x7f60bc10ba90 2024-04-17T22:30:55.225 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.237 7f60bb7fe700 1 -- 172.21.15.102:0/126878641 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f60b4010a30 con 0x7f60bc10ba90 2024-04-17T22:30:55.225 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.237 7f60c3cbb700 1 -- 172.21.15.102:0/126878641 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f60bc10ba90 msgr2=0x7f60bc1a02f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:30:55.225 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.237 7f60c3cbb700 1 --2- 172.21.15.102:0/126878641 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f60bc10ba90 0x7f60bc1a02f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7f60bc0f46a0 tx=0x7f60b400d5d0).stop 2024-04-17T22:30:55.225 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.238 7f60c3cbb700 1 -- 172.21.15.102:0/126878641 shutdown_connections 2024-04-17T22:30:55.225 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.238 7f60c3cbb700 1 --2- 172.21.15.102:0/126878641 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f60bc10ba90 0x7f60bc1a02f0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:30:55.225 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.238 7f60c3cbb700 1 --2- 172.21.15.102:0/126878641 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f60bc0f5840 0x7f60bc10b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:30:55.225 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.238 7f60c3cbb700 1 -- 172.21.15.102:0/126878641 >> 172.21.15.102:0/126878641 conn(0x7f60bc1138a0 msgr2=0x7f60bc0f4f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:30:55.225 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.238 7f60c3cbb700 1 -- 172.21.15.102:0/126878641 shutdown_connections 2024-04-17T22:30:55.225 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.238 7f60c3cbb700 1 -- 172.21.15.102:0/126878641 wait complete. 2024-04-17T22:30:55.232 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.245 7f1f188b7700 1 Processor -- start 2024-04-17T22:30:55.232 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.246 7f1f188b7700 1 -- start start 2024-04-17T22:30:55.233 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.246 7f1f188b7700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f1f140f5750 0x7f1f1410c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:30:55.233 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.246 7f1f188b7700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f1f1410c720 0x7f1f141a0be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:30:55.233 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.246 7f1f188b7700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f1f1410cb70 0x7f1f141a1210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:30:55.233 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.246 7f1f188b7700 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f140f93c0 con 0x7f1f1410c720 2024-04-17T22:30:55.233 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.246 7f1f188b7700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f141a1840 con 0x7f1f140f5750 2024-04-17T22:30:55.233 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.246 7f1f188b7700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1f141a1a70 con 0x7f1f1410cb70 2024-04-17T22:30:55.233 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.246 7f1f10dfa700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f1f1410c720 0x7f1f141a0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:30:55.233 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.247 7f1f10dfa700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f1f1410c720 0x7f1f141a0be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:40296/0 (socket says 172.21.15.102:40296) 2024-04-17T22:30:55.233 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.247 7f1f10dfa700 1 -- 172.21.15.102:0/1023264860 learned_addr learned my addr 172.21.15.102:0/1023264860 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:30:55.233 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.247 7f1f10dfa700 1 -- 172.21.15.102:0/1023264860 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f1f1410cb70 msgr2=0x7f1f141a1210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:30:55.234 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.247 7f1f10dfa700 1 --2- 172.21.15.102:0/1023264860 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f1f1410cb70 0x7f1f141a1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2024-04-17T22:30:55.234 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.247 7f1f10dfa700 1 -- 172.21.15.102:0/1023264860 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f1f140f5750 msgr2=0x7f1f1410c0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:30:55.234 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.247 7f1f10dfa700 1 --2- 172.21.15.102:0/1023264860 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f1f140f5750 0x7f1f1410c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2024-04-17T22:30:55.234 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.247 7f1f10dfa700 1 -- 172.21.15.102:0/1023264860 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f141a82f0 con 0x7f1f1410c720 2024-04-17T22:30:55.234 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.247 7f1f10dfa700 1 --2- 172.21.15.102:0/1023264860 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1f1410c720 0x7f1f141a0be0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 rx=0x7f1f04002f70 tx=0x7f1f04009a50).ready entity=mon.0 client_cookie=c9ac57bc7cba3158 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:30:55.235 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.247 7f1f03fff700 1 -- 172.21.15.102:0/1023264860 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f1f141a82f0 con 0x7f1f1410c720 2024-04-17T22:30:55.235 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.247 7f1f03fff700 1 -- 172.21.15.102:0/1023264860 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f0400f510 con 0x7f1f1410c720 2024-04-17T22:30:55.235 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.247 7f1f03fff700 1 -- 172.21.15.102:0/1023264860 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f1f0400f6f0 con 0x7f1f1410c720 2024-04-17T22:30:55.235 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.247 7f1f188b7700 1 -- 172.21.15.102:0/1023264860 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1f1410c720 msgr2=0x7f1f141a0be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:30:55.235 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.247 7f1f188b7700 1 --2- 172.21.15.102:0/1023264860 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1f1410c720 0x7f1f141a0be0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 rx=0x7f1f04002f70 tx=0x7f1f04009a50).stop 2024-04-17T22:30:55.235 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.248 7f1f188b7700 1 -- 172.21.15.102:0/1023264860 shutdown_connections 2024-04-17T22:30:55.235 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.248 7f1f188b7700 1 --2- 172.21.15.102:0/1023264860 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f1f1410cb70 0x7f1f141a1210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2024-04-17T22:30:55.235 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.248 7f1f188b7700 1 --2- 172.21.15.102:0/1023264860 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f1f140f5750 0x7f1f1410c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2024-04-17T22:30:55.235 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.248 7f1f188b7700 1 --2- 172.21.15.102:0/1023264860 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1f1410c720 0x7f1f141a0be0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:30:55.235 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.248 7f1f188b7700 1 -- 172.21.15.102:0/1023264860 >> 172.21.15.102:0/1023264860 conn(0x7f1f141138a0 msgr2=0x7f1f140f4f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:30:55.235 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.248 7f1f188b7700 1 -- 172.21.15.102:0/1023264860 shutdown_connections 2024-04-17T22:30:55.235 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.248 7f1f188b7700 1 -- 172.21.15.102:0/1023264860 wait complete. 2024-04-17T22:30:55.259 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.273 7f60c3cbb700 1 Processor -- start 2024-04-17T22:30:55.259 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.273 7f60c3cbb700 1 -- start start 2024-04-17T22:30:55.260 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.273 7f60c3cbb700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f60bc1a94c0 0x7f60bc1a9910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:30:55.260 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.273 7f60c3cbb700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f60bc1af9e0 0x7f60bc1afe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:30:55.261 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.273 7f60c3cbb700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f60bc1b0460 0x7f60bc1b6ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:30:55.261 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.273 7f60c3cbb700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60bc0c3090 con 0x7f60bc1af9e0 2024-04-17T22:30:55.261 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.273 7f60c3cbb700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60bc0d9420 con 0x7f60bc1b0460 2024-04-17T22:30:55.261 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.273 7f60c3cbb700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f60bc1a9f40 con 0x7f60bc1a94c0 2024-04-17T22:30:55.261 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.273 7f60c12b7700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f60bc1b0460 0x7f60bc1b6ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:30:55.261 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.273 7f60c0ab6700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f60bc1a94c0 0x7f60bc1a9910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:30:55.261 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.273 7f60c12b7700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f60bc1b0460 0x7f60bc1b6ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:48710/0 (socket says 172.21.15.102:48710) 2024-04-17T22:30:55.261 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.273 7f60c12b7700 1 -- 172.21.15.102:0/2118805974 learned_addr learned my addr 172.21.15.102:0/2118805974 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:30:55.261 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.273 7f60bbfff700 1 --2- 172.21.15.102:0/2118805974 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f60bc1af9e0 0x7f60bc1afe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:30:55.261 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.274 7f60c12b7700 1 -- 172.21.15.102:0/2118805974 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f60bc1a94c0 msgr2=0x7f60bc1a9910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:30:55.261 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.274 7f60c12b7700 1 --2- 172.21.15.102:0/2118805974 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f60bc1a94c0 0x7f60bc1a9910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:30:55.261 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.274 7f60c12b7700 1 -- 172.21.15.102:0/2118805974 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f60bc1af9e0 msgr2=0x7f60bc1afe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:30:55.261 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.274 7f60c12b7700 1 --2- 172.21.15.102:0/2118805974 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f60bc1af9e0 0x7f60bc1afe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:30:55.261 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.274 7f60c12b7700 1 -- 172.21.15.102:0/2118805974 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60bc0d9660 con 0x7f60bc1b0460 2024-04-17T22:30:55.261 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.274 7f60c0ab6700 1 --2- 172.21.15.102:0/2118805974 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f60bc1a94c0 0x7f60bc1a9910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:30:55.262 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.274 7f60c12b7700 1 --2- 172.21.15.102:0/2118805974 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f60bc1b0460 0x7f60bc1b6ec0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0x7f60bc0c4170 tx=0x7f60b4006c40).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:30:55.262 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.274 7f60b9ffb700 1 -- 172.21.15.102:0/2118805974 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f60b4018a20 con 0x7f60bc1b0460 2024-04-17T22:30:55.262 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.274 7f60b9ffb700 1 -- 172.21.15.102:0/2118805974 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60b401f070 con 0x7f60bc1b0460 2024-04-17T22:30:55.262 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.274 7f60b9ffb700 1 -- 172.21.15.102:0/2118805974 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f60b4023440 con 0x7f60bc1b0460 2024-04-17T22:30:55.262 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.274 7f60c3cbb700 1 -- 172.21.15.102:0/2118805974 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60bc1b08b0 con 0x7f60bc1b0460 2024-04-17T22:30:55.262 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.274 7f60c3cbb700 1 -- 172.21.15.102:0/2118805974 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60bc1b74f0 con 0x7f60bc1b0460 2024-04-17T22:30:55.262 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.275 7f60b9ffb700 1 -- 172.21.15.102:0/2118805974 <== mon.1 v2:172.21.15.102:3301/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7f60b402a060 con 0x7f60bc1b0460 2024-04-17T22:30:55.263 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.275 7f60b9ffb700 1 -- 172.21.15.102:0/2118805974 <== mon.1 v2:172.21.15.102:3301/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3194+0+0 (secure 0 0 0) 0x7f60b4020c30 con 0x7f60bc1b0460 2024-04-17T22:30:55.263 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.276 7f60c3cbb700 1 -- 172.21.15.102:0/2118805974 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60bc0f0800 con 0x7f60bc1b0460 2024-04-17T22:30:55.268 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.282 7f60b9ffb700 1 -- 172.21.15.102:0/2118805974 <== mon.1 v2:172.21.15.102:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+110455 (secure 0 0 0) 0x7f60b400dc60 con 0x7f60bc1b0460 2024-04-17T22:30:55.272 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.284 7f1f188b7700 1 Processor -- start 2024-04-17T22:30:55.272 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.284 7f1f188b7700 1 -- start start 2024-04-17T22:30:55.272 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.284 7f1f188b7700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f1f141a60e0 0x7f1f140d9180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:30:55.272 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.284 7f1f188b7700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f1f140d97b0 0x7f1f141ad440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:30:55.272 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.284 7f1f188b7700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1f140d9c00 0x7f1f141ada70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:30:55.272 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.284 7f1f188b7700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f140f7e00 con 0x7f1f140d9c00 2024-04-17T22:30:55.272 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.284 7f1f188b7700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f141ae0a0 con 0x7f1f141a60e0 2024-04-17T22:30:55.272 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.284 7f1f188b7700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1f141bb590 con 0x7f1f140d97b0 2024-04-17T22:30:55.273 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.285 7f1f10dfa700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f1f140d97b0 0x7f1f141ad440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:30:55.273 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.285 7f1f10dfa700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f1f140d97b0 0x7f1f141ad440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:59134/0 (socket says 172.21.15.102:59134) 2024-04-17T22:30:55.273 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.285 7f1f10dfa700 1 -- 172.21.15.102:0/4170237831 learned_addr learned my addr 172.21.15.102:0/4170237831 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:30:55.273 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.285 7f1f11dfc700 1 --2- 172.21.15.102:0/4170237831 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1f140d9c00 0x7f1f141ada70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:30:55.273 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.285 7f1f10dfa700 1 -- 172.21.15.102:0/4170237831 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f1f141a60e0 msgr2=0x7f1f140d9180 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-17T22:30:55.273 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.285 7f1f115fb700 1 --2- 172.21.15.102:0/4170237831 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f1f141a60e0 0x7f1f140d9180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:30:55.273 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.285 7f1f10dfa700 1 --2- 172.21.15.102:0/4170237831 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f1f141a60e0 0x7f1f140d9180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:30:55.273 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.285 7f1f10dfa700 1 -- 172.21.15.102:0/4170237831 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1f140d9c00 msgr2=0x7f1f141ada70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:30:55.273 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.285 7f1f10dfa700 1 --2- 172.21.15.102:0/4170237831 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1f140d9c00 0x7f1f141ada70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:30:55.273 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.285 7f1f10dfa700 1 -- 172.21.15.102:0/4170237831 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f141bb740 con 0x7f1f140d97b0 2024-04-17T22:30:55.273 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.285 7f1f11dfc700 1 --2- 172.21.15.102:0/4170237831 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1f140d9c00 0x7f1f141ada70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:30:55.273 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.285 7f1f115fb700 1 --2- 172.21.15.102:0/4170237831 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f1f141a60e0 0x7f1f140d9180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-04-17T22:30:55.273 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.285 7f1f10dfa700 1 --2- 172.21.15.102:0/4170237831 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f1f140d97b0 0x7f1f141ad440 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0x7f1f04002f70 tx=0x7f1f040068c0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:30:55.273 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.285 7f1f027fc700 1 -- 172.21.15.102:0/4170237831 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f1f141bb740 con 0x7f1f140d97b0 2024-04-17T22:30:55.273 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.285 7f1f027fc700 1 -- 172.21.15.102:0/4170237831 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f04011980 con 0x7f1f140d97b0 2024-04-17T22:30:55.274 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.285 7f1f027fc700 1 -- 172.21.15.102:0/4170237831 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f1f04011d50 con 0x7f1f140d97b0 2024-04-17T22:30:55.274 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.285 7f1f188b7700 1 -- 172.21.15.102:0/4170237831 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f141bb940 con 0x7f1f140d97b0 2024-04-17T22:30:55.274 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.286 7f1f188b7700 1 -- 172.21.15.102:0/4170237831 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f141bbd20 con 0x7f1f140d97b0 2024-04-17T22:30:55.274 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.286 7f1f027fc700 1 -- 172.21.15.102:0/4170237831 <== mon.2 v2:172.21.15.102:3302/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7f1f04019030 con 0x7f1f140d97b0 2024-04-17T22:30:55.274 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.286 7f1f027fc700 1 -- 172.21.15.102:0/4170237831 <== mon.2 v2:172.21.15.102:3302/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3194+0+0 (secure 0 0 0) 0x7f1f04018040 con 0x7f1f140d97b0 2024-04-17T22:30:55.274 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.287 7f1f188b7700 1 -- 172.21.15.102:0/4170237831 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f140f0800 con 0x7f1f140d97b0 2024-04-17T22:30:55.279 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.293 7f1f027fc700 1 -- 172.21.15.102:0/4170237831 <== mon.2 v2:172.21.15.102:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+110455 (secure 0 0 0) 0x7f1f04011770 con 0x7f1f140d97b0 2024-04-17T22:30:55.328 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.342 7fd6a0a99700 1 -- 172.21.15.102:0/1575495704 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fd69c1ae350 con 0x7fd69c0d5eb0 2024-04-17T22:30:55.328 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.342 7fd68affd700 1 -- 172.21.15.102:0/1575495704 <== mon.2 v2:172.21.15.102:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fd68c000c30 con 0x7fd69c0d5eb0 2024-04-17T22:30:55.329 INFO:teuthology.orchestra.run.smithi102.stdout:38654705690 2024-04-17T22:30:55.331 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.344 7fd6a0a99700 1 -- 172.21.15.102:0/1575495704 >> v1:172.21.15.102:6816/42360 conn(0x7fd6780145a0 legacy=0x7fd678016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:30:55.331 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.344 7fd6a0a99700 1 -- 172.21.15.102:0/1575495704 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd69c0d5eb0 msgr2=0x7fd69c1ad1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:30:55.331 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.344 7fd6a0a99700 1 --2- 172.21.15.102:0/1575495704 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd69c0d5eb0 0x7fd69c1ad1a0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0x7fd69c0e66c0 tx=0x7fd68c00dc10).stop 2024-04-17T22:30:55.331 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.344 7fd6a0a99700 1 -- 172.21.15.102:0/1575495704 shutdown_connections 2024-04-17T22:30:55.332 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.344 7fd6a0a99700 1 --2- 172.21.15.102:0/1575495704 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd69c1adc20 0x7fd69c1b4260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:30:55.332 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.344 7fd6a0a99700 1 --2- 172.21.15.102:0/1575495704 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd69c0d5eb0 0x7fd69c1ad1a0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:30:55.332 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.344 7fd6a0a99700 1 --2- 172.21.15.102:0/1575495704 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd69c1ad7d0 0x7fd69c1b3c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-04-17T22:30:55.332 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.344 7fd6a0a99700 1 -- 172.21.15.102:0/1575495704 >> 172.21.15.102:0/1575495704 conn(0x7fd69c0c5aa0 msgr2=0x7fd69c1a7b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:30:55.332 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.344 7fd6a0a99700 1 -- 172.21.15.102:0/1575495704 shutdown_connections 2024-04-17T22:30:55.332 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.344 7fd6a0a99700 1 -- 172.21.15.102:0/1575495704 wait complete. 2024-04-17T22:30:55.336 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.350 7fbf2545b700 1 -- 172.21.15.102:0/382970588 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fbf201aa160 con 0x7fbf201a9a80 2024-04-17T22:30:55.336 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.350 7fbf0f7fe700 1 -- 172.21.15.102:0/382970588 <== mon.2 v2:172.21.15.102:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fbf14000c30 con 0x7fbf201a9a80 2024-04-17T22:30:55.337 INFO:teuthology.orchestra.run.smithi102.stdout:38654705690 2024-04-17T22:30:55.338 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.352 7fbf2545b700 1 -- 172.21.15.102:0/382970588 >> v1:172.21.15.102:6816/42360 conn(0x7fbefc0145a0 legacy=0x7fbefc016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:30:55.339 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.352 7fbf2545b700 1 -- 172.21.15.102:0/382970588 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fbf201a9a80 msgr2=0x7fbf201afa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:30:55.339 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.352 7fbf2545b700 1 --2- 172.21.15.102:0/382970588 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fbf201a9a80 0x7fbf201afa40 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0x7fbf200c4380 tx=0x7fbf14007f70).stop 2024-04-17T22:30:55.339 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.352 7fbf2545b700 1 -- 172.21.15.102:0/382970588 shutdown_connections 2024-04-17T22:30:55.339 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.352 7fbf2545b700 1 --2- 172.21.15.102:0/382970588 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbf201b0070 0x7fbf201b04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-04-17T22:30:55.339 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.352 7fbf2545b700 1 --2- 172.21.15.102:0/382970588 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fbf201a9000 0x7fbf201a9450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:30:55.339 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.352 7fbf2545b700 1 --2- 172.21.15.102:0/382970588 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fbf201a9a80 0x7fbf201afa40 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:30:55.339 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.352 7fbf2545b700 1 -- 172.21.15.102:0/382970588 >> 172.21.15.102:0/382970588 conn(0x7fbf200e0e70 msgr2=0x7fbf200c4950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:30:55.340 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.352 7fbf2545b700 1 -- 172.21.15.102:0/382970588 shutdown_connections 2024-04-17T22:30:55.340 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.352 7fbf2545b700 1 -- 172.21.15.102:0/382970588 wait complete. 2024-04-17T22:30:55.348 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705691 got 38654705690 for osd.0 2024-04-17T22:30:55.350 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705691 got 38654705690 for osd.2 2024-04-17T22:30:55.383 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.397 7f60c3cbb700 1 -- 172.21.15.102:0/2118805974 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f60bc1aa120 con 0x7f60bc1b0460 2024-04-17T22:30:55.383 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.397 7f60b9ffb700 1 -- 172.21.15.102:0/2118805974 <== mon.1 v2:172.21.15.102:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f60b4018810 con 0x7f60bc1b0460 2024-04-17T22:30:55.383 INFO:teuthology.orchestra.run.smithi102.stdout:38654705691 2024-04-17T22:30:55.385 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.398 7f60c3cbb700 1 -- 172.21.15.102:0/2118805974 >> v1:172.21.15.102:6816/42360 conn(0x7f60a40145a0 legacy=0x7f60a4016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:30:55.385 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.398 7f60c3cbb700 1 -- 172.21.15.102:0/2118805974 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f60bc1b0460 msgr2=0x7f60bc1b6ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:30:55.385 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.398 7f60c3cbb700 1 --2- 172.21.15.102:0/2118805974 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f60bc1b0460 0x7f60bc1b6ec0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0x7f60bc0c4170 tx=0x7f60b4006c40).stop 2024-04-17T22:30:55.385 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.399 7f60c3cbb700 1 -- 172.21.15.102:0/2118805974 shutdown_connections 2024-04-17T22:30:55.386 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.399 7f60c3cbb700 1 --2- 172.21.15.102:0/2118805974 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f60bc1b0460 0x7f60bc1b6ec0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:30:55.386 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.399 7f60c3cbb700 1 --2- 172.21.15.102:0/2118805974 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f60bc1a94c0 0x7f60bc1a9910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:30:55.386 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.399 7f60c3cbb700 1 --2- 172.21.15.102:0/2118805974 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f60bc1af9e0 0x7f60bc1afe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:30:55.386 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.399 7f60c3cbb700 1 -- 172.21.15.102:0/2118805974 >> 172.21.15.102:0/2118805974 conn(0x7f60bc0e0d10 msgr2=0x7f60bc0c47b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:30:55.386 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.399 7f60c3cbb700 1 -- 172.21.15.102:0/2118805974 shutdown_connections 2024-04-17T22:30:55.386 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.399 7f60c3cbb700 1 -- 172.21.15.102:0/2118805974 wait complete. 2024-04-17T22:30:55.399 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705691 got 38654705691 for osd.3 2024-04-17T22:30:55.399 DEBUG:teuthology.parallel:result is None 2024-04-17T22:30:55.405 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.419 7f1f188b7700 1 -- 172.21.15.102:0/4170237831 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f1f141ae2d0 con 0x7f1f140d97b0 2024-04-17T22:30:55.405 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.419 7f1f027fc700 1 -- 172.21.15.102:0/4170237831 <== mon.2 v2:172.21.15.102:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f1f0400f300 con 0x7f1f140d97b0 2024-04-17T22:30:55.405 INFO:teuthology.orchestra.run.smithi102.stdout:38654705690 2024-04-17T22:30:55.407 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.420 7f1f188b7700 1 -- 172.21.15.102:0/4170237831 >> v1:172.21.15.102:6816/42360 conn(0x7f1ef80145a0 legacy=0x7f1ef8016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:30:55.407 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.420 7f1f188b7700 1 -- 172.21.15.102:0/4170237831 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f1f140d97b0 msgr2=0x7f1f141ad440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:30:55.407 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.420 7f1f188b7700 1 --2- 172.21.15.102:0/4170237831 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f1f140d97b0 0x7f1f141ad440 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0x7f1f04002f70 tx=0x7f1f040068c0).stop 2024-04-17T22:30:55.407 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.420 7f1f188b7700 1 -- 172.21.15.102:0/4170237831 shutdown_connections 2024-04-17T22:30:55.407 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.420 7f1f188b7700 1 --2- 172.21.15.102:0/4170237831 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1f140d9c00 0x7f1f141ada70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:30:55.407 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.420 7f1f188b7700 1 --2- 172.21.15.102:0/4170237831 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f1f141a60e0 0x7f1f140d9180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:30:55.407 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.420 7f1f188b7700 1 --2- 172.21.15.102:0/4170237831 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f1f140d97b0 0x7f1f141ad440 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:30:55.407 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.420 7f1f188b7700 1 -- 172.21.15.102:0/4170237831 >> 172.21.15.102:0/4170237831 conn(0x7f1f140c5650 msgr2=0x7f1f140d62b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:30:55.407 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.420 7f1f188b7700 1 -- 172.21.15.102:0/4170237831 shutdown_connections 2024-04-17T22:30:55.407 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:55.421 7f1f188b7700 1 -- 172.21.15.102:0/4170237831 wait complete. 2024-04-17T22:30:55.417 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705690 got 38654705690 for osd.1 2024-04-17T22:30:55.417 DEBUG:teuthology.parallel:result is None 2024-04-17T22:30:56.350 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-17T22:30:56.351 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-17T22:30:56.490 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.504 7f54182cd700 1 Processor -- start 2024-04-17T22:30:56.491 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.504 7f65aba85700 1 Processor -- start 2024-04-17T22:30:56.491 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.505 7f65aba85700 1 -- start start 2024-04-17T22:30:56.491 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.505 7f54182cd700 1 -- start start 2024-04-17T22:30:56.492 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.505 7f65aba85700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f65a40faed0 0x7f65a40ff5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:30:56.492 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.505 7f54182cd700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f541010e1d0 0x7f541010e620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:30:56.492 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.505 7f65aba85700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f65a41a1780 0x7f65a41a9020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:30:56.493 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.505 7f54182cd700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f54100f1ec0 0x7f54101ab470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:30:56.493 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.505 7f54182cd700 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54100f2310 con 0x7f541010ec50 2024-04-17T22:30:56.493 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.505 7f54182cd700 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54100f9ea0 con 0x7f541010e1d0 2024-04-17T22:30:56.493 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.505 7f65aba85700 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65a4111d50 con 0x7f65a41a0b30 2024-04-17T22:30:56.493 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.505 7f65aba85700 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65a40ffbe0 con 0x7f65a40faed0 2024-04-17T22:30:56.493 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.505 7f65aba85700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65a40f2180 con 0x7f65a41a1780 2024-04-17T22:30:56.494 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.505 7f54182cd700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f54100f8fc0 con 0x7f54100f1ec0 2024-04-17T22:30:56.494 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.505 7f65a9081700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f65a41a1780 0x7f65a41a9020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:30:56.494 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.505 7f54158c9700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f54100f1ec0 0x7f54101ab470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:30:56.494 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.505 7f54158c9700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f54100f1ec0 0x7f54101ab470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:59140/0 (socket says 172.21.15.102:59140) 2024-04-17T22:30:56.494 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.505 7f54148c7700 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f541010ec50 0x7f54100f16c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:40930/0 (socket says 172.21.15.102:40930) 2024-04-17T22:30:56.494 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.505 7f54158c9700 1 -- 172.21.15.102:0/1829203353 learned_addr learned my addr 172.21.15.102:0/1829203353 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:30:56.494 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.505 7f54150c8700 1 --2- 172.21.15.102:0/1829203353 >> v2:172.21.15.102:3300/0 conn(0x7f541010e1d0 0x7f541010e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:30:56.495 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.505 7f65a9081700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f65a41a1780 0x7f65a41a9020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:48738/0 (socket says 172.21.15.102:48738) 2024-04-17T22:30:56.495 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.505 7f65a9081700 1 -- 172.21.15.102:0/3609808005 learned_addr learned my addr 172.21.15.102:0/3609808005 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:30:56.495 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.505 7f65a8880700 1 --2- 172.21.15.102:0/3609808005 >> v2:172.21.15.102:3300/0 conn(0x7f65a40faed0 0x7f65a40ff5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:30:56.495 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.505 7f54158c9700 1 -- 172.21.15.102:0/1829203353 >> v2:172.21.15.102:3300/0 conn(0x7f541010e1d0 msgr2=0x7f541010e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:30:56.495 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.505 7f54158c9700 1 --2- 172.21.15.102:0/1829203353 >> v2:172.21.15.102:3300/0 conn(0x7f541010e1d0 0x7f541010e620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:30:56.495 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.506 7f54158c9700 1 -- 172.21.15.102:0/1829203353 >> v1:172.21.15.102:6789/0 conn(0x7f541010ec50 legacy=0x7f54100f16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:30:56.495 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.506 7f54158c9700 1 -- 172.21.15.102:0/1829203353 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54100f2530 con 0x7f54100f1ec0 2024-04-17T22:30:56.496 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.506 7f65a8880700 1 -- 172.21.15.102:0/3609808005 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f65a41a1780 msgr2=0x7f65a41a9020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:30:56.496 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.506 7f65a8880700 1 --2- 172.21.15.102:0/3609808005 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f65a41a1780 0x7f65a41a9020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:30:56.496 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.506 7f65a8880700 1 -- 172.21.15.102:0/3609808005 >> v1:172.21.15.102:6789/0 conn(0x7f65a41a0b30 legacy=0x7f65a41a0f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:30:56.496 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.506 7f65a8880700 1 -- 172.21.15.102:0/3609808005 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65a4106590 con 0x7f65a40faed0 2024-04-17T22:30:56.496 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.506 7f54158c9700 1 --2- 172.21.15.102:0/1829203353 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f54100f1ec0 0x7f54101ab470 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7f5400001380 tx=0x7f5400001080).ready entity=mon.2 client_cookie=a9bcd89c33712c64 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:30:56.497 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.506 7f65a9081700 1 --2- 172.21.15.102:0/3609808005 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f65a41a1780 0x7f65a41a9020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:30:56.497 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.506 7f65a8880700 1 --2- 172.21.15.102:0/3609808005 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f65a40faed0 0x7f65a40ff5b0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 rx=0x7f6598002f70 tx=0x7f659800b9f0).ready entity=mon.0 client_cookie=63b632cb8eaa7fd9 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:30:56.498 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.506 7f53fffff700 1 -- 172.21.15.102:0/1829203353 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f54100f2530 con 0x7f54100f1ec0 2024-04-17T22:30:56.498 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.506 7f53fffff700 1 -- 172.21.15.102:0/1829203353 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f540800b740 con 0x7f54100f1ec0 2024-04-17T22:30:56.498 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.506 7f53fffff700 1 -- 172.21.15.102:0/1829203353 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f540800b920 con 0x7f54100f1ec0 2024-04-17T22:30:56.498 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.506 7f54182cd700 1 -- 172.21.15.102:0/1829203353 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f54100f1ec0 msgr2=0x7f54101ab470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:30:56.498 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.506 7f54182cd700 1 --2- 172.21.15.102:0/1829203353 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f54100f1ec0 0x7f54101ab470 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7f5400001380 tx=0x7f5400001080).stop 2024-04-17T22:30:56.499 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.506 7f65a37fe700 1 -- 172.21.15.102:0/3609808005 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f65a4106590 con 0x7f65a40faed0 2024-04-17T22:30:56.499 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.506 7f65a37fe700 1 -- 172.21.15.102:0/3609808005 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f659800d5c0 con 0x7f65a40faed0 2024-04-17T22:30:56.499 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.506 7f65a37fe700 1 -- 172.21.15.102:0/3609808005 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f659800d990 con 0x7f65a40faed0 2024-04-17T22:30:56.499 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.506 7f65aba85700 1 -- 172.21.15.102:0/3609808005 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f65a40faed0 msgr2=0x7f65a40ff5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:30:56.499 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.506 7f65aba85700 1 --2- 172.21.15.102:0/3609808005 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f65a40faed0 0x7f65a40ff5b0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 rx=0x7f6598002f70 tx=0x7f659800b9f0).stop 2024-04-17T22:30:56.499 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.507 7f54182cd700 1 -- 172.21.15.102:0/1829203353 shutdown_connections 2024-04-17T22:30:56.500 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.507 7f54182cd700 1 --2- 172.21.15.102:0/1829203353 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f54100f1ec0 0x7f54101ab470 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:30:56.500 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.507 7f54182cd700 1 --2- 172.21.15.102:0/1829203353 >> v2:172.21.15.102:3300/0 conn(0x7f541010e1d0 0x7f541010e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:30:56.500 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.507 7f54182cd700 1 -- 172.21.15.102:0/1829203353 >> 172.21.15.102:0/1829203353 conn(0x7f5410113830 msgr2=0x7f54100f4ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:30:56.500 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.507 7f54182cd700 1 -- 172.21.15.102:0/1829203353 shutdown_connections 2024-04-17T22:30:56.500 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.507 7f54182cd700 1 -- 172.21.15.102:0/1829203353 wait complete. 2024-04-17T22:30:56.501 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.507 7f65aba85700 1 -- 172.21.15.102:0/3609808005 shutdown_connections 2024-04-17T22:30:56.501 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.507 7f65aba85700 1 --2- 172.21.15.102:0/3609808005 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f65a41a1780 0x7f65a41a9020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:30:56.501 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.507 7f65aba85700 1 --2- 172.21.15.102:0/3609808005 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f65a40faed0 0x7f65a40ff5b0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:30:56.501 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.507 7f65aba85700 1 -- 172.21.15.102:0/3609808005 >> 172.21.15.102:0/3609808005 conn(0x7f65a41146f0 msgr2=0x7f65a4114b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:30:56.501 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.507 7f65aba85700 1 -- 172.21.15.102:0/3609808005 shutdown_connections 2024-04-17T22:30:56.501 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.507 7f65aba85700 1 -- 172.21.15.102:0/3609808005 wait complete. 2024-04-17T22:30:56.516 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.530 7f54182cd700 1 Processor -- start 2024-04-17T22:30:56.516 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.530 7f65aba85700 1 Processor -- start 2024-04-17T22:30:56.517 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.530 7f54182cd700 1 -- start start 2024-04-17T22:30:56.517 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.530 7f65aba85700 1 -- start start 2024-04-17T22:30:56.518 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.530 7f54182cd700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f54100cde40 0x7f54100d5ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:30:56.518 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.530 7f54182cd700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f54100d6620 0x7f54101b3e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:30:56.518 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.530 7f54182cd700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f54100d6a70 0x7f54101b4490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:30:56.518 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.530 7f54182cd700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54100e2d30 con 0x7f54100d6a70 2024-04-17T22:30:56.518 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.530 7f54182cd700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54100fa140 con 0x7f54100d6620 2024-04-17T22:30:56.518 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.530 7f54182cd700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f54100cd5a0 con 0x7f54100cde40 2024-04-17T22:30:56.518 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.531 7f54148c7700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f54100d6620 0x7f54101b3e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:30:56.519 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.531 7f54158c9700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f54100d6a70 0x7f54101b4490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:30:56.519 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.531 7f54148c7700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f54100d6620 0x7f54101b3e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:48744/0 (socket says 172.21.15.102:48744) 2024-04-17T22:30:56.519 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.531 7f54148c7700 1 -- 172.21.15.102:0/1319763659 learned_addr learned my addr 172.21.15.102:0/1319763659 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:30:56.519 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.531 7f54150c8700 1 --2- 172.21.15.102:0/1319763659 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f54100cde40 0x7f54100d5ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:30:56.520 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.531 7f65aba85700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f65a40ce0a0 0x7f65a40d6010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:30:56.520 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.531 7f65aba85700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f65a40d6640 0x7f65a41b1c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:30:56.520 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.531 7f65aba85700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f65a40d6a90 0x7f65a41b22b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:30:56.520 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.531 7f65aba85700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65a41a6b90 con 0x7f65a40ce0a0 2024-04-17T22:30:56.520 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.531 7f65aba85700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65a40cd360 con 0x7f65a40d6640 2024-04-17T22:30:56.520 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.531 7f65aba85700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f65a41b28e0 con 0x7f65a40d6a90 2024-04-17T22:30:56.521 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.531 7f65a9081700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f65a40d6a90 0x7f65a41b22b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:30:56.521 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.531 7f65a9081700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f65a40d6a90 0x7f65a41b22b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:59166/0 (socket says 172.21.15.102:59166) 2024-04-17T22:30:56.521 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.531 7f65a9081700 1 -- 172.21.15.102:0/314827940 learned_addr learned my addr 172.21.15.102:0/314827940 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:30:56.521 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.531 7f65a8880700 1 --2- 172.21.15.102:0/314827940 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f65a40ce0a0 0x7f65a40d6010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:30:56.521 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.531 7f65a3fff700 1 --2- 172.21.15.102:0/314827940 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f65a40d6640 0x7f65a41b1c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:30:56.521 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.531 7f54148c7700 1 -- 172.21.15.102:0/1319763659 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f54100cde40 msgr2=0x7f54100d5ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:30:56.521 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.531 7f54148c7700 1 --2- 172.21.15.102:0/1319763659 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f54100cde40 0x7f54100d5ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:30:56.521 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.531 7f54148c7700 1 -- 172.21.15.102:0/1319763659 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f54100d6a70 msgr2=0x7f54101b4490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:30:56.521 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.531 7f54148c7700 1 --2- 172.21.15.102:0/1319763659 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f54100d6a70 0x7f54101b4490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:30:56.522 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.531 7f54148c7700 1 -- 172.21.15.102:0/1319763659 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54100c4500 con 0x7f54100d6620 2024-04-17T22:30:56.522 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.531 7f54150c8700 1 --2- 172.21.15.102:0/1319763659 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f54100cde40 0x7f54100d5ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:30:56.522 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.531 7f54148c7700 1 --2- 172.21.15.102:0/1319763659 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f54100d6620 0x7f54101b3e60 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 rx=0x7f54101b08f0 tx=0x7f540000cbc0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:30:56.522 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.531 7f65a3fff700 1 -- 172.21.15.102:0/314827940 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f65a40d6a90 msgr2=0x7f65a41b22b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:30:56.522 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.531 7f65a3fff700 1 --2- 172.21.15.102:0/314827940 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f65a40d6a90 0x7f65a41b22b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:30:56.522 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.531 7f65a3fff700 1 -- 172.21.15.102:0/314827940 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f65a40ce0a0 msgr2=0x7f65a40d6010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:30:56.522 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.531 7f65a3fff700 1 --2- 172.21.15.102:0/314827940 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f65a40ce0a0 0x7f65a40d6010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:30:56.522 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.531 7f65a3fff700 1 -- 172.21.15.102:0/314827940 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65a41b2b10 con 0x7f65a40d6640 2024-04-17T22:30:56.523 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.531 7f65a8880700 1 --2- 172.21.15.102:0/314827940 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f65a40ce0a0 0x7f65a40d6010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:30:56.523 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.531 7f53fe7fc700 1 -- 172.21.15.102:0/1319763659 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f5400007320 con 0x7f54100d6620 2024-04-17T22:30:56.523 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.532 7f53fe7fc700 1 -- 172.21.15.102:0/1319763659 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5400005560 con 0x7f54100d6620 2024-04-17T22:30:56.523 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.532 7f53fe7fc700 1 -- 172.21.15.102:0/1319763659 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f5400005930 con 0x7f54100d6620 2024-04-17T22:30:56.524 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.532 7f65a3fff700 1 --2- 172.21.15.102:0/314827940 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f65a40d6640 0x7f65a41b1c80 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 rx=0x7f6590000bc0 tx=0x7f659000c7b0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:30:56.524 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.532 7f54182cd700 1 -- 172.21.15.102:0/1319763659 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54101b4ac0 con 0x7f54100d6620 2024-04-17T22:30:56.525 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.532 7f65a1ffb700 1 -- 172.21.15.102:0/314827940 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f65a41b2b10 con 0x7f65a40d6640 2024-04-17T22:30:56.525 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.532 7f65a1ffb700 1 -- 172.21.15.102:0/314827940 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6590007030 con 0x7f65a40d6640 2024-04-17T22:30:56.525 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.532 7f65aba85700 1 -- 172.21.15.102:0/314827940 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65a41b93f0 con 0x7f65a40d6640 2024-04-17T22:30:56.525 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.532 7f54182cd700 1 -- 172.21.15.102:0/1319763659 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54101bb5d0 con 0x7f54100d6620 2024-04-17T22:30:56.526 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.532 7f65aba85700 1 -- 172.21.15.102:0/314827940 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65a41b97d0 con 0x7f65a40d6640 2024-04-17T22:30:56.526 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.532 7f65a1ffb700 1 -- 172.21.15.102:0/314827940 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f6590006520 con 0x7f65a40d6640 2024-04-17T22:30:56.527 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.532 7f53fe7fc700 1 -- 172.21.15.102:0/1319763659 <== mon.1 v2:172.21.15.102:3301/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7f540001a020 con 0x7f54100d6620 2024-04-17T22:30:56.527 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.532 7f53fe7fc700 1 -- 172.21.15.102:0/1319763659 <== mon.1 v2:172.21.15.102:3301/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3194+0+0 (secure 0 0 0) 0x7f5400009d90 con 0x7f54100d6620 2024-04-17T22:30:56.528 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.533 7f65a1ffb700 1 -- 172.21.15.102:0/314827940 <== mon.1 v2:172.21.15.102:3301/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7f659001a020 con 0x7f65a40d6640 2024-04-17T22:30:56.528 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.533 7f65a1ffb700 1 -- 172.21.15.102:0/314827940 <== mon.1 v2:172.21.15.102:3301/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3194+0+0 (secure 0 0 0) 0x7f6590003c20 con 0x7f65a40d6640 2024-04-17T22:30:56.528 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.533 7f65aba85700 1 -- 172.21.15.102:0/314827940 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6580003420 con 0x7f65a40d6640 2024-04-17T22:30:56.528 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.533 7f54182cd700 1 -- 172.21.15.102:0/1319763659 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54100f0790 con 0x7f54100d6620 2024-04-17T22:30:56.529 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.536 7f65a1ffb700 1 -- 172.21.15.102:0/314827940 <== mon.1 v2:172.21.15.102:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+110455 (secure 0 0 0) 0x7f6590000c30 con 0x7f65a40d6640 2024-04-17T22:30:56.529 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.541 7f53fe7fc700 1 -- 172.21.15.102:0/1319763659 <== mon.1 v2:172.21.15.102:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+110455 (secure 0 0 0) 0x7f5400007110 con 0x7f54100d6620 2024-04-17T22:30:56.633 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.647 7f65aba85700 1 -- 172.21.15.102:0/314827940 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f6580003a70 con 0x7f65a40d6640 2024-04-17T22:30:56.633 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.647 7f65a1ffb700 1 -- 172.21.15.102:0/314827940 <== mon.1 v2:172.21.15.102:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f659000cda0 con 0x7f65a40d6640 2024-04-17T22:30:56.633 INFO:teuthology.orchestra.run.smithi102.stdout:38654705691 2024-04-17T22:30:56.635 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.648 7f65aba85700 1 -- 172.21.15.102:0/314827940 >> v1:172.21.15.102:6816/42360 conn(0x7f658c0145a0 legacy=0x7f658c016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:30:56.635 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.648 7f65aba85700 1 -- 172.21.15.102:0/314827940 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f65a40d6640 msgr2=0x7f65a41b1c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:30:56.635 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.648 7f65aba85700 1 --2- 172.21.15.102:0/314827940 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f65a40d6640 0x7f65a41b1c80 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 rx=0x7f6590000bc0 tx=0x7f659000c7b0).stop 2024-04-17T22:30:56.635 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.648 7f65aba85700 1 -- 172.21.15.102:0/314827940 shutdown_connections 2024-04-17T22:30:56.635 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.648 7f65aba85700 1 --2- 172.21.15.102:0/314827940 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f65a40d6a90 0x7f65a41b22b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:30:56.635 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.648 7f65aba85700 1 --2- 172.21.15.102:0/314827940 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f65a40ce0a0 0x7f65a40d6010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:30:56.635 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.648 7f65aba85700 1 --2- 172.21.15.102:0/314827940 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f65a40d6640 0x7f65a41b1c80 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:30:56.635 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.648 7f65aba85700 1 -- 172.21.15.102:0/314827940 >> 172.21.15.102:0/314827940 conn(0x7f65a40c38f0 msgr2=0x7f65a40dae90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:30:56.635 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.648 7f65aba85700 1 -- 172.21.15.102:0/314827940 shutdown_connections 2024-04-17T22:30:56.635 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.648 7f65aba85700 1 -- 172.21.15.102:0/314827940 wait complete. 2024-04-17T22:30:56.636 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.650 7f54182cd700 1 -- 172.21.15.102:0/1319763659 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f54101ae560 con 0x7f54100d6620 2024-04-17T22:30:56.637 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.651 7f53fe7fc700 1 -- 172.21.15.102:0/1319763659 <== mon.1 v2:172.21.15.102:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f54100f13d0 con 0x7f54100d6620 2024-04-17T22:30:56.637 INFO:teuthology.orchestra.run.smithi102.stdout:38654705691 2024-04-17T22:30:56.638 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.652 7f54182cd700 1 -- 172.21.15.102:0/1319763659 >> v1:172.21.15.102:6816/42360 conn(0x7f53f00145a0 legacy=0x7f53f0016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:30:56.638 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.652 7f54182cd700 1 -- 172.21.15.102:0/1319763659 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f54100d6620 msgr2=0x7f54101b3e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:30:56.638 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.652 7f54182cd700 1 --2- 172.21.15.102:0/1319763659 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f54100d6620 0x7f54101b3e60 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 rx=0x7f54101b08f0 tx=0x7f540000cbc0).stop 2024-04-17T22:30:56.642 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.656 7f54182cd700 1 -- 172.21.15.102:0/1319763659 shutdown_connections 2024-04-17T22:30:56.642 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.656 7f54182cd700 1 --2- 172.21.15.102:0/1319763659 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f54100d6a70 0x7f54101b4490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:30:56.642 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.656 7f54182cd700 1 --2- 172.21.15.102:0/1319763659 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f54100cde40 0x7f54100d5ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:30:56.642 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.656 7f54182cd700 1 --2- 172.21.15.102:0/1319763659 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f54100d6620 0x7f54101b3e60 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:30:56.642 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.656 7f54182cd700 1 -- 172.21.15.102:0/1319763659 >> 172.21.15.102:0/1319763659 conn(0x7f54100c38d0 msgr2=0x7f54100dae30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:30:56.642 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.656 7f54182cd700 1 -- 172.21.15.102:0/1319763659 shutdown_connections 2024-04-17T22:30:56.642 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.656 7f54182cd700 1 -- 172.21.15.102:0/1319763659 wait complete. 2024-04-17T22:30:56.645 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705691 got 38654705691 for osd.2 2024-04-17T22:30:56.646 DEBUG:teuthology.parallel:result is None 2024-04-17T22:30:56.653 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705691 got 38654705691 for osd.0 2024-04-17T22:30:56.653 DEBUG:teuthology.parallel:result is None 2024-04-17T22:30:56.653 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-17T22:30:56.653 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-17T22:30:56.819 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.831 7f889791a700 1 Processor -- start 2024-04-17T22:30:56.819 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.831 7f889791a700 1 -- start start 2024-04-17T22:30:56.819 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.831 7f889791a700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f88901a64d0 0x7f88901a8930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:30:56.819 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.831 7f889791a700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f88901a8f60 0x7f88901ab3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:30:56.820 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.831 7f889791a700 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f889010eeb0 con 0x7f889010e260 2024-04-17T22:30:56.820 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.831 7f889791a700 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88900f9f30 con 0x7f88901a8f60 2024-04-17T22:30:56.820 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.831 7f889791a700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88900f9050 con 0x7f88901a64d0 2024-04-17T22:30:56.820 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.831 7f8894f16700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f88901a8f60 0x7f88901ab3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:30:56.820 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.831 7f8887fff700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f88901a64d0 0x7f88901a8930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:30:56.820 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.831 7f8894f16700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f88901a8f60 0x7f88901ab3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:40348/0 (socket says 172.21.15.102:40348) 2024-04-17T22:30:56.820 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.832 7f8894f16700 1 -- 172.21.15.102:0/1630016511 learned_addr learned my addr 172.21.15.102:0/1630016511 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:30:56.820 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.832 7f8887fff700 1 -- 172.21.15.102:0/1630016511 >> v2:172.21.15.102:3300/0 conn(0x7f88901a8f60 msgr2=0x7f88901ab3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:30:56.820 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.832 7f8887fff700 1 --2- 172.21.15.102:0/1630016511 >> v2:172.21.15.102:3300/0 conn(0x7f88901a8f60 0x7f88901ab3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:30:56.820 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.832 7f8887fff700 1 -- 172.21.15.102:0/1630016511 >> v1:172.21.15.102:6789/0 conn(0x7f889010e260 legacy=0x7f889010e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:30:56.820 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.832 7f8887fff700 1 -- 172.21.15.102:0/1630016511 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88901aca00 con 0x7f88901a64d0 2024-04-17T22:30:56.820 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.832 7f888f7fe700 1 -- 172.21.15.102:0/1630016511 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 180514568 0 0) 0x7f8880001110 con 0x7f889010e260 2024-04-17T22:30:56.820 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.832 7f8894f16700 1 --2- 172.21.15.102:0/1630016511 >> v2:172.21.15.102:3300/0 conn(0x7f88901a8f60 0x7f88901ab3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:30:56.820 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.832 7f8887fff700 1 --2- 172.21.15.102:0/1630016511 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f88901a64d0 0x7f88901a8930 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0x7f8878002f70 tx=0x7f887800b970).ready entity=mon.1 client_cookie=2ec6ff5fda34a508 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:30:56.821 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.832 7f888f7fe700 1 -- 172.21.15.102:0/1630016511 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f88901aca00 con 0x7f88901a64d0 2024-04-17T22:30:56.821 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.832 7f888f7fe700 1 -- 172.21.15.102:0/1630016511 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f887800d610 con 0x7f88901a64d0 2024-04-17T22:30:56.821 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.832 7f888f7fe700 1 -- 172.21.15.102:0/1630016511 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f887800d9e0 con 0x7f88901a64d0 2024-04-17T22:30:56.821 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.832 7f889791a700 1 -- 172.21.15.102:0/1630016511 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f88901a64d0 msgr2=0x7f88901a8930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:30:56.821 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.832 7f889791a700 1 --2- 172.21.15.102:0/1630016511 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f88901a64d0 0x7f88901a8930 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0x7f8878002f70 tx=0x7f887800b970).stop 2024-04-17T22:30:56.821 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.833 7f889791a700 1 -- 172.21.15.102:0/1630016511 shutdown_connections 2024-04-17T22:30:56.821 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.833 7f889791a700 1 --2- 172.21.15.102:0/1630016511 >> v2:172.21.15.102:3300/0 conn(0x7f88901a8f60 0x7f88901ab3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:30:56.821 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.833 7f889791a700 1 --2- 172.21.15.102:0/1630016511 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f88901a64d0 0x7f88901a8930 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:30:56.821 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.833 7f889791a700 1 -- 172.21.15.102:0/1630016511 >> 172.21.15.102:0/1630016511 conn(0x7f88901138c0 msgr2=0x7f88900f4f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:30:56.821 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.833 7f889791a700 1 -- 172.21.15.102:0/1630016511 shutdown_connections 2024-04-17T22:30:56.821 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.833 7f889791a700 1 -- 172.21.15.102:0/1630016511 wait complete. 2024-04-17T22:30:56.844 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.855 7f889791a700 1 Processor -- start 2024-04-17T22:30:56.844 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.855 7f889791a700 1 -- start start 2024-04-17T22:30:56.844 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.855 7f889791a700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f88900d5f40 0x7f88900d6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:30:56.844 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.855 7f889791a700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f88900d69c0 0x7f88901b3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:30:56.844 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.855 7f889791a700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f88901b4520 0x7f88901b4970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:30:56.844 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.855 7f889791a700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88900fa1d0 con 0x7f88901b4520 2024-04-17T22:30:56.844 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.855 7f889791a700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88900c43d0 con 0x7f88900d5f40 2024-04-17T22:30:56.844 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.855 7f889791a700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f88900cd520 con 0x7f88900d69c0 2024-04-17T22:30:56.844 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.855 7f888ffff700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f88900d5f40 0x7f88900d6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:30:56.844 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.855 7f8894f16700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f88901b4520 0x7f88901b4970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:30:56.845 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.855 7f8887fff700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f88900d69c0 0x7f88901b3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:30:56.845 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.855 7f8887fff700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f88900d69c0 0x7f88901b3ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:59168/0 (socket says 172.21.15.102:59168) 2024-04-17T22:30:56.845 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.855 7f888ffff700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f88900d5f40 0x7f88900d6390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:48770/0 (socket says 172.21.15.102:48770) 2024-04-17T22:30:56.845 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.855 7f8894f16700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f88901b4520 0x7f88901b4970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:40352/0 (socket says 172.21.15.102:40352) 2024-04-17T22:30:56.845 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.856 7f8887fff700 1 -- 172.21.15.102:0/3052273714 learned_addr learned my addr 172.21.15.102:0/3052273714 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:30:56.845 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.856 7f8887fff700 1 -- 172.21.15.102:0/3052273714 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f88900d5f40 msgr2=0x7f88900d6390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:30:56.845 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.856 7f8887fff700 1 --2- 172.21.15.102:0/3052273714 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f88900d5f40 0x7f88900d6390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:30:56.845 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.856 7f8887fff700 1 -- 172.21.15.102:0/3052273714 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f88901b4520 msgr2=0x7f88901b4970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:30:56.845 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.856 7f8887fff700 1 --2- 172.21.15.102:0/3052273714 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f88901b4520 0x7f88901b4970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:30:56.845 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.856 7f8887fff700 1 -- 172.21.15.102:0/3052273714 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88900cf720 con 0x7f88900d69c0 2024-04-17T22:30:56.846 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.856 7f8887fff700 1 --2- 172.21.15.102:0/3052273714 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f88900d69c0 0x7f88901b3ef0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7f88900c4a40 tx=0x7f887800a040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:30:56.846 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.856 7f888dffb700 1 -- 172.21.15.102:0/3052273714 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f88900cf720 con 0x7f88900d69c0 2024-04-17T22:30:56.846 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.856 7f888dffb700 1 -- 172.21.15.102:0/3052273714 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8878013a70 con 0x7f88900d69c0 2024-04-17T22:30:56.846 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.856 7f888dffb700 1 -- 172.21.15.102:0/3052273714 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f887801f450 con 0x7f88900d69c0 2024-04-17T22:30:56.846 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.856 7f889791a700 1 -- 172.21.15.102:0/3052273714 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88901bb660 con 0x7f88900d69c0 2024-04-17T22:30:56.846 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.856 7f889791a700 1 -- 172.21.15.102:0/3052273714 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88901bba40 con 0x7f88900d69c0 2024-04-17T22:30:56.846 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.857 7f888dffb700 1 -- 172.21.15.102:0/3052273714 <== mon.2 v2:172.21.15.102:3302/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7f8878027070 con 0x7f88900d69c0 2024-04-17T22:30:56.846 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.857 7f888dffb700 1 -- 172.21.15.102:0/3052273714 <== mon.2 v2:172.21.15.102:3302/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3194+0+0 (secure 0 0 0) 0x7f8878008820 con 0x7f88900d69c0 2024-04-17T22:30:56.846 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.857 7f889791a700 1 -- 172.21.15.102:0/3052273714 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88900f0820 con 0x7f88900d69c0 2024-04-17T22:30:56.847 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.860 7f888dffb700 1 -- 172.21.15.102:0/3052273714 <== mon.2 v2:172.21.15.102:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+110455 (secure 0 0 0) 0x7f8878013860 con 0x7f88900d69c0 2024-04-17T22:30:56.947 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.961 7f889791a700 1 -- 172.21.15.102:0/3052273714 --> v1:172.21.15.102:6816/42360 -- command(tid 0: {"prefix": "pg dump", "target": ["mgr", ""], "format": "json"}) v1 -- 0x7f889008a300 con 0x7f88740145a0 2024-04-17T22:30:56.955 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.969 7f888dffb700 1 -- 172.21.15.102:0/3052273714 <== mgr.4104 v1:172.21.15.102:6816/42360 1 ==== command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311258 (unknown 2965378022 0 3038669756) 0x7f888000c200 con 0x7f88740145a0 2024-04-17T22:30:56.959 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-17T22:30:56.962 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.972 7f889791a700 1 -- 172.21.15.102:0/3052273714 >> v1:172.21.15.102:6816/42360 conn(0x7f88740145a0 legacy=0x7f8874016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:30:56.962 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.972 7f889791a700 1 -- 172.21.15.102:0/3052273714 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f88900d69c0 msgr2=0x7f88901b3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:30:56.962 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.972 7f889791a700 1 --2- 172.21.15.102:0/3052273714 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f88900d69c0 0x7f88901b3ef0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7f88900c4a40 tx=0x7f887800a040).stop 2024-04-17T22:30:56.962 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.972 7f889791a700 1 -- 172.21.15.102:0/3052273714 shutdown_connections 2024-04-17T22:30:56.962 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.972 7f889791a700 1 --2- 172.21.15.102:0/3052273714 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f88901b4520 0x7f88901b4970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:30:56.962 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.972 7f889791a700 1 --2- 172.21.15.102:0/3052273714 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f88900d5f40 0x7f88900d6390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:30:56.962 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.972 7f889791a700 1 --2- 172.21.15.102:0/3052273714 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f88900d69c0 0x7f88901b3ef0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:30:56.962 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.972 7f889791a700 1 -- 172.21.15.102:0/3052273714 >> 172.21.15.102:0/3052273714 conn(0x7f88900c37a0 msgr2=0x7f88900dadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:30:56.962 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.972 7f889791a700 1 -- 172.21.15.102:0/3052273714 shutdown_connections 2024-04-17T22:30:56.962 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:56.972 7f889791a700 1 -- 172.21.15.102:0/3052273714 wait complete. 2024-04-17T22:30:56.962 INFO:teuthology.orchestra.run.smithi102.stderr:dumped all 2024-04-17T22:30:56.969 INFO:teuthology.orchestra.run.smithi102.stdout:{"pg_ready":true,"pg_map":{"version":74,"stamp":"2024-04-17 22:30:55.212806","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":3597,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":43,"ondisk_log_size":43,"up":272,"acting":272,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":272,"num_osds":4,"num_per_pool_osds":4,"kb":377487360,"kb_used":4194976,"kb_used_data":640,"kb_used_omap":0,"kb_used_meta":4194304,"kb_avail":373292384,"statfs":{"total":386547056640,"available":382251401216,"internally_reserved":4294967296,"allocated":655360,"data_stored":177858,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4294967296},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002821"},"pg_stats":[{"pgid":"3.1a","version":"0'0","reported_seq":"10","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:08.799742","last_change":"2024-04-17 22:29:07.814049","last_active":"2024-04-17 22:29:08.799742","last_peered":"2024-04-17 22:29:08.799742","last_clean":"2024-04-17 22:29:08.799742","last_became_active":"2024-04-17 22:29:07.811049","last_became_peered":"2024-04-17 22:29:07.811049","last_unstale":"2024-04-17 22:29:08.799742","last_undegraded":"2024-04-17 22:29:08.799742","last_fullsized":"2024-04-17 22:29:08.799742","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":"11","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:09.398176","last_change":"2024-04-17 22:29:07.477053","last_active":"2024-04-17 22:29:09.398176","last_peered":"2024-04-17 22:29:09.398176","last_clean":"2024-04-17 22:29:09.398176","last_became_active":"2024-04-17 22:29:07.476568","last_became_peered":"2024-04-17 22:29:07.476568","last_unstale":"2024-04-17 22:29:09.398176","last_undegraded":"2024-04-17 22:29:09.398176","last_fullsized":"2024-04-17 22:29:09.398176","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":"10","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:08.802676","last_change":"2024-04-17 22:29:07.818787","last_active":"2024-04-17 22:29:08.802676","last_peered":"2024-04-17 22:29:08.802676","last_clean":"2024-04-17 22:29:08.802676","last_became_active":"2024-04-17 22:29:07.818347","last_became_peered":"2024-04-17 22:29:07.818347","last_unstale":"2024-04-17 22:29:08.802676","last_undegraded":"2024-04-17 22:29:08.802676","last_fullsized":"2024-04-17 22:29:08.802676","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":"11","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:09.398049","last_change":"2024-04-17 22:29:07.477933","last_active":"2024-04-17 22:29:09.398049","last_peered":"2024-04-17 22:29:09.398049","last_clean":"2024-04-17 22:29:09.398049","last_became_active":"2024-04-17 22:29:07.472128","last_became_peered":"2024-04-17 22:29:07.472128","last_unstale":"2024-04-17 22:29:09.398049","last_undegraded":"2024-04-17 22:29:09.398049","last_fullsized":"2024-04-17 22:29:09.398049","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":"10","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:09.397770","last_change":"2024-04-17 22:29:07.833107","last_active":"2024-04-17 22:29:09.397770","last_peered":"2024-04-17 22:29:09.397770","last_clean":"2024-04-17 22:29:09.397770","last_became_active":"2024-04-17 22:29:07.832822","last_became_peered":"2024-04-17 22:29:07.832822","last_unstale":"2024-04-17 22:29:09.397770","last_undegraded":"2024-04-17 22:29:09.397770","last_fullsized":"2024-04-17 22:29:09.397770","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":"11","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:09.397993","last_change":"2024-04-17 22:29:07.471612","last_active":"2024-04-17 22:29:09.397993","last_peered":"2024-04-17 22:29:09.397993","last_clean":"2024-04-17 22:29:09.397993","last_became_active":"2024-04-17 22:29:07.471322","last_became_peered":"2024-04-17 22:29:07.471322","last_unstale":"2024-04-17 22:29:09.397993","last_undegraded":"2024-04-17 22:29:09.397993","last_fullsized":"2024-04-17 22:29:09.397993","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":"10","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:08.799757","last_change":"2024-04-17 22:29:07.810081","last_active":"2024-04-17 22:29:08.799757","last_peered":"2024-04-17 22:29:08.799757","last_clean":"2024-04-17 22:29:08.799757","last_became_active":"2024-04-17 22:29:07.809858","last_became_peered":"2024-04-17 22:29:07.809858","last_unstale":"2024-04-17 22:29:08.799757","last_undegraded":"2024-04-17 22:29:08.799757","last_fullsized":"2024-04-17 22:29:08.799757","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":"11","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:09.397518","last_change":"2024-04-17 22:29:07.474346","last_active":"2024-04-17 22:29:09.397518","last_peered":"2024-04-17 22:29:09.397518","last_clean":"2024-04-17 22:29:09.397518","last_became_active":"2024-04-17 22:29:07.473783","last_became_peered":"2024-04-17 22:29:07.473783","last_unstale":"2024-04-17 22:29:09.397518","last_undegraded":"2024-04-17 22:29:09.397518","last_fullsized":"2024-04-17 22:29:09.397518","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":"10","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:08.801136","last_change":"2024-04-17 22:29:07.811047","last_active":"2024-04-17 22:29:08.801136","last_peered":"2024-04-17 22:29:08.801136","last_clean":"2024-04-17 22:29:08.801136","last_became_active":"2024-04-17 22:29:07.810747","last_became_peered":"2024-04-17 22:29:07.810747","last_unstale":"2024-04-17 22:29:08.801136","last_undegraded":"2024-04-17 22:29:08.801136","last_fullsized":"2024-04-17 22:29:08.801136","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":"11","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:08.801135","last_change":"2024-04-17 22:29:07.460228","last_active":"2024-04-17 22:29:08.801135","last_peered":"2024-04-17 22:29:08.801135","last_clean":"2024-04-17 22:29:08.801135","last_became_active":"2024-04-17 22:29:07.459910","last_became_peered":"2024-04-17 22:29:07.459910","last_unstale":"2024-04-17 22:29:08.801135","last_undegraded":"2024-04-17 22:29:08.801135","last_fullsized":"2024-04-17 22:29:08.801135","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":"10","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:08.805762","last_change":"2024-04-17 22:29:07.827143","last_active":"2024-04-17 22:29:08.805762","last_peered":"2024-04-17 22:29:08.805762","last_clean":"2024-04-17 22:29:08.805762","last_became_active":"2024-04-17 22:29:07.826842","last_became_peered":"2024-04-17 22:29:07.826842","last_unstale":"2024-04-17 22:29:08.805762","last_undegraded":"2024-04-17 22:29:08.805762","last_fullsized":"2024-04-17 22:29:08.805762","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":"11","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:08.802625","last_change":"2024-04-17 22:29:07.477248","last_active":"2024-04-17 22:29:08.802625","last_peered":"2024-04-17 22:29:08.802625","last_clean":"2024-04-17 22:29:08.802625","last_became_active":"2024-04-17 22:29:07.476944","last_became_peered":"2024-04-17 22:29:07.476944","last_unstale":"2024-04-17 22:29:08.802625","last_undegraded":"2024-04-17 22:29:08.802625","last_fullsized":"2024-04-17 22:29:08.802625","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":"10","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:09.398192","last_change":"2024-04-17 22:29:07.822961","last_active":"2024-04-17 22:29:09.398192","last_peered":"2024-04-17 22:29:09.398192","last_clean":"2024-04-17 22:29:09.398192","last_became_active":"2024-04-17 22:29:07.822592","last_became_peered":"2024-04-17 22:29:07.822592","last_unstale":"2024-04-17 22:29:09.398192","last_undegraded":"2024-04-17 22:29:09.398192","last_fullsized":"2024-04-17 22:29:09.398192","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.15","version":"15'3","reported_seq":"14","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:08.800188","last_change":"2024-04-17 22:29:07.485166","last_active":"2024-04-17 22:29:08.800188","last_peered":"2024-04-17 22:29:08.800188","last_clean":"2024-04-17 22:29:08.800188","last_became_active":"2024-04-17 22:29:07.484870","last_became_peered":"2024-04-17 22:29:07.484870","last_unstale":"2024-04-17 22:29:08.800188","last_undegraded":"2024-04-17 22:29:08.800188","last_fullsized":"2024-04-17 22:29:08.800188","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":"10","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:08.803451","last_change":"2024-04-17 22:29:07.814509","last_active":"2024-04-17 22:29:08.803451","last_peered":"2024-04-17 22:29:08.803451","last_clean":"2024-04-17 22:29:08.803451","last_became_active":"2024-04-17 22:29:07.814119","last_became_peered":"2024-04-17 22:29:07.814119","last_unstale":"2024-04-17 22:29:08.803451","last_undegraded":"2024-04-17 22:29:08.803451","last_fullsized":"2024-04-17 22:29:08.803451","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.14","version":"17'3","reported_seq":"14","reported_epoch":"17","state":"active+clean","last_fresh":"2024-04-17 22:29:23.262235","last_change":"2024-04-17 22:29:07.473309","last_active":"2024-04-17 22:29:23.262235","last_peered":"2024-04-17 22:29:23.262235","last_clean":"2024-04-17 22:29:23.262235","last_became_active":"2024-04-17 22:29:07.472976","last_became_peered":"2024-04-17 22:29:07.472976","last_unstale":"2024-04-17 22:29:23.262235","last_undegraded":"2024-04-17 22:29:23.262235","last_fullsized":"2024-04-17 22:29:23.262235","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":634,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":"10","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:08.802306","last_change":"2024-04-17 22:29:07.827915","last_active":"2024-04-17 22:29:08.802306","last_peered":"2024-04-17 22:29:08.802306","last_clean":"2024-04-17 22:29:08.802306","last_became_active":"2024-04-17 22:29:07.827670","last_became_peered":"2024-04-17 22:29:07.827670","last_unstale":"2024-04-17 22:29:08.802306","last_undegraded":"2024-04-17 22:29:08.802306","last_fullsized":"2024-04-17 22:29:08.802306","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"15'1","reported_seq":"12","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:08.802767","last_change":"2024-04-17 22:29:07.471458","last_active":"2024-04-17 22:29:08.802767","last_peered":"2024-04-17 22:29:08.802767","last_clean":"2024-04-17 22:29:08.802767","last_became_active":"2024-04-17 22:29:07.471085","last_became_peered":"2024-04-17 22:29:07.471085","last_unstale":"2024-04-17 22:29:08.802767","last_undegraded":"2024-04-17 22:29:08.802767","last_fullsized":"2024-04-17 22:29:08.802767","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":"10","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:09.398266","last_change":"2024-04-17 22:29:07.829288","last_active":"2024-04-17 22:29:09.398266","last_peered":"2024-04-17 22:29:09.398266","last_clean":"2024-04-17 22:29:09.398266","last_became_active":"2024-04-17 22:29:07.828994","last_became_peered":"2024-04-17 22:29:07.828994","last_unstale":"2024-04-17 22:29:09.398266","last_undegraded":"2024-04-17 22:29:09.398266","last_fullsized":"2024-04-17 22:29:09.398266","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":"11","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:09.397936","last_change":"2024-04-17 22:29:07.480194","last_active":"2024-04-17 22:29:09.397936","last_peered":"2024-04-17 22:29:09.397936","last_clean":"2024-04-17 22:29:09.397936","last_became_active":"2024-04-17 22:29:07.479728","last_became_peered":"2024-04-17 22:29:07.479728","last_unstale":"2024-04-17 22:29:09.397936","last_undegraded":"2024-04-17 22:29:09.397936","last_fullsized":"2024-04-17 22:29:09.397936","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":"10","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:08.799216","last_change":"2024-04-17 22:29:07.815790","last_active":"2024-04-17 22:29:08.799216","last_peered":"2024-04-17 22:29:08.799216","last_clean":"2024-04-17 22:29:08.799216","last_became_active":"2024-04-17 22:29:07.815632","last_became_peered":"2024-04-17 22:29:07.815632","last_unstale":"2024-04-17 22:29:08.799216","last_undegraded":"2024-04-17 22:29:08.799216","last_fullsized":"2024-04-17 22:29:08.799216","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":"11","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:08.802288","last_change":"2024-04-17 22:29:07.475830","last_active":"2024-04-17 22:29:08.802288","last_peered":"2024-04-17 22:29:08.802288","last_clean":"2024-04-17 22:29:08.802288","last_became_active":"2024-04-17 22:29:07.474411","last_became_peered":"2024-04-17 22:29:07.474411","last_unstale":"2024-04-17 22:29:08.802288","last_undegraded":"2024-04-17 22:29:08.802288","last_fullsized":"2024-04-17 22:29:08.802288","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":"10","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:09.397680","last_change":"2024-04-17 22:29:07.826101","last_active":"2024-04-17 22:29:09.397680","last_peered":"2024-04-17 22:29:09.397680","last_clean":"2024-04-17 22:29:09.397680","last_became_active":"2024-04-17 22:29:07.822917","last_became_peered":"2024-04-17 22:29:07.822917","last_unstale":"2024-04-17 22:29:09.397680","last_undegraded":"2024-04-17 22:29:09.397680","last_fullsized":"2024-04-17 22:29:09.397680","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.10","version":"15'4","reported_seq":"15","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:08.800588","last_change":"2024-04-17 22:29:07.457017","last_active":"2024-04-17 22:29:08.800588","last_peered":"2024-04-17 22:29:08.800588","last_clean":"2024-04-17 22:29:08.800588","last_became_active":"2024-04-17 22:29:07.456705","last_became_peered":"2024-04-17 22:29:07.456705","last_unstale":"2024-04-17 22:29:08.800588","last_undegraded":"2024-04-17 22:29:08.800588","last_fullsized":"2024-04-17 22:29:08.800588","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":"10","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:08.800899","last_change":"2024-04-17 22:29:07.820464","last_active":"2024-04-17 22:29:08.800899","last_peered":"2024-04-17 22:29:08.800899","last_clean":"2024-04-17 22:29:08.800899","last_became_active":"2024-04-17 22:29:07.820151","last_became_peered":"2024-04-17 22:29:07.820151","last_unstale":"2024-04-17 22:29:08.800899","last_undegraded":"2024-04-17 22:29:08.800899","last_fullsized":"2024-04-17 22:29:08.800899","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.f","version":"15'1","reported_seq":"12","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:08.806028","last_change":"2024-04-17 22:29:07.478464","last_active":"2024-04-17 22:29:08.806028","last_peered":"2024-04-17 22:29:08.806028","last_clean":"2024-04-17 22:29:08.806028","last_became_active":"2024-04-17 22:29:07.478144","last_became_peered":"2024-04-17 22:29:07.478144","last_unstale":"2024-04-17 22:29:08.806028","last_undegraded":"2024-04-17 22:29:08.806028","last_fullsized":"2024-04-17 22:29:08.806028","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":"10","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:08.805853","last_change":"2024-04-17 22:29:07.815171","last_active":"2024-04-17 22:29:08.805853","last_peered":"2024-04-17 22:29:08.805853","last_clean":"2024-04-17 22:29:08.805853","last_became_active":"2024-04-17 22:29:07.814830","last_became_peered":"2024-04-17 22:29:07.814830","last_unstale":"2024-04-17 22:29:08.805853","last_undegraded":"2024-04-17 22:29:08.805853","last_fullsized":"2024-04-17 22:29:08.805853","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"15'2","reported_seq":"13","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:08.801038","last_change":"2024-04-17 22:29:07.481865","last_active":"2024-04-17 22:29:08.801038","last_peered":"2024-04-17 22:29:08.801038","last_clean":"2024-04-17 22:29:08.801038","last_became_active":"2024-04-17 22:29:07.481506","last_became_peered":"2024-04-17 22:29:07.481506","last_unstale":"2024-04-17 22:29:08.801038","last_undegraded":"2024-04-17 22:29:08.801038","last_fullsized":"2024-04-17 22:29:08.801038","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":"10","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:08.802658","last_change":"2024-04-17 22:29:07.818238","last_active":"2024-04-17 22:29:08.802658","last_peered":"2024-04-17 22:29:08.802658","last_clean":"2024-04-17 22:29:08.802658","last_became_active":"2024-04-17 22:29:07.818060","last_became_peered":"2024-04-17 22:29:07.818060","last_unstale":"2024-04-17 22:29:08.802658","last_undegraded":"2024-04-17 22:29:08.802658","last_fullsized":"2024-04-17 22:29:08.802658","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":"11","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:08.800285","last_change":"2024-04-17 22:29:07.484999","last_active":"2024-04-17 22:29:08.800285","last_peered":"2024-04-17 22:29:08.800285","last_clean":"2024-04-17 22:29:08.800285","last_became_active":"2024-04-17 22:29:07.484668","last_became_peered":"2024-04-17 22:29:07.484668","last_unstale":"2024-04-17 22:29:08.800285","last_undegraded":"2024-04-17 22:29:08.800285","last_fullsized":"2024-04-17 22:29:08.800285","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":"10","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:09.398470","last_change":"2024-04-17 22:29:07.826426","last_active":"2024-04-17 22:29:09.398470","last_peered":"2024-04-17 22:29:09.398470","last_clean":"2024-04-17 22:29:09.398470","last_became_active":"2024-04-17 22:29:07.826109","last_became_peered":"2024-04-17 22:29:07.826109","last_unstale":"2024-04-17 22:29:09.398470","last_undegraded":"2024-04-17 22:29:09.398470","last_fullsized":"2024-04-17 22:29:09.398470","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":"11","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:08.800646","last_change":"2024-04-17 22:29:07.462218","last_active":"2024-04-17 22:29:08.800646","last_peered":"2024-04-17 22:29:08.800646","last_clean":"2024-04-17 22:29:08.800646","last_became_active":"2024-04-17 22:29:07.461842","last_became_peered":"2024-04-17 22:29:07.461842","last_unstale":"2024-04-17 22:29:08.800646","last_undegraded":"2024-04-17 22:29:08.800646","last_fullsized":"2024-04-17 22:29:08.800646","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":"10","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:09.397712","last_change":"2024-04-17 22:29:07.817144","last_active":"2024-04-17 22:29:09.397712","last_peered":"2024-04-17 22:29:09.397712","last_clean":"2024-04-17 22:29:09.397712","last_became_active":"2024-04-17 22:29:07.816859","last_became_peered":"2024-04-17 22:29:07.816859","last_unstale":"2024-04-17 22:29:09.397712","last_undegraded":"2024-04-17 22:29:09.397712","last_fullsized":"2024-04-17 22:29:09.397712","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"15'1","reported_seq":"12","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:09.398046","last_change":"2024-04-17 22:29:07.479816","last_active":"2024-04-17 22:29:09.398046","last_peered":"2024-04-17 22:29:09.398046","last_clean":"2024-04-17 22:29:09.398046","last_became_active":"2024-04-17 22:29:07.479251","last_became_peered":"2024-04-17 22:29:07.479251","last_unstale":"2024-04-17 22:29:09.398046","last_undegraded":"2024-04-17 22:29:09.398046","last_fullsized":"2024-04-17 22:29:09.398046","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":"10","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:09.398161","last_change":"2024-04-17 22:29:07.829215","last_active":"2024-04-17 22:29:09.398161","last_peered":"2024-04-17 22:29:09.398161","last_clean":"2024-04-17 22:29:09.398161","last_became_active":"2024-04-17 22:29:07.828803","last_became_peered":"2024-04-17 22:29:07.828803","last_unstale":"2024-04-17 22:29:09.398161","last_undegraded":"2024-04-17 22:29:09.398161","last_fullsized":"2024-04-17 22:29:09.398161","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":"11","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:08.799648","last_change":"2024-04-17 22:29:07.473128","last_active":"2024-04-17 22:29:08.799648","last_peered":"2024-04-17 22:29:08.799648","last_clean":"2024-04-17 22:29:08.799648","last_became_active":"2024-04-17 22:29:07.472806","last_became_peered":"2024-04-17 22:29:07.472806","last_unstale":"2024-04-17 22:29:08.799648","last_undegraded":"2024-04-17 22:29:08.799648","last_fullsized":"2024-04-17 22:29:08.799648","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":"10","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:09.397925","last_change":"2024-04-17 22:29:07.817630","last_active":"2024-04-17 22:29:09.397925","last_peered":"2024-04-17 22:29:09.397925","last_clean":"2024-04-17 22:29:09.397925","last_became_active":"2024-04-17 22:29:07.817333","last_became_peered":"2024-04-17 22:29:07.817333","last_unstale":"2024-04-17 22:29:09.397925","last_undegraded":"2024-04-17 22:29:09.397925","last_fullsized":"2024-04-17 22:29:09.397925","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":"11","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:08.799500","last_change":"2024-04-17 22:29:07.465053","last_active":"2024-04-17 22:29:08.799500","last_peered":"2024-04-17 22:29:08.799500","last_clean":"2024-04-17 22:29:08.799500","last_became_active":"2024-04-17 22:29:07.464766","last_became_peered":"2024-04-17 22:29:07.464766","last_unstale":"2024-04-17 22:29:08.799500","last_undegraded":"2024-04-17 22:29:08.799500","last_fullsized":"2024-04-17 22:29:08.799500","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":"10","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:09.398181","last_change":"2024-04-17 22:29:07.823514","last_active":"2024-04-17 22:29:09.398181","last_peered":"2024-04-17 22:29:09.398181","last_clean":"2024-04-17 22:29:09.398181","last_became_active":"2024-04-17 22:29:07.823007","last_became_peered":"2024-04-17 22:29:07.823007","last_unstale":"2024-04-17 22:29:09.398181","last_undegraded":"2024-04-17 22:29:09.398181","last_fullsized":"2024-04-17 22:29:09.398181","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":"11","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:08.802037","last_change":"2024-04-17 22:29:07.461568","last_active":"2024-04-17 22:29:08.802037","last_peered":"2024-04-17 22:29:08.802037","last_clean":"2024-04-17 22:29:08.802037","last_became_active":"2024-04-17 22:29:07.461249","last_became_peered":"2024-04-17 22:29:07.461249","last_unstale":"2024-04-17 22:29:08.802037","last_undegraded":"2024-04-17 22:29:08.802037","last_fullsized":"2024-04-17 22:29:08.802037","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":"10","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:09.398949","last_change":"2024-04-17 22:29:07.826342","last_active":"2024-04-17 22:29:09.398949","last_peered":"2024-04-17 22:29:09.398949","last_clean":"2024-04-17 22:29:09.398949","last_became_active":"2024-04-17 22:29:07.826029","last_became_peered":"2024-04-17 22:29:07.826029","last_unstale":"2024-04-17 22:29:09.398949","last_undegraded":"2024-04-17 22:29:09.398949","last_fullsized":"2024-04-17 22:29:09.398949","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":"14","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:08.801226","last_change":"2024-04-17 22:28:59.253977","last_active":"2024-04-17 22:29:08.801226","last_peered":"2024-04-17 22:29:08.801226","last_clean":"2024-04-17 22:29:08.801226","last_became_active":"2024-04-17 22:28:59.253641","last_became_peered":"2024-04-17 22:28:59.253641","last_unstale":"2024-04-17 22:29:08.801226","last_undegraded":"2024-04-17 22:29:08.801226","last_fullsized":"2024-04-17 22:29:08.801226","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:28:58.239187","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:28:58.239187","last_clean_scrub_stamp":"2024-04-17 22:28:58.239187","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":"11","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:08.801068","last_change":"2024-04-17 22:29:07.466907","last_active":"2024-04-17 22:29:08.801068","last_peered":"2024-04-17 22:29:08.801068","last_clean":"2024-04-17 22:29:08.801068","last_became_active":"2024-04-17 22:29:07.466583","last_became_peered":"2024-04-17 22:29:07.466583","last_unstale":"2024-04-17 22:29:08.801068","last_undegraded":"2024-04-17 22:29:08.801068","last_fullsized":"2024-04-17 22:29:08.801068","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":"10","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:08.802634","last_change":"2024-04-17 22:29:07.824875","last_active":"2024-04-17 22:29:08.802634","last_peered":"2024-04-17 22:29:08.802634","last_clean":"2024-04-17 22:29:08.802634","last_became_active":"2024-04-17 22:29:07.824377","last_became_peered":"2024-04-17 22:29:07.824377","last_unstale":"2024-04-17 22:29:08.802634","last_undegraded":"2024-04-17 22:29:08.802634","last_fullsized":"2024-04-17 22:29:08.802634","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":"14","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:09.398292","last_change":"2024-04-17 22:28:59.255733","last_active":"2024-04-17 22:29:09.398292","last_peered":"2024-04-17 22:29:09.398292","last_clean":"2024-04-17 22:29:09.398292","last_became_active":"2024-04-17 22:28:59.255411","last_became_peered":"2024-04-17 22:28:59.255411","last_unstale":"2024-04-17 22:29:09.398292","last_undegraded":"2024-04-17 22:29:09.398292","last_fullsized":"2024-04-17 22:29:09.398292","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:28:58.239187","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:28:58.239187","last_clean_scrub_stamp":"2024-04-17 22:28:58.239187","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"15'2","reported_seq":"13","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:09.398794","last_change":"2024-04-17 22:29:07.478196","last_active":"2024-04-17 22:29:09.398794","last_peered":"2024-04-17 22:29:09.398794","last_clean":"2024-04-17 22:29:09.398794","last_became_active":"2024-04-17 22:29:07.477887","last_became_peered":"2024-04-17 22:29:07.477887","last_unstale":"2024-04-17 22:29:09.398794","last_undegraded":"2024-04-17 22:29:09.398794","last_fullsized":"2024-04-17 22:29:09.398794","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":"14","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:08.799327","last_change":"2024-04-17 22:28:59.252728","last_active":"2024-04-17 22:29:08.799327","last_peered":"2024-04-17 22:29:08.799327","last_clean":"2024-04-17 22:29:08.799327","last_became_active":"2024-04-17 22:28:59.252345","last_became_peered":"2024-04-17 22:28:59.252345","last_unstale":"2024-04-17 22:29:08.799327","last_undegraded":"2024-04-17 22:29:08.799327","last_fullsized":"2024-04-17 22:29:08.799327","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:28:58.239187","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:28:58.239187","last_clean_scrub_stamp":"2024-04-17 22:28:58.239187","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":"11","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:09.397593","last_change":"2024-04-17 22:29:07.474520","last_active":"2024-04-17 22:29:09.397593","last_peered":"2024-04-17 22:29:09.397593","last_clean":"2024-04-17 22:29:09.397593","last_became_active":"2024-04-17 22:29:07.474029","last_became_peered":"2024-04-17 22:29:07.474029","last_unstale":"2024-04-17 22:29:09.397593","last_undegraded":"2024-04-17 22:29:09.397593","last_fullsized":"2024-04-17 22:29:09.397593","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":"10","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:08.802181","last_change":"2024-04-17 22:29:07.825091","last_active":"2024-04-17 22:29:08.802181","last_peered":"2024-04-17 22:29:08.802181","last_clean":"2024-04-17 22:29:08.802181","last_became_active":"2024-04-17 22:29:07.824832","last_became_peered":"2024-04-17 22:29:07.824832","last_unstale":"2024-04-17 22:29:08.802181","last_undegraded":"2024-04-17 22:29:08.802181","last_fullsized":"2024-04-17 22:29:08.802181","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":"14","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:08.802193","last_change":"2024-04-17 22:28:59.253392","last_active":"2024-04-17 22:29:08.802193","last_peered":"2024-04-17 22:29:08.802193","last_clean":"2024-04-17 22:29:08.802193","last_became_active":"2024-04-17 22:28:59.253049","last_became_peered":"2024-04-17 22:28:59.253049","last_unstale":"2024-04-17 22:29:08.802193","last_undegraded":"2024-04-17 22:29:08.802193","last_fullsized":"2024-04-17 22:29:08.802193","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:28:58.239187","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:28:58.239187","last_clean_scrub_stamp":"2024-04-17 22:28:58.239187","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":"11","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:08.800158","last_change":"2024-04-17 22:29:07.471913","last_active":"2024-04-17 22:29:08.800158","last_peered":"2024-04-17 22:29:08.800158","last_clean":"2024-04-17 22:29:08.800158","last_became_active":"2024-04-17 22:29:07.471686","last_became_peered":"2024-04-17 22:29:07.471686","last_unstale":"2024-04-17 22:29:08.800158","last_undegraded":"2024-04-17 22:29:08.800158","last_fullsized":"2024-04-17 22:29:08.800158","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":"10","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:08.799248","last_change":"2024-04-17 22:29:07.812999","last_active":"2024-04-17 22:29:08.799248","last_peered":"2024-04-17 22:29:08.799248","last_clean":"2024-04-17 22:29:08.799248","last_became_active":"2024-04-17 22:29:07.812701","last_became_peered":"2024-04-17 22:29:07.812701","last_unstale":"2024-04-17 22:29:08.799248","last_undegraded":"2024-04-17 22:29:08.799248","last_fullsized":"2024-04-17 22:29:08.799248","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":"14","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:08.799327","last_change":"2024-04-17 22:28:59.254907","last_active":"2024-04-17 22:29:08.799327","last_peered":"2024-04-17 22:29:08.799327","last_clean":"2024-04-17 22:29:08.799327","last_became_active":"2024-04-17 22:28:59.254633","last_became_peered":"2024-04-17 22:28:59.254633","last_unstale":"2024-04-17 22:29:08.799327","last_undegraded":"2024-04-17 22:29:08.799327","last_fullsized":"2024-04-17 22:29:08.799327","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:28:58.239187","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:28:58.239187","last_clean_scrub_stamp":"2024-04-17 22:28:58.239187","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":"11","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:09.398108","last_change":"2024-04-17 22:29:07.480020","last_active":"2024-04-17 22:29:09.398108","last_peered":"2024-04-17 22:29:09.398108","last_clean":"2024-04-17 22:29:09.398108","last_became_active":"2024-04-17 22:29:07.479492","last_became_peered":"2024-04-17 22:29:07.479492","last_unstale":"2024-04-17 22:29:09.398108","last_undegraded":"2024-04-17 22:29:09.398108","last_fullsized":"2024-04-17 22:29:09.398108","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":"10","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:09.397806","last_change":"2024-04-17 22:29:07.821883","last_active":"2024-04-17 22:29:09.397806","last_peered":"2024-04-17 22:29:09.397806","last_clean":"2024-04-17 22:29:09.397806","last_became_active":"2024-04-17 22:29:07.821569","last_became_peered":"2024-04-17 22:29:07.821569","last_unstale":"2024-04-17 22:29:09.397806","last_undegraded":"2024-04-17 22:29:09.397806","last_fullsized":"2024-04-17 22:29:09.397806","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":"14","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:09.397706","last_change":"2024-04-17 22:28:59.254217","last_active":"2024-04-17 22:29:09.397706","last_peered":"2024-04-17 22:29:09.397706","last_clean":"2024-04-17 22:29:09.397706","last_became_active":"2024-04-17 22:28:59.253877","last_became_peered":"2024-04-17 22:28:59.253877","last_unstale":"2024-04-17 22:29:09.397706","last_undegraded":"2024-04-17 22:29:09.397706","last_fullsized":"2024-04-17 22:29:09.397706","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:28:58.239187","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:28:58.239187","last_clean_scrub_stamp":"2024-04-17 22:28:58.239187","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":"11","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:09.398009","last_change":"2024-04-17 22:29:07.480378","last_active":"2024-04-17 22:29:09.398009","last_peered":"2024-04-17 22:29:09.398009","last_clean":"2024-04-17 22:29:09.398009","last_became_active":"2024-04-17 22:29:07.479866","last_became_peered":"2024-04-17 22:29:07.479866","last_unstale":"2024-04-17 22:29:09.398009","last_undegraded":"2024-04-17 22:29:09.398009","last_fullsized":"2024-04-17 22:29:09.398009","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":"10","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:08.802861","last_change":"2024-04-17 22:29:07.821315","last_active":"2024-04-17 22:29:08.802861","last_peered":"2024-04-17 22:29:08.802861","last_clean":"2024-04-17 22:29:08.802861","last_became_active":"2024-04-17 22:29:07.820937","last_became_peered":"2024-04-17 22:29:07.820937","last_unstale":"2024-04-17 22:29:08.802861","last_undegraded":"2024-04-17 22:29:08.802861","last_fullsized":"2024-04-17 22:29:08.802861","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":"14","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:09.398670","last_change":"2024-04-17 22:28:59.254830","last_active":"2024-04-17 22:29:09.398670","last_peered":"2024-04-17 22:29:09.398670","last_clean":"2024-04-17 22:29:09.398670","last_became_active":"2024-04-17 22:28:59.254505","last_became_peered":"2024-04-17 22:28:59.254505","last_unstale":"2024-04-17 22:29:09.398670","last_undegraded":"2024-04-17 22:29:09.398670","last_fullsized":"2024-04-17 22:29:09.398670","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:28:58.239187","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:28:58.239187","last_clean_scrub_stamp":"2024-04-17 22:28:58.239187","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":"11","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:09.398561","last_change":"2024-04-17 22:29:07.475017","last_active":"2024-04-17 22:29:09.398561","last_peered":"2024-04-17 22:29:09.398561","last_clean":"2024-04-17 22:29:09.398561","last_became_active":"2024-04-17 22:29:07.473559","last_became_peered":"2024-04-17 22:29:07.473559","last_unstale":"2024-04-17 22:29:09.398561","last_undegraded":"2024-04-17 22:29:09.398561","last_fullsized":"2024-04-17 22:29:09.398561","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":"10","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:08.801429","last_change":"2024-04-17 22:29:07.808988","last_active":"2024-04-17 22:29:08.801429","last_peered":"2024-04-17 22:29:08.801429","last_clean":"2024-04-17 22:29:08.801429","last_became_active":"2024-04-17 22:29:07.808663","last_became_peered":"2024-04-17 22:29:07.808663","last_unstale":"2024-04-17 22:29:08.801429","last_undegraded":"2024-04-17 22:29:08.801429","last_fullsized":"2024-04-17 22:29:08.801429","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":"14","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:08.800951","last_change":"2024-04-17 22:28:59.254055","last_active":"2024-04-17 22:29:08.800951","last_peered":"2024-04-17 22:29:08.800951","last_clean":"2024-04-17 22:29:08.800951","last_became_active":"2024-04-17 22:28:59.253750","last_became_peered":"2024-04-17 22:28:59.253750","last_unstale":"2024-04-17 22:29:08.800951","last_undegraded":"2024-04-17 22:29:08.800951","last_fullsized":"2024-04-17 22:29:08.800951","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:28:58.239187","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:28:58.239187","last_clean_scrub_stamp":"2024-04-17 22:28:58.239187","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":"11","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:08.801348","last_change":"2024-04-17 22:29:07.466238","last_active":"2024-04-17 22:29:08.801348","last_peered":"2024-04-17 22:29:08.801348","last_clean":"2024-04-17 22:29:08.801348","last_became_active":"2024-04-17 22:29:07.465806","last_became_peered":"2024-04-17 22:29:07.465806","last_unstale":"2024-04-17 22:29:08.801348","last_undegraded":"2024-04-17 22:29:08.801348","last_fullsized":"2024-04-17 22:29:08.801348","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":"10","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:08.800621","last_change":"2024-04-17 22:29:07.809155","last_active":"2024-04-17 22:29:08.800621","last_peered":"2024-04-17 22:29:08.800621","last_clean":"2024-04-17 22:29:08.800621","last_became_active":"2024-04-17 22:29:07.808850","last_became_peered":"2024-04-17 22:29:07.808850","last_unstale":"2024-04-17 22:29:08.800621","last_undegraded":"2024-04-17 22:29:08.800621","last_fullsized":"2024-04-17 22:29:08.800621","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":"10","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:09.398583","last_change":"2024-04-17 22:29:07.832794","last_active":"2024-04-17 22:29:09.398583","last_peered":"2024-04-17 22:29:09.398583","last_clean":"2024-04-17 22:29:09.398583","last_became_active":"2024-04-17 22:29:07.832467","last_became_peered":"2024-04-17 22:29:07.832467","last_unstale":"2024-04-17 22:29:09.398583","last_undegraded":"2024-04-17 22:29:09.398583","last_fullsized":"2024-04-17 22:29:09.398583","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":"11","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:08.802745","last_change":"2024-04-17 22:29:07.477743","last_active":"2024-04-17 22:29:08.802745","last_peered":"2024-04-17 22:29:08.802745","last_clean":"2024-04-17 22:29:08.802745","last_became_active":"2024-04-17 22:29:07.477503","last_became_peered":"2024-04-17 22:29:07.477503","last_unstale":"2024-04-17 22:29:08.802745","last_undegraded":"2024-04-17 22:29:08.802745","last_fullsized":"2024-04-17 22:29:08.802745","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":"10","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:08.801269","last_change":"2024-04-17 22:29:07.828401","last_active":"2024-04-17 22:29:08.801269","last_peered":"2024-04-17 22:29:08.801269","last_clean":"2024-04-17 22:29:08.801269","last_became_active":"2024-04-17 22:29:07.828093","last_became_peered":"2024-04-17 22:29:07.828093","last_unstale":"2024-04-17 22:29:08.801269","last_undegraded":"2024-04-17 22:29:08.801269","last_fullsized":"2024-04-17 22:29:08.801269","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"15'2","reported_seq":"13","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:08.803325","last_change":"2024-04-17 22:29:07.478361","last_active":"2024-04-17 22:29:08.803325","last_peered":"2024-04-17 22:29:08.803325","last_clean":"2024-04-17 22:29:08.803325","last_became_active":"2024-04-17 22:29:07.477793","last_became_peered":"2024-04-17 22:29:07.477793","last_unstale":"2024-04-17 22:29:08.803325","last_undegraded":"2024-04-17 22:29:08.803325","last_fullsized":"2024-04-17 22:29:08.803325","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":"10","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:08.805677","last_change":"2024-04-17 22:29:07.815548","last_active":"2024-04-17 22:29:08.805677","last_peered":"2024-04-17 22:29:08.805677","last_clean":"2024-04-17 22:29:08.805677","last_became_active":"2024-04-17 22:29:07.815263","last_became_peered":"2024-04-17 22:29:07.815263","last_unstale":"2024-04-17 22:29:08.805677","last_undegraded":"2024-04-17 22:29:08.805677","last_fullsized":"2024-04-17 22:29:08.805677","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"15'1","reported_seq":"12","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:09.398690","last_change":"2024-04-17 22:29:07.482507","last_active":"2024-04-17 22:29:09.398690","last_peered":"2024-04-17 22:29:09.398690","last_clean":"2024-04-17 22:29:09.398690","last_became_active":"2024-04-17 22:29:07.481981","last_became_peered":"2024-04-17 22:29:07.481981","last_unstale":"2024-04-17 22:29:09.398690","last_undegraded":"2024-04-17 22:29:09.398690","last_fullsized":"2024-04-17 22:29:09.398690","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":"10","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:09.398811","last_change":"2024-04-17 22:29:07.828459","last_active":"2024-04-17 22:29:09.398811","last_peered":"2024-04-17 22:29:09.398811","last_clean":"2024-04-17 22:29:09.398811","last_became_active":"2024-04-17 22:29:07.828157","last_became_peered":"2024-04-17 22:29:07.828157","last_unstale":"2024-04-17 22:29:09.398811","last_undegraded":"2024-04-17 22:29:09.398811","last_fullsized":"2024-04-17 22:29:09.398811","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"15'1","reported_seq":"12","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:08.805937","last_change":"2024-04-17 22:29:07.471751","last_active":"2024-04-17 22:29:08.805937","last_peered":"2024-04-17 22:29:08.805937","last_clean":"2024-04-17 22:29:08.805937","last_became_active":"2024-04-17 22:29:07.471385","last_became_peered":"2024-04-17 22:29:07.471385","last_unstale":"2024-04-17 22:29:08.805937","last_undegraded":"2024-04-17 22:29:08.805937","last_fullsized":"2024-04-17 22:29:08.805937","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":"10","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:08.800362","last_change":"2024-04-17 22:29:07.812125","last_active":"2024-04-17 22:29:08.800362","last_peered":"2024-04-17 22:29:08.800362","last_clean":"2024-04-17 22:29:08.800362","last_became_active":"2024-04-17 22:29:07.811846","last_became_peered":"2024-04-17 22:29:07.811846","last_unstale":"2024-04-17 22:29:08.800362","last_undegraded":"2024-04-17 22:29:08.800362","last_fullsized":"2024-04-17 22:29:08.800362","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.20","version":"0'0","reported_seq":"11","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:09.397688","last_change":"2024-04-17 22:29:07.477127","last_active":"2024-04-17 22:29:09.397688","last_peered":"2024-04-17 22:29:09.397688","last_clean":"2024-04-17 22:29:09.397688","last_became_active":"2024-04-17 22:29:07.476773","last_became_peered":"2024-04-17 22:29:07.476773","last_unstale":"2024-04-17 22:29:09.397688","last_undegraded":"2024-04-17 22:29:09.397688","last_fullsized":"2024-04-17 22:29:09.397688","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":"10","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:08.800687","last_change":"2024-04-17 22:29:07.821132","last_active":"2024-04-17 22:29:08.800687","last_peered":"2024-04-17 22:29:08.800687","last_clean":"2024-04-17 22:29:08.800687","last_became_active":"2024-04-17 22:29:07.820838","last_became_peered":"2024-04-17 22:29:07.820838","last_unstale":"2024-04-17 22:29:08.800687","last_undegraded":"2024-04-17 22:29:08.800687","last_fullsized":"2024-04-17 22:29:08.800687","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.21","version":"0'0","reported_seq":"11","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:08.800070","last_change":"2024-04-17 22:29:07.460110","last_active":"2024-04-17 22:29:08.800070","last_peered":"2024-04-17 22:29:08.800070","last_clean":"2024-04-17 22:29:08.800070","last_became_active":"2024-04-17 22:29:07.459611","last_became_peered":"2024-04-17 22:29:07.459611","last_unstale":"2024-04-17 22:29:08.800070","last_undegraded":"2024-04-17 22:29:08.800070","last_fullsized":"2024-04-17 22:29:08.800070","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":"10","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:08.802908","last_change":"2024-04-17 22:29:07.818496","last_active":"2024-04-17 22:29:08.802908","last_peered":"2024-04-17 22:29:08.802908","last_clean":"2024-04-17 22:29:08.802908","last_became_active":"2024-04-17 22:29:07.818184","last_became_peered":"2024-04-17 22:29:07.818184","last_unstale":"2024-04-17 22:29:08.802908","last_undegraded":"2024-04-17 22:29:08.802908","last_fullsized":"2024-04-17 22:29:08.802908","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.22","version":"0'0","reported_seq":"11","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:09.398098","last_change":"2024-04-17 22:29:07.481695","last_active":"2024-04-17 22:29:09.398098","last_peered":"2024-04-17 22:29:09.398098","last_clean":"2024-04-17 22:29:09.398098","last_became_active":"2024-04-17 22:29:07.479982","last_became_peered":"2024-04-17 22:29:07.479982","last_unstale":"2024-04-17 22:29:09.398098","last_undegraded":"2024-04-17 22:29:09.398098","last_fullsized":"2024-04-17 22:29:09.398098","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":"10","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:09.397819","last_change":"2024-04-17 22:29:07.833317","last_active":"2024-04-17 22:29:09.397819","last_peered":"2024-04-17 22:29:09.397819","last_clean":"2024-04-17 22:29:09.397819","last_became_active":"2024-04-17 22:29:07.827050","last_became_peered":"2024-04-17 22:29:07.827050","last_unstale":"2024-04-17 22:29:09.397819","last_undegraded":"2024-04-17 22:29:09.397819","last_fullsized":"2024-04-17 22:29:09.397819","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.23","version":"0'0","reported_seq":"11","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:09.398251","last_change":"2024-04-17 22:29:07.477449","last_active":"2024-04-17 22:29:09.398251","last_peered":"2024-04-17 22:29:09.398251","last_clean":"2024-04-17 22:29:09.398251","last_became_active":"2024-04-17 22:29:07.477155","last_became_peered":"2024-04-17 22:29:07.477155","last_unstale":"2024-04-17 22:29:09.398251","last_undegraded":"2024-04-17 22:29:09.398251","last_fullsized":"2024-04-17 22:29:09.398251","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":"10","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:08.800540","last_change":"2024-04-17 22:29:07.808974","last_active":"2024-04-17 22:29:08.800540","last_peered":"2024-04-17 22:29:08.800540","last_clean":"2024-04-17 22:29:08.800540","last_became_active":"2024-04-17 22:29:07.808747","last_became_peered":"2024-04-17 22:29:07.808747","last_unstale":"2024-04-17 22:29:08.800540","last_undegraded":"2024-04-17 22:29:08.800540","last_fullsized":"2024-04-17 22:29:08.800540","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.24","version":"0'0","reported_seq":"11","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:08.799263","last_change":"2024-04-17 22:29:07.466052","last_active":"2024-04-17 22:29:08.799263","last_peered":"2024-04-17 22:29:08.799263","last_clean":"2024-04-17 22:29:08.799263","last_became_active":"2024-04-17 22:29:07.465576","last_became_peered":"2024-04-17 22:29:07.465576","last_unstale":"2024-04-17 22:29:08.799263","last_undegraded":"2024-04-17 22:29:08.799263","last_fullsized":"2024-04-17 22:29:08.799263","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":"10","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:09.398346","last_change":"2024-04-17 22:29:07.829066","last_active":"2024-04-17 22:29:09.398346","last_peered":"2024-04-17 22:29:09.398346","last_clean":"2024-04-17 22:29:09.398346","last_became_active":"2024-04-17 22:29:07.828735","last_became_peered":"2024-04-17 22:29:07.828735","last_unstale":"2024-04-17 22:29:09.398346","last_undegraded":"2024-04-17 22:29:09.398346","last_fullsized":"2024-04-17 22:29:09.398346","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.25","version":"0'0","reported_seq":"11","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:08.800709","last_change":"2024-04-17 22:29:07.462011","last_active":"2024-04-17 22:29:08.800709","last_peered":"2024-04-17 22:29:08.800709","last_clean":"2024-04-17 22:29:08.800709","last_became_active":"2024-04-17 22:29:07.461693","last_became_peered":"2024-04-17 22:29:07.461693","last_unstale":"2024-04-17 22:29:08.800709","last_undegraded":"2024-04-17 22:29:08.800709","last_fullsized":"2024-04-17 22:29:08.800709","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":"10","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:09.398722","last_change":"2024-04-17 22:29:07.826655","last_active":"2024-04-17 22:29:09.398722","last_peered":"2024-04-17 22:29:09.398722","last_clean":"2024-04-17 22:29:09.398722","last_became_active":"2024-04-17 22:29:07.823665","last_became_peered":"2024-04-17 22:29:07.823665","last_unstale":"2024-04-17 22:29:09.398722","last_undegraded":"2024-04-17 22:29:09.398722","last_fullsized":"2024-04-17 22:29:09.398722","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.26","version":"16'1","reported_seq":"12","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:09.396977","last_change":"2024-04-17 22:29:07.480854","last_active":"2024-04-17 22:29:09.396977","last_peered":"2024-04-17 22:29:09.396977","last_clean":"2024-04-17 22:29:09.396977","last_became_active":"2024-04-17 22:29:07.479975","last_became_peered":"2024-04-17 22:29:07.479975","last_unstale":"2024-04-17 22:29:09.396977","last_undegraded":"2024-04-17 22:29:09.396977","last_fullsized":"2024-04-17 22:29:09.396977","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":"10","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:08.801236","last_change":"2024-04-17 22:29:07.813744","last_active":"2024-04-17 22:29:08.801236","last_peered":"2024-04-17 22:29:08.801236","last_clean":"2024-04-17 22:29:08.801236","last_became_active":"2024-04-17 22:29:07.810646","last_became_peered":"2024-04-17 22:29:07.810646","last_unstale":"2024-04-17 22:29:08.801236","last_undegraded":"2024-04-17 22:29:08.801236","last_fullsized":"2024-04-17 22:29:08.801236","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.27","version":"0'0","reported_seq":"11","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:09.398895","last_change":"2024-04-17 22:29:07.476428","last_active":"2024-04-17 22:29:09.398895","last_peered":"2024-04-17 22:29:09.398895","last_clean":"2024-04-17 22:29:09.398895","last_became_active":"2024-04-17 22:29:07.476128","last_became_peered":"2024-04-17 22:29:07.476128","last_unstale":"2024-04-17 22:29:09.398895","last_undegraded":"2024-04-17 22:29:09.398895","last_fullsized":"2024-04-17 22:29:09.398895","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":"10","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:09.397602","last_change":"2024-04-17 22:29:07.823340","last_active":"2024-04-17 22:29:09.397602","last_peered":"2024-04-17 22:29:09.397602","last_clean":"2024-04-17 22:29:09.397602","last_became_active":"2024-04-17 22:29:07.822764","last_became_peered":"2024-04-17 22:29:07.822764","last_unstale":"2024-04-17 22:29:09.397602","last_undegraded":"2024-04-17 22:29:09.397602","last_fullsized":"2024-04-17 22:29:09.397602","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.28","version":"0'0","reported_seq":"11","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:09.397875","last_change":"2024-04-17 22:29:07.477137","last_active":"2024-04-17 22:29:09.397875","last_peered":"2024-04-17 22:29:09.397875","last_clean":"2024-04-17 22:29:09.397875","last_became_active":"2024-04-17 22:29:07.476796","last_became_peered":"2024-04-17 22:29:07.476796","last_unstale":"2024-04-17 22:29:09.397875","last_undegraded":"2024-04-17 22:29:09.397875","last_fullsized":"2024-04-17 22:29:09.397875","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":"10","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:09.397527","last_change":"2024-04-17 22:29:07.823119","last_active":"2024-04-17 22:29:09.397527","last_peered":"2024-04-17 22:29:09.397527","last_clean":"2024-04-17 22:29:09.397527","last_became_active":"2024-04-17 22:29:07.822640","last_became_peered":"2024-04-17 22:29:07.822640","last_unstale":"2024-04-17 22:29:09.397527","last_undegraded":"2024-04-17 22:29:09.397527","last_fullsized":"2024-04-17 22:29:09.397527","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.29","version":"0'0","reported_seq":"11","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:08.799663","last_change":"2024-04-17 22:29:07.473467","last_active":"2024-04-17 22:29:08.799663","last_peered":"2024-04-17 22:29:08.799663","last_clean":"2024-04-17 22:29:08.799663","last_became_active":"2024-04-17 22:29:07.473028","last_became_peered":"2024-04-17 22:29:07.473028","last_unstale":"2024-04-17 22:29:08.799663","last_undegraded":"2024-04-17 22:29:08.799663","last_fullsized":"2024-04-17 22:29:08.799663","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":"10","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:08.802996","last_change":"2024-04-17 22:29:07.817097","last_active":"2024-04-17 22:29:08.802996","last_peered":"2024-04-17 22:29:08.802996","last_clean":"2024-04-17 22:29:08.802996","last_became_active":"2024-04-17 22:29:07.816813","last_became_peered":"2024-04-17 22:29:07.816813","last_unstale":"2024-04-17 22:29:08.802996","last_undegraded":"2024-04-17 22:29:08.802996","last_fullsized":"2024-04-17 22:29:08.802996","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2a","version":"0'0","reported_seq":"11","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:08.799564","last_change":"2024-04-17 22:29:07.464330","last_active":"2024-04-17 22:29:08.799564","last_peered":"2024-04-17 22:29:08.799564","last_clean":"2024-04-17 22:29:08.799564","last_became_active":"2024-04-17 22:29:07.464002","last_became_peered":"2024-04-17 22:29:07.464002","last_unstale":"2024-04-17 22:29:08.799564","last_undegraded":"2024-04-17 22:29:08.799564","last_fullsized":"2024-04-17 22:29:08.799564","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":"10","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:08.799834","last_change":"2024-04-17 22:29:07.828690","last_active":"2024-04-17 22:29:08.799834","last_peered":"2024-04-17 22:29:08.799834","last_clean":"2024-04-17 22:29:08.799834","last_became_active":"2024-04-17 22:29:07.828407","last_became_peered":"2024-04-17 22:29:07.828407","last_unstale":"2024-04-17 22:29:08.799834","last_undegraded":"2024-04-17 22:29:08.799834","last_fullsized":"2024-04-17 22:29:08.799834","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2b","version":"15'1","reported_seq":"12","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:08.800356","last_change":"2024-04-17 22:29:07.471300","last_active":"2024-04-17 22:29:08.800356","last_peered":"2024-04-17 22:29:08.800356","last_clean":"2024-04-17 22:29:08.800356","last_became_active":"2024-04-17 22:29:07.470886","last_became_peered":"2024-04-17 22:29:07.470886","last_unstale":"2024-04-17 22:29:08.800356","last_undegraded":"2024-04-17 22:29:08.800356","last_fullsized":"2024-04-17 22:29:08.800356","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":"10","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:09.398460","last_change":"2024-04-17 22:29:07.826555","last_active":"2024-04-17 22:29:09.398460","last_peered":"2024-04-17 22:29:09.398460","last_clean":"2024-04-17 22:29:09.398460","last_became_active":"2024-04-17 22:29:07.826262","last_became_peered":"2024-04-17 22:29:07.826262","last_unstale":"2024-04-17 22:29:09.398460","last_undegraded":"2024-04-17 22:29:09.398460","last_fullsized":"2024-04-17 22:29:09.398460","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2c","version":"0'0","reported_seq":"11","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:08.801315","last_change":"2024-04-17 22:29:07.462176","last_active":"2024-04-17 22:29:08.801315","last_peered":"2024-04-17 22:29:08.801315","last_clean":"2024-04-17 22:29:08.801315","last_became_active":"2024-04-17 22:29:07.461594","last_became_peered":"2024-04-17 22:29:07.461594","last_unstale":"2024-04-17 22:29:08.801315","last_undegraded":"2024-04-17 22:29:08.801315","last_fullsized":"2024-04-17 22:29:08.801315","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":"10","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:08.802743","last_change":"2024-04-17 22:29:07.816896","last_active":"2024-04-17 22:29:08.802743","last_peered":"2024-04-17 22:29:08.802743","last_clean":"2024-04-17 22:29:08.802743","last_became_active":"2024-04-17 22:29:07.816538","last_became_peered":"2024-04-17 22:29:07.816538","last_unstale":"2024-04-17 22:29:08.802743","last_undegraded":"2024-04-17 22:29:08.802743","last_fullsized":"2024-04-17 22:29:08.802743","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2d","version":"15'1","reported_seq":"12","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:09.398663","last_change":"2024-04-17 22:29:07.479214","last_active":"2024-04-17 22:29:09.398663","last_peered":"2024-04-17 22:29:09.398663","last_clean":"2024-04-17 22:29:09.398663","last_became_active":"2024-04-17 22:29:07.478790","last_became_peered":"2024-04-17 22:29:07.478790","last_unstale":"2024-04-17 22:29:09.398663","last_undegraded":"2024-04-17 22:29:09.398663","last_fullsized":"2024-04-17 22:29:09.398663","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":"10","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:09.398672","last_change":"2024-04-17 22:29:07.832349","last_active":"2024-04-17 22:29:09.398672","last_peered":"2024-04-17 22:29:09.398672","last_clean":"2024-04-17 22:29:09.398672","last_became_active":"2024-04-17 22:29:07.831999","last_became_peered":"2024-04-17 22:29:07.831999","last_unstale":"2024-04-17 22:29:09.398672","last_undegraded":"2024-04-17 22:29:09.398672","last_fullsized":"2024-04-17 22:29:09.398672","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2e","version":"0'0","reported_seq":"11","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:08.801154","last_change":"2024-04-17 22:29:07.483244","last_active":"2024-04-17 22:29:08.801154","last_peered":"2024-04-17 22:29:08.801154","last_clean":"2024-04-17 22:29:08.801154","last_became_active":"2024-04-17 22:29:07.482893","last_became_peered":"2024-04-17 22:29:07.482893","last_unstale":"2024-04-17 22:29:08.801154","last_undegraded":"2024-04-17 22:29:08.801154","last_fullsized":"2024-04-17 22:29:08.801154","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":"10","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:08.801323","last_change":"2024-04-17 22:29:07.806512","last_active":"2024-04-17 22:29:08.801323","last_peered":"2024-04-17 22:29:08.801323","last_clean":"2024-04-17 22:29:08.801323","last_became_active":"2024-04-17 22:29:07.806182","last_became_peered":"2024-04-17 22:29:07.806182","last_unstale":"2024-04-17 22:29:08.801323","last_undegraded":"2024-04-17 22:29:08.801323","last_fullsized":"2024-04-17 22:29:08.801323","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2f","version":"0'0","reported_seq":"11","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:09.398900","last_change":"2024-04-17 22:29:07.478655","last_active":"2024-04-17 22:29:09.398900","last_peered":"2024-04-17 22:29:09.398900","last_clean":"2024-04-17 22:29:09.398900","last_became_active":"2024-04-17 22:29:07.478346","last_became_peered":"2024-04-17 22:29:07.478346","last_unstale":"2024-04-17 22:29:09.398900","last_undegraded":"2024-04-17 22:29:09.398900","last_fullsized":"2024-04-17 22:29:09.398900","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":"10","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:08.799410","last_change":"2024-04-17 22:29:07.825658","last_active":"2024-04-17 22:29:08.799410","last_peered":"2024-04-17 22:29:08.799410","last_clean":"2024-04-17 22:29:08.799410","last_became_active":"2024-04-17 22:29:07.825349","last_became_peered":"2024-04-17 22:29:07.825349","last_unstale":"2024-04-17 22:29:08.799410","last_undegraded":"2024-04-17 22:29:08.799410","last_fullsized":"2024-04-17 22:29:08.799410","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.30","version":"15'4","reported_seq":"15","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:08.802113","last_change":"2024-04-17 22:29:07.475470","last_active":"2024-04-17 22:29:08.802113","last_peered":"2024-04-17 22:29:08.802113","last_clean":"2024-04-17 22:29:08.802113","last_became_active":"2024-04-17 22:29:07.474917","last_became_peered":"2024-04-17 22:29:07.474917","last_unstale":"2024-04-17 22:29:08.802113","last_undegraded":"2024-04-17 22:29:08.802113","last_fullsized":"2024-04-17 22:29:08.802113","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":"10","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:09.397646","last_change":"2024-04-17 22:29:07.823624","last_active":"2024-04-17 22:29:09.397646","last_peered":"2024-04-17 22:29:09.397646","last_clean":"2024-04-17 22:29:09.397646","last_became_active":"2024-04-17 22:29:07.822815","last_became_peered":"2024-04-17 22:29:07.822815","last_unstale":"2024-04-17 22:29:09.397646","last_undegraded":"2024-04-17 22:29:09.397646","last_fullsized":"2024-04-17 22:29:09.397646","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.31","version":"0'0","reported_seq":"11","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:08.802386","last_change":"2024-04-17 22:29:07.474318","last_active":"2024-04-17 22:29:08.802386","last_peered":"2024-04-17 22:29:08.802386","last_clean":"2024-04-17 22:29:08.802386","last_became_active":"2024-04-17 22:29:07.473754","last_became_peered":"2024-04-17 22:29:07.473754","last_unstale":"2024-04-17 22:29:08.802386","last_undegraded":"2024-04-17 22:29:08.802386","last_fullsized":"2024-04-17 22:29:08.802386","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":"10","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:09.398038","last_change":"2024-04-17 22:29:07.829098","last_active":"2024-04-17 22:29:09.398038","last_peered":"2024-04-17 22:29:09.398038","last_clean":"2024-04-17 22:29:09.398038","last_became_active":"2024-04-17 22:29:07.828773","last_became_peered":"2024-04-17 22:29:07.828773","last_unstale":"2024-04-17 22:29:09.398038","last_undegraded":"2024-04-17 22:29:09.398038","last_fullsized":"2024-04-17 22:29:09.398038","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.32","version":"0'0","reported_seq":"11","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:08.800366","last_change":"2024-04-17 22:29:07.470666","last_active":"2024-04-17 22:29:08.800366","last_peered":"2024-04-17 22:29:08.800366","last_clean":"2024-04-17 22:29:08.800366","last_became_active":"2024-04-17 22:29:07.470363","last_became_peered":"2024-04-17 22:29:07.470363","last_unstale":"2024-04-17 22:29:08.800366","last_undegraded":"2024-04-17 22:29:08.800366","last_fullsized":"2024-04-17 22:29:08.800366","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":"10","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:08.802294","last_change":"2024-04-17 22:29:07.827580","last_active":"2024-04-17 22:29:08.802294","last_peered":"2024-04-17 22:29:08.802294","last_clean":"2024-04-17 22:29:08.802294","last_became_active":"2024-04-17 22:29:07.827233","last_became_peered":"2024-04-17 22:29:07.827233","last_unstale":"2024-04-17 22:29:08.802294","last_undegraded":"2024-04-17 22:29:08.802294","last_fullsized":"2024-04-17 22:29:08.802294","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.33","version":"15'1","reported_seq":"12","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:08.799425","last_change":"2024-04-17 22:29:07.460978","last_active":"2024-04-17 22:29:08.799425","last_peered":"2024-04-17 22:29:08.799425","last_clean":"2024-04-17 22:29:08.799425","last_became_active":"2024-04-17 22:29:07.460659","last_became_peered":"2024-04-17 22:29:07.460659","last_unstale":"2024-04-17 22:29:08.799425","last_undegraded":"2024-04-17 22:29:08.799425","last_fullsized":"2024-04-17 22:29:08.799425","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":"10","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:08.803388","last_change":"2024-04-17 22:29:07.816335","last_active":"2024-04-17 22:29:08.803388","last_peered":"2024-04-17 22:29:08.803388","last_clean":"2024-04-17 22:29:08.803388","last_became_active":"2024-04-17 22:29:07.816090","last_became_peered":"2024-04-17 22:29:07.816090","last_unstale":"2024-04-17 22:29:08.803388","last_undegraded":"2024-04-17 22:29:08.803388","last_fullsized":"2024-04-17 22:29:08.803388","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.34","version":"17'7","reported_seq":"18","reported_epoch":"17","state":"active+clean","last_fresh":"2024-04-17 22:29:29.339150","last_change":"2024-04-17 22:29:07.475702","last_active":"2024-04-17 22:29:29.339150","last_peered":"2024-04-17 22:29:29.339150","last_clean":"2024-04-17 22:29:29.339150","last_became_active":"2024-04-17 22:29:07.475395","last_became_peered":"2024-04-17 22:29:07.475395","last_unstale":"2024-04-17 22:29:29.339150","last_undegraded":"2024-04-17 22:29:29.339150","last_fullsized":"2024-04-17 22:29:29.339150","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":2203,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":"10","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:08.802632","last_change":"2024-04-17 22:29:07.818538","last_active":"2024-04-17 22:29:08.802632","last_peered":"2024-04-17 22:29:08.802632","last_clean":"2024-04-17 22:29:08.802632","last_became_active":"2024-04-17 22:29:07.817164","last_became_peered":"2024-04-17 22:29:07.817164","last_unstale":"2024-04-17 22:29:08.802632","last_undegraded":"2024-04-17 22:29:08.802632","last_fullsized":"2024-04-17 22:29:08.802632","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.35","version":"0'0","reported_seq":"11","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:08.800453","last_change":"2024-04-17 22:29:07.467651","last_active":"2024-04-17 22:29:08.800453","last_peered":"2024-04-17 22:29:08.800453","last_clean":"2024-04-17 22:29:08.800453","last_became_active":"2024-04-17 22:29:07.467327","last_became_peered":"2024-04-17 22:29:07.467327","last_unstale":"2024-04-17 22:29:08.800453","last_undegraded":"2024-04-17 22:29:08.800453","last_fullsized":"2024-04-17 22:29:08.800453","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":"10","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:08.801055","last_change":"2024-04-17 22:29:07.809693","last_active":"2024-04-17 22:29:08.801055","last_peered":"2024-04-17 22:29:08.801055","last_clean":"2024-04-17 22:29:08.801055","last_became_active":"2024-04-17 22:29:07.809415","last_became_peered":"2024-04-17 22:29:07.809415","last_unstale":"2024-04-17 22:29:08.801055","last_undegraded":"2024-04-17 22:29:08.801055","last_fullsized":"2024-04-17 22:29:08.801055","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.36","version":"0'0","reported_seq":"11","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:08.800982","last_change":"2024-04-17 22:29:07.483402","last_active":"2024-04-17 22:29:08.800982","last_peered":"2024-04-17 22:29:08.800982","last_clean":"2024-04-17 22:29:08.800982","last_became_active":"2024-04-17 22:29:07.482914","last_became_peered":"2024-04-17 22:29:07.482914","last_unstale":"2024-04-17 22:29:08.800982","last_undegraded":"2024-04-17 22:29:08.800982","last_fullsized":"2024-04-17 22:29:08.800982","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":"10","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:08.802739","last_change":"2024-04-17 22:29:07.827623","last_active":"2024-04-17 22:29:08.802739","last_peered":"2024-04-17 22:29:08.802739","last_clean":"2024-04-17 22:29:08.802739","last_became_active":"2024-04-17 22:29:07.827337","last_became_peered":"2024-04-17 22:29:07.827337","last_unstale":"2024-04-17 22:29:08.802739","last_undegraded":"2024-04-17 22:29:08.802739","last_fullsized":"2024-04-17 22:29:08.802739","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.37","version":"0'0","reported_seq":"11","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:08.806112","last_change":"2024-04-17 22:29:07.463449","last_active":"2024-04-17 22:29:08.806112","last_peered":"2024-04-17 22:29:08.806112","last_clean":"2024-04-17 22:29:08.806112","last_became_active":"2024-04-17 22:29:07.463110","last_became_peered":"2024-04-17 22:29:07.463110","last_unstale":"2024-04-17 22:29:08.806112","last_undegraded":"2024-04-17 22:29:08.806112","last_fullsized":"2024-04-17 22:29:08.806112","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":"10","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:08.802179","last_change":"2024-04-17 22:29:07.815214","last_active":"2024-04-17 22:29:08.802179","last_peered":"2024-04-17 22:29:08.802179","last_clean":"2024-04-17 22:29:08.802179","last_became_active":"2024-04-17 22:29:07.814938","last_became_peered":"2024-04-17 22:29:07.814938","last_unstale":"2024-04-17 22:29:08.802179","last_undegraded":"2024-04-17 22:29:08.802179","last_fullsized":"2024-04-17 22:29:08.802179","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.38","version":"0'0","reported_seq":"11","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:08.802121","last_change":"2024-04-17 22:29:07.474682","last_active":"2024-04-17 22:29:08.802121","last_peered":"2024-04-17 22:29:08.802121","last_clean":"2024-04-17 22:29:08.802121","last_became_active":"2024-04-17 22:29:07.474305","last_became_peered":"2024-04-17 22:29:07.474305","last_unstale":"2024-04-17 22:29:08.802121","last_undegraded":"2024-04-17 22:29:08.802121","last_fullsized":"2024-04-17 22:29:08.802121","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":"10","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:09.397814","last_change":"2024-04-17 22:29:07.829769","last_active":"2024-04-17 22:29:09.397814","last_peered":"2024-04-17 22:29:09.397814","last_clean":"2024-04-17 22:29:09.397814","last_became_active":"2024-04-17 22:29:07.829484","last_became_peered":"2024-04-17 22:29:07.829484","last_unstale":"2024-04-17 22:29:09.397814","last_undegraded":"2024-04-17 22:29:09.397814","last_fullsized":"2024-04-17 22:29:09.397814","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.39","version":"0'0","reported_seq":"11","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:08.799584","last_change":"2024-04-17 22:29:07.458386","last_active":"2024-04-17 22:29:08.799584","last_peered":"2024-04-17 22:29:08.799584","last_clean":"2024-04-17 22:29:08.799584","last_became_active":"2024-04-17 22:29:07.456695","last_became_peered":"2024-04-17 22:29:07.456695","last_unstale":"2024-04-17 22:29:08.799584","last_undegraded":"2024-04-17 22:29:08.799584","last_fullsized":"2024-04-17 22:29:08.799584","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":"10","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:08.802578","last_change":"2024-04-17 22:29:07.821490","last_active":"2024-04-17 22:29:08.802578","last_peered":"2024-04-17 22:29:08.802578","last_clean":"2024-04-17 22:29:08.802578","last_became_active":"2024-04-17 22:29:07.821103","last_became_peered":"2024-04-17 22:29:07.821103","last_unstale":"2024-04-17 22:29:08.802578","last_undegraded":"2024-04-17 22:29:08.802578","last_fullsized":"2024-04-17 22:29:08.802578","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3a","version":"15'1","reported_seq":"12","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:08.800480","last_change":"2024-04-17 22:29:07.472610","last_active":"2024-04-17 22:29:08.800480","last_peered":"2024-04-17 22:29:08.800480","last_clean":"2024-04-17 22:29:08.800480","last_became_active":"2024-04-17 22:29:07.472297","last_became_peered":"2024-04-17 22:29:07.472297","last_unstale":"2024-04-17 22:29:08.800480","last_undegraded":"2024-04-17 22:29:08.800480","last_fullsized":"2024-04-17 22:29:08.800480","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":"10","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:08.800273","last_change":"2024-04-17 22:29:07.820738","last_active":"2024-04-17 22:29:08.800273","last_peered":"2024-04-17 22:29:08.800273","last_clean":"2024-04-17 22:29:08.800273","last_became_active":"2024-04-17 22:29:07.820354","last_became_peered":"2024-04-17 22:29:07.820354","last_unstale":"2024-04-17 22:29:08.800273","last_undegraded":"2024-04-17 22:29:08.800273","last_fullsized":"2024-04-17 22:29:08.800273","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3b","version":"15'1","reported_seq":"12","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:08.802950","last_change":"2024-04-17 22:29:07.473963","last_active":"2024-04-17 22:29:08.802950","last_peered":"2024-04-17 22:29:08.802950","last_clean":"2024-04-17 22:29:08.802950","last_became_active":"2024-04-17 22:29:07.472018","last_became_peered":"2024-04-17 22:29:07.472018","last_unstale":"2024-04-17 22:29:08.802950","last_undegraded":"2024-04-17 22:29:08.802950","last_fullsized":"2024-04-17 22:29:08.802950","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":"10","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:08.803375","last_change":"2024-04-17 22:29:07.824919","last_active":"2024-04-17 22:29:08.803375","last_peered":"2024-04-17 22:29:08.803375","last_clean":"2024-04-17 22:29:08.803375","last_became_active":"2024-04-17 22:29:07.824600","last_became_peered":"2024-04-17 22:29:07.824600","last_unstale":"2024-04-17 22:29:08.803375","last_undegraded":"2024-04-17 22:29:08.803375","last_fullsized":"2024-04-17 22:29:08.803375","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3c","version":"15'2","reported_seq":"13","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:08.802864","last_change":"2024-04-17 22:29:07.475485","last_active":"2024-04-17 22:29:08.802864","last_peered":"2024-04-17 22:29:08.802864","last_clean":"2024-04-17 22:29:08.802864","last_became_active":"2024-04-17 22:29:07.475182","last_became_peered":"2024-04-17 22:29:07.475182","last_unstale":"2024-04-17 22:29:08.802864","last_undegraded":"2024-04-17 22:29:08.802864","last_fullsized":"2024-04-17 22:29:08.802864","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":"10","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:08.800557","last_change":"2024-04-17 22:29:07.810455","last_active":"2024-04-17 22:29:08.800557","last_peered":"2024-04-17 22:29:08.800557","last_clean":"2024-04-17 22:29:08.800557","last_became_active":"2024-04-17 22:29:07.810143","last_became_peered":"2024-04-17 22:29:07.810143","last_unstale":"2024-04-17 22:29:08.800557","last_undegraded":"2024-04-17 22:29:08.800557","last_fullsized":"2024-04-17 22:29:08.800557","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3d","version":"0'0","reported_seq":"11","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:08.800361","last_change":"2024-04-17 22:29:07.468187","last_active":"2024-04-17 22:29:08.800361","last_peered":"2024-04-17 22:29:08.800361","last_clean":"2024-04-17 22:29:08.800361","last_became_active":"2024-04-17 22:29:07.467849","last_became_peered":"2024-04-17 22:29:07.467849","last_unstale":"2024-04-17 22:29:08.800361","last_undegraded":"2024-04-17 22:29:08.800361","last_fullsized":"2024-04-17 22:29:08.800361","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":"10","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:08.805405","last_change":"2024-04-17 22:29:07.822173","last_active":"2024-04-17 22:29:08.805405","last_peered":"2024-04-17 22:29:08.805405","last_clean":"2024-04-17 22:29:08.805405","last_became_active":"2024-04-17 22:29:07.821875","last_became_peered":"2024-04-17 22:29:07.821875","last_unstale":"2024-04-17 22:29:08.805405","last_undegraded":"2024-04-17 22:29:08.805405","last_fullsized":"2024-04-17 22:29:08.805405","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3e","version":"0'0","reported_seq":"11","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:08.802827","last_change":"2024-04-17 22:29:07.479547","last_active":"2024-04-17 22:29:08.802827","last_peered":"2024-04-17 22:29:08.802827","last_clean":"2024-04-17 22:29:08.802827","last_became_active":"2024-04-17 22:29:07.479004","last_became_peered":"2024-04-17 22:29:07.479004","last_unstale":"2024-04-17 22:29:08.802827","last_undegraded":"2024-04-17 22:29:08.802827","last_fullsized":"2024-04-17 22:29:08.802827","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":"10","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:08.800895","last_change":"2024-04-17 22:29:07.819432","last_active":"2024-04-17 22:29:08.800895","last_peered":"2024-04-17 22:29:08.800895","last_clean":"2024-04-17 22:29:08.800895","last_became_active":"2024-04-17 22:29:07.819001","last_became_peered":"2024-04-17 22:29:07.819001","last_unstale":"2024-04-17 22:29:08.800895","last_undegraded":"2024-04-17 22:29:08.800895","last_fullsized":"2024-04-17 22:29:08.800895","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3f","version":"15'3","reported_seq":"14","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:08.801001","last_change":"2024-04-17 22:29:07.466765","last_active":"2024-04-17 22:29:08.801001","last_peered":"2024-04-17 22:29:08.801001","last_clean":"2024-04-17 22:29:08.801001","last_became_active":"2024-04-17 22:29:07.463177","last_became_peered":"2024-04-17 22:29:07.463177","last_unstale":"2024-04-17 22:29:08.801001","last_undegraded":"2024-04-17 22:29:08.801001","last_fullsized":"2024-04-17 22:29:08.801001","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":544,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":4},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":4},{"poolid":2,"num_pg":64,"stat_sum":{"num_bytes":3597,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":7194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":43,"ondisk_log_size":43,"up":128,"acting":128,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705691,"num_pgs":82,"num_osds":1,"num_per_pool_osds":1,"kb":94371840,"kb_used":1048752,"kb_used_data":168,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323088,"statfs":{"total":96636764160,"available":95562842112,"internally_reserved":1073741824,"allocated":172032,"data_stored":45697,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 17 22:29:57 2024","interfaces":[{"interface":"back","average":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"min":{"1min":0.19900000000000001,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.443,"5min":0.443,"15min":0.443},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.317,"5min":0.317,"15min":0.317},"min":{"1min":0.214,"5min":0.214,"15min":0.214},"max":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"last":0.34599999999999997}]},{"osd":1,"last update":"Wed Apr 17 22:29:57 2024","interfaces":[{"interface":"back","average":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"min":{"1min":0.19600000000000001,"5min":0.19600000000000001,"15min":0.19600000000000001},"max":{"1min":0.44,"5min":0.44,"15min":0.44},"last":0.376},{"interface":"front","average":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"last":0.36899999999999999}]},{"osd":2,"last update":"Wed Apr 17 22:29:57 2024","interfaces":[{"interface":"back","average":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"min":{"1min":0.20699999999999999,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"last":0.40699999999999997}]}]},{"osd":2,"up_from":9,"seq":38654705691,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"kb":94371840,"kb_used":1048740,"kb_used_data":156,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323100,"statfs":{"total":96636764160,"available":95562854400,"internally_reserved":1073741824,"allocated":159744,"data_stored":43370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 17 22:29:58 2024","interfaces":[{"interface":"back","average":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"min":{"1min":0.112,"5min":0.112,"15min":0.112},"max":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"last":0.27400000000000002},{"interface":"front","average":{"1min":0.309,"5min":0.309,"15min":0.309},"min":{"1min":0.155,"5min":0.155,"15min":0.155},"max":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"last":0.34399999999999997}]},{"osd":1,"last update":"Wed Apr 17 22:29:58 2024","interfaces":[{"interface":"back","average":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"min":{"1min":0.17899999999999999,"5min":0.17899999999999999,"15min":0.17899999999999999},"max":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"last":0.26400000000000001},{"interface":"front","average":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"min":{"1min":0.192,"5min":0.192,"15min":0.192},"max":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"last":0.30399999999999999}]},{"osd":3,"last update":"Wed Apr 17 22:29:58 2024","interfaces":[{"interface":"back","average":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"min":{"1min":0.191,"5min":0.191,"15min":0.191},"max":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"last":0.318},{"interface":"front","average":{"1min":0.31,"5min":0.31,"15min":0.31},"min":{"1min":0.214,"5min":0.214,"15min":0.214},"max":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"last":0.28799999999999998}]}]},{"osd":0,"up_from":9,"seq":38654705691,"num_pgs":78,"num_osds":1,"num_per_pool_osds":1,"kb":94371840,"kb_used":1048744,"kb_used_data":160,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323096,"statfs":{"total":96636764160,"available":95562850304,"internally_reserved":1073741824,"allocated":163840,"data_stored":45469,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Apr 17 22:29:55 2024","interfaces":[{"interface":"back","average":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"min":{"1min":0.17299999999999999,"5min":0.17299999999999999,"15min":0.17299999999999999},"max":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"min":{"1min":0.16600000000000001,"5min":0.16600000000000001,"15min":0.16600000000000001},"max":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"last":0.35299999999999998}]},{"osd":2,"last update":"Wed Apr 17 22:29:55 2024","interfaces":[{"interface":"back","average":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"min":{"1min":0.20300000000000001,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"min":{"1min":0.20799999999999999,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"last":0.371}]},{"osd":3,"last update":"Wed Apr 17 22:29:55 2024","interfaces":[{"interface":"back","average":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.505,"5min":0.505,"15min":0.505},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"min":{"1min":0.20300000000000001,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"last":0.38300000000000001}]}]},{"osd":1,"up_from":9,"seq":38654705690,"num_pgs":63,"num_osds":1,"num_per_pool_osds":1,"kb":94371840,"kb_used":1048740,"kb_used_data":156,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323100,"statfs":{"total":96636764160,"available":95562854400,"internally_reserved":1073741824,"allocated":159744,"data_stored":43322,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 17 22:29:58 2024","interfaces":[{"interface":"back","average":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"min":{"1min":0.16200000000000001,"5min":0.16200000000000001,"15min":0.16200000000000001},"max":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"last":0.27400000000000002},{"interface":"front","average":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"min":{"1min":0.191,"5min":0.191,"15min":0.191},"max":{"1min":0.376,"5min":0.376,"15min":0.376},"last":0.20899999999999999}]},{"osd":2,"last update":"Wed Apr 17 22:29:58 2024","interfaces":[{"interface":"back","average":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"min":{"1min":0.20499999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"last":0.216},{"interface":"front","average":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"min":{"1min":0.19800000000000001,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"last":0.25700000000000001}]},{"osd":3,"last update":"Wed Apr 17 22:29:58 2024","interfaces":[{"interface":"back","average":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"min":{"1min":0.218,"5min":0.218,"15min":0.218},"max":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"last":0.23699999999999999},{"interface":"front","average":{"1min":0.317,"5min":0.317,"15min":0.317},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"last":0.245}]}]}]}} 2024-04-17T22:30:56.975 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-17T22:30:57.135 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.147 7fa214067700 1 Processor -- start 2024-04-17T22:30:57.136 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.147 7fa214067700 1 -- start start 2024-04-17T22:30:57.136 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.148 7fa214067700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa20c10e7a0 0x7fa20c10ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:30:57.136 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.148 7fa214067700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fa20c1a92d0 0x7fa20c1ab730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:30:57.136 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.148 7fa214067700 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa20c1067f0 con 0x7fa20c0f0f40 2024-04-17T22:30:57.136 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.148 7fa214067700 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa20c106330 con 0x7fa20c1a92d0 2024-04-17T22:30:57.136 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.148 7fa214067700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa20c0fb480 con 0x7fa20c10e7a0 2024-04-17T22:30:57.136 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.148 7fa211663700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fa20c1a92d0 0x7fa20c1ab730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:30:57.136 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.148 7fa210e62700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa20c10e7a0 0x7fa20c10ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:30:57.136 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.148 7fa211663700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fa20c1a92d0 0x7fa20c1ab730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:40356/0 (socket says 172.21.15.102:40356) 2024-04-17T22:30:57.136 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.148 7fa210e62700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa20c10e7a0 0x7fa20c10ebf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:48772/0 (socket says 172.21.15.102:48772) 2024-04-17T22:30:57.136 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.148 7fa211663700 1 -- 172.21.15.102:0/517090118 learned_addr learned my addr 172.21.15.102:0/517090118 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:30:57.136 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.148 7fa20bfff700 1 --1- 172.21.15.102:0/517090118 >> v1:172.21.15.102:6789/0 conn(0x7fa20c0f0f40 0x7fa20c0f1390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:40934/0 (socket says 172.21.15.102:40934) 2024-04-17T22:30:57.137 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.148 7fa211663700 1 -- 172.21.15.102:0/517090118 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa20c10e7a0 msgr2=0x7fa20c10ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:30:57.137 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.148 7fa211663700 1 --2- 172.21.15.102:0/517090118 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa20c10e7a0 0x7fa20c10ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:30:57.137 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.148 7fa211663700 1 -- 172.21.15.102:0/517090118 >> v1:172.21.15.102:6789/0 conn(0x7fa20c0f0f40 legacy=0x7fa20c0f1390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:30:57.137 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.148 7fa211663700 1 -- 172.21.15.102:0/517090118 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa20c111d00 con 0x7fa20c1a92d0 2024-04-17T22:30:57.137 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.148 7fa20b7fe700 1 -- 172.21.15.102:0/517090118 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2763373667 0 0) 0x7fa200001110 con 0x7fa20c0f0f40 2024-04-17T22:30:57.137 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.148 7fa210e62700 1 --2- 172.21.15.102:0/517090118 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa20c10e7a0 0x7fa20c10ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:30:57.137 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.148 7fa211663700 1 --2- 172.21.15.102:0/517090118 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa20c1a92d0 0x7fa20c1ab730 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 rx=0x7fa20c0fb740 tx=0x7fa1fc009a80).ready entity=mon.0 client_cookie=8ee4cbe5e27c009a server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:30:57.137 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.149 7fa20b7fe700 1 -- 172.21.15.102:0/517090118 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa20c111d00 con 0x7fa20c1a92d0 2024-04-17T22:30:57.137 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.149 7fa20b7fe700 1 -- 172.21.15.102:0/517090118 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1fc00f510 con 0x7fa20c1a92d0 2024-04-17T22:30:57.137 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.149 7fa20b7fe700 1 -- 172.21.15.102:0/517090118 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa1fc00f740 con 0x7fa20c1a92d0 2024-04-17T22:30:57.137 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.149 7fa214067700 1 -- 172.21.15.102:0/517090118 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa20c1a92d0 msgr2=0x7fa20c1ab730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:30:57.137 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.149 7fa214067700 1 --2- 172.21.15.102:0/517090118 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa20c1a92d0 0x7fa20c1ab730 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 rx=0x7fa20c0fb740 tx=0x7fa1fc009a80).stop 2024-04-17T22:30:57.138 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.149 7fa214067700 1 -- 172.21.15.102:0/517090118 shutdown_connections 2024-04-17T22:30:57.138 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.149 7fa214067700 1 --2- 172.21.15.102:0/517090118 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa20c10e7a0 0x7fa20c10ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:30:57.138 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.149 7fa214067700 1 --2- 172.21.15.102:0/517090118 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa20c1a92d0 0x7fa20c1ab730 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:30:57.138 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.149 7fa214067700 1 -- 172.21.15.102:0/517090118 >> 172.21.15.102:0/517090118 conn(0x7fa20c114780 msgr2=0x7fa20c114bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:30:57.138 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.149 7fa214067700 1 -- 172.21.15.102:0/517090118 shutdown_connections 2024-04-17T22:30:57.138 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.149 7fa214067700 1 -- 172.21.15.102:0/517090118 wait complete. 2024-04-17T22:30:57.158 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.172 7fa214067700 1 Processor -- start 2024-04-17T22:30:57.158 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.172 7fa214067700 1 -- start start 2024-04-17T22:30:57.161 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.172 7fa214067700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa20c0d4ee0 0x7fa20c1ad9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:30:57.161 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.172 7fa214067700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa20c1adfd0 0x7fa20c1b3ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:30:57.161 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.172 7fa214067700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fa20c1b44f0 0x7fa20c1b4940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:30:57.161 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.172 7fa214067700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa20c1ae420 con 0x7fa20c0d4ee0 2024-04-17T22:30:57.161 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.172 7fa214067700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa20c0d4b50 con 0x7fa20c1adfd0 2024-04-17T22:30:57.161 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.172 7fa214067700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa20c0d4500 con 0x7fa20c1b44f0 2024-04-17T22:30:57.162 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.172 7fa20bfff700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa20c1adfd0 0x7fa20c1b3ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:30:57.162 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.173 7fa210e62700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa20c0d4ee0 0x7fa20c1ad9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:30:57.162 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.173 7fa211663700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fa20c1b44f0 0x7fa20c1b4940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:30:57.162 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.173 7fa20bfff700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa20c1adfd0 0x7fa20c1b3ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:48778/0 (socket says 172.21.15.102:48778) 2024-04-17T22:30:57.162 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.173 7fa20bfff700 1 -- 172.21.15.102:0/3038281029 learned_addr learned my addr 172.21.15.102:0/3038281029 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:30:57.162 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.173 7fa210e62700 1 -- 172.21.15.102:0/3038281029 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fa20c1b44f0 msgr2=0x7fa20c1b4940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:30:57.162 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.173 7fa210e62700 1 --2- 172.21.15.102:0/3038281029 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fa20c1b44f0 0x7fa20c1b4940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:30:57.162 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.173 7fa210e62700 1 -- 172.21.15.102:0/3038281029 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa20c1adfd0 msgr2=0x7fa20c1b3ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:30:57.162 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.173 7fa210e62700 1 --2- 172.21.15.102:0/3038281029 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa20c1adfd0 0x7fa20c1b3ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:30:57.162 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.173 7fa210e62700 1 -- 172.21.15.102:0/3038281029 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa20c1bb3a0 con 0x7fa20c0d4ee0 2024-04-17T22:30:57.162 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.173 7fa211663700 1 --2- 172.21.15.102:0/3038281029 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fa20c1b44f0 0x7fa20c1b4940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:30:57.163 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.173 7fa20bfff700 1 --2- 172.21.15.102:0/3038281029 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa20c1adfd0 0x7fa20c1b3ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:30:57.163 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.173 7fa210e62700 1 --2- 172.21.15.102:0/3038281029 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa20c0d4ee0 0x7fa20c1ad9a0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 rx=0x7fa20c0c7050 tx=0x7fa20400ee80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:30:57.163 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.173 7fa209ffb700 1 -- 172.21.15.102:0/3038281029 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa20c1bb3a0 con 0x7fa20c0d4ee0 2024-04-17T22:30:57.163 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.173 7fa209ffb700 1 -- 172.21.15.102:0/3038281029 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa20400c950 con 0x7fa20c0d4ee0 2024-04-17T22:30:57.163 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.173 7fa214067700 1 -- 172.21.15.102:0/3038281029 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa20c1bb5a0 con 0x7fa20c0d4ee0 2024-04-17T22:30:57.163 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.173 7fa209ffb700 1 -- 172.21.15.102:0/3038281029 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa20400f040 con 0x7fa20c0d4ee0 2024-04-17T22:30:57.163 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.173 7fa214067700 1 -- 172.21.15.102:0/3038281029 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa20c1bb980 con 0x7fa20c0d4ee0 2024-04-17T22:30:57.163 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.174 7fa209ffb700 1 -- 172.21.15.102:0/3038281029 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7fa204022060 con 0x7fa20c0d4ee0 2024-04-17T22:30:57.163 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.174 7fa209ffb700 1 -- 172.21.15.102:0/3038281029 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3194+0+0 (secure 0 0 0) 0x7fa204010040 con 0x7fa20c0d4ee0 2024-04-17T22:30:57.163 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.174 7fa214067700 1 -- 172.21.15.102:0/3038281029 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa20c0f0820 con 0x7fa20c0d4ee0 2024-04-17T22:30:57.163 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.177 7fa209ffb700 1 -- 172.21.15.102:0/3038281029 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+110455 (secure 0 0 0) 0x7fa20400c740 con 0x7fa20c0d4ee0 2024-04-17T22:30:57.266 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.280 7fa214067700 1 -- 172.21.15.102:0/3038281029 --> v1:172.21.15.102:6816/42360 -- command(tid 0: {"prefix": "pg dump", "target": ["mgr", ""], "format": "json"}) v1 -- 0x7fa20c1ae600 con 0x7fa1ec0145a0 2024-04-17T22:30:57.273 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.287 7fa209ffb700 1 -- 172.21.15.102:0/3038281029 <== mgr.4104 v1:172.21.15.102:6816/42360 1 ==== command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311258 (unknown 2965378022 0 1196534818) 0x7fa20c1ae600 con 0x7fa1ec0145a0 2024-04-17T22:30:57.277 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-17T22:30:57.279 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.290 7fa214067700 1 -- 172.21.15.102:0/3038281029 >> v1:172.21.15.102:6816/42360 conn(0x7fa1ec0145a0 legacy=0x7fa1ec016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:30:57.279 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.290 7fa214067700 1 -- 172.21.15.102:0/3038281029 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa20c0d4ee0 msgr2=0x7fa20c1ad9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:30:57.280 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.290 7fa214067700 1 --2- 172.21.15.102:0/3038281029 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa20c0d4ee0 0x7fa20c1ad9a0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 rx=0x7fa20c0c7050 tx=0x7fa20400ee80).stop 2024-04-17T22:30:57.280 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.290 7fa214067700 1 -- 172.21.15.102:0/3038281029 shutdown_connections 2024-04-17T22:30:57.280 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.290 7fa214067700 1 --2- 172.21.15.102:0/3038281029 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fa20c1b44f0 0x7fa20c1b4940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:30:57.280 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.290 7fa214067700 1 --2- 172.21.15.102:0/3038281029 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa20c0d4ee0 0x7fa20c1ad9a0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:30:57.280 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.290 7fa214067700 1 --2- 172.21.15.102:0/3038281029 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa20c1adfd0 0x7fa20c1b3ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:30:57.280 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.290 7fa214067700 1 -- 172.21.15.102:0/3038281029 >> 172.21.15.102:0/3038281029 conn(0x7fa20c0c6ae0 msgr2=0x7fa20c1a8170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:30:57.280 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.290 7fa214067700 1 -- 172.21.15.102:0/3038281029 shutdown_connections 2024-04-17T22:30:57.280 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.290 7fa214067700 1 -- 172.21.15.102:0/3038281029 wait complete. 2024-04-17T22:30:57.280 INFO:teuthology.orchestra.run.smithi102.stderr:dumped all 2024-04-17T22:30:57.287 INFO:teuthology.orchestra.run.smithi102.stdout:{"pg_ready":true,"pg_map":{"version":75,"stamp":"2024-04-17 22:30:57.213281","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":3597,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":43,"ondisk_log_size":43,"up":272,"acting":272,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":272,"num_osds":4,"num_per_pool_osds":4,"kb":377487360,"kb_used":4194976,"kb_used_data":640,"kb_used_omap":0,"kb_used_meta":4194304,"kb_avail":373292384,"statfs":{"total":386547056640,"available":382251401216,"internally_reserved":4294967296,"allocated":655360,"data_stored":177858,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4294967296},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002787"},"pg_stats":[{"pgid":"3.1a","version":"0'0","reported_seq":"10","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:08.799742","last_change":"2024-04-17 22:29:07.814049","last_active":"2024-04-17 22:29:08.799742","last_peered":"2024-04-17 22:29:08.799742","last_clean":"2024-04-17 22:29:08.799742","last_became_active":"2024-04-17 22:29:07.811049","last_became_peered":"2024-04-17 22:29:07.811049","last_unstale":"2024-04-17 22:29:08.799742","last_undegraded":"2024-04-17 22:29:08.799742","last_fullsized":"2024-04-17 22:29:08.799742","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":"11","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:09.398176","last_change":"2024-04-17 22:29:07.477053","last_active":"2024-04-17 22:29:09.398176","last_peered":"2024-04-17 22:29:09.398176","last_clean":"2024-04-17 22:29:09.398176","last_became_active":"2024-04-17 22:29:07.476568","last_became_peered":"2024-04-17 22:29:07.476568","last_unstale":"2024-04-17 22:29:09.398176","last_undegraded":"2024-04-17 22:29:09.398176","last_fullsized":"2024-04-17 22:29:09.398176","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":"10","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:08.802676","last_change":"2024-04-17 22:29:07.818787","last_active":"2024-04-17 22:29:08.802676","last_peered":"2024-04-17 22:29:08.802676","last_clean":"2024-04-17 22:29:08.802676","last_became_active":"2024-04-17 22:29:07.818347","last_became_peered":"2024-04-17 22:29:07.818347","last_unstale":"2024-04-17 22:29:08.802676","last_undegraded":"2024-04-17 22:29:08.802676","last_fullsized":"2024-04-17 22:29:08.802676","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":"11","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:09.398049","last_change":"2024-04-17 22:29:07.477933","last_active":"2024-04-17 22:29:09.398049","last_peered":"2024-04-17 22:29:09.398049","last_clean":"2024-04-17 22:29:09.398049","last_became_active":"2024-04-17 22:29:07.472128","last_became_peered":"2024-04-17 22:29:07.472128","last_unstale":"2024-04-17 22:29:09.398049","last_undegraded":"2024-04-17 22:29:09.398049","last_fullsized":"2024-04-17 22:29:09.398049","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":"10","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:09.397770","last_change":"2024-04-17 22:29:07.833107","last_active":"2024-04-17 22:29:09.397770","last_peered":"2024-04-17 22:29:09.397770","last_clean":"2024-04-17 22:29:09.397770","last_became_active":"2024-04-17 22:29:07.832822","last_became_peered":"2024-04-17 22:29:07.832822","last_unstale":"2024-04-17 22:29:09.397770","last_undegraded":"2024-04-17 22:29:09.397770","last_fullsized":"2024-04-17 22:29:09.397770","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":"11","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:09.397993","last_change":"2024-04-17 22:29:07.471612","last_active":"2024-04-17 22:29:09.397993","last_peered":"2024-04-17 22:29:09.397993","last_clean":"2024-04-17 22:29:09.397993","last_became_active":"2024-04-17 22:29:07.471322","last_became_peered":"2024-04-17 22:29:07.471322","last_unstale":"2024-04-17 22:29:09.397993","last_undegraded":"2024-04-17 22:29:09.397993","last_fullsized":"2024-04-17 22:29:09.397993","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":"10","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:08.799757","last_change":"2024-04-17 22:29:07.810081","last_active":"2024-04-17 22:29:08.799757","last_peered":"2024-04-17 22:29:08.799757","last_clean":"2024-04-17 22:29:08.799757","last_became_active":"2024-04-17 22:29:07.809858","last_became_peered":"2024-04-17 22:29:07.809858","last_unstale":"2024-04-17 22:29:08.799757","last_undegraded":"2024-04-17 22:29:08.799757","last_fullsized":"2024-04-17 22:29:08.799757","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":"11","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:09.397518","last_change":"2024-04-17 22:29:07.474346","last_active":"2024-04-17 22:29:09.397518","last_peered":"2024-04-17 22:29:09.397518","last_clean":"2024-04-17 22:29:09.397518","last_became_active":"2024-04-17 22:29:07.473783","last_became_peered":"2024-04-17 22:29:07.473783","last_unstale":"2024-04-17 22:29:09.397518","last_undegraded":"2024-04-17 22:29:09.397518","last_fullsized":"2024-04-17 22:29:09.397518","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":"10","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:08.801136","last_change":"2024-04-17 22:29:07.811047","last_active":"2024-04-17 22:29:08.801136","last_peered":"2024-04-17 22:29:08.801136","last_clean":"2024-04-17 22:29:08.801136","last_became_active":"2024-04-17 22:29:07.810747","last_became_peered":"2024-04-17 22:29:07.810747","last_unstale":"2024-04-17 22:29:08.801136","last_undegraded":"2024-04-17 22:29:08.801136","last_fullsized":"2024-04-17 22:29:08.801136","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":"11","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:08.801135","last_change":"2024-04-17 22:29:07.460228","last_active":"2024-04-17 22:29:08.801135","last_peered":"2024-04-17 22:29:08.801135","last_clean":"2024-04-17 22:29:08.801135","last_became_active":"2024-04-17 22:29:07.459910","last_became_peered":"2024-04-17 22:29:07.459910","last_unstale":"2024-04-17 22:29:08.801135","last_undegraded":"2024-04-17 22:29:08.801135","last_fullsized":"2024-04-17 22:29:08.801135","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":"10","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:08.805762","last_change":"2024-04-17 22:29:07.827143","last_active":"2024-04-17 22:29:08.805762","last_peered":"2024-04-17 22:29:08.805762","last_clean":"2024-04-17 22:29:08.805762","last_became_active":"2024-04-17 22:29:07.826842","last_became_peered":"2024-04-17 22:29:07.826842","last_unstale":"2024-04-17 22:29:08.805762","last_undegraded":"2024-04-17 22:29:08.805762","last_fullsized":"2024-04-17 22:29:08.805762","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":"11","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:08.802625","last_change":"2024-04-17 22:29:07.477248","last_active":"2024-04-17 22:29:08.802625","last_peered":"2024-04-17 22:29:08.802625","last_clean":"2024-04-17 22:29:08.802625","last_became_active":"2024-04-17 22:29:07.476944","last_became_peered":"2024-04-17 22:29:07.476944","last_unstale":"2024-04-17 22:29:08.802625","last_undegraded":"2024-04-17 22:29:08.802625","last_fullsized":"2024-04-17 22:29:08.802625","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":"10","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:09.398192","last_change":"2024-04-17 22:29:07.822961","last_active":"2024-04-17 22:29:09.398192","last_peered":"2024-04-17 22:29:09.398192","last_clean":"2024-04-17 22:29:09.398192","last_became_active":"2024-04-17 22:29:07.822592","last_became_peered":"2024-04-17 22:29:07.822592","last_unstale":"2024-04-17 22:29:09.398192","last_undegraded":"2024-04-17 22:29:09.398192","last_fullsized":"2024-04-17 22:29:09.398192","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.15","version":"15'3","reported_seq":"14","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:08.800188","last_change":"2024-04-17 22:29:07.485166","last_active":"2024-04-17 22:29:08.800188","last_peered":"2024-04-17 22:29:08.800188","last_clean":"2024-04-17 22:29:08.800188","last_became_active":"2024-04-17 22:29:07.484870","last_became_peered":"2024-04-17 22:29:07.484870","last_unstale":"2024-04-17 22:29:08.800188","last_undegraded":"2024-04-17 22:29:08.800188","last_fullsized":"2024-04-17 22:29:08.800188","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":"10","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:08.803451","last_change":"2024-04-17 22:29:07.814509","last_active":"2024-04-17 22:29:08.803451","last_peered":"2024-04-17 22:29:08.803451","last_clean":"2024-04-17 22:29:08.803451","last_became_active":"2024-04-17 22:29:07.814119","last_became_peered":"2024-04-17 22:29:07.814119","last_unstale":"2024-04-17 22:29:08.803451","last_undegraded":"2024-04-17 22:29:08.803451","last_fullsized":"2024-04-17 22:29:08.803451","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.14","version":"17'3","reported_seq":"14","reported_epoch":"17","state":"active+clean","last_fresh":"2024-04-17 22:29:23.262235","last_change":"2024-04-17 22:29:07.473309","last_active":"2024-04-17 22:29:23.262235","last_peered":"2024-04-17 22:29:23.262235","last_clean":"2024-04-17 22:29:23.262235","last_became_active":"2024-04-17 22:29:07.472976","last_became_peered":"2024-04-17 22:29:07.472976","last_unstale":"2024-04-17 22:29:23.262235","last_undegraded":"2024-04-17 22:29:23.262235","last_fullsized":"2024-04-17 22:29:23.262235","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":634,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":"10","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:08.802306","last_change":"2024-04-17 22:29:07.827915","last_active":"2024-04-17 22:29:08.802306","last_peered":"2024-04-17 22:29:08.802306","last_clean":"2024-04-17 22:29:08.802306","last_became_active":"2024-04-17 22:29:07.827670","last_became_peered":"2024-04-17 22:29:07.827670","last_unstale":"2024-04-17 22:29:08.802306","last_undegraded":"2024-04-17 22:29:08.802306","last_fullsized":"2024-04-17 22:29:08.802306","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"15'1","reported_seq":"12","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:08.802767","last_change":"2024-04-17 22:29:07.471458","last_active":"2024-04-17 22:29:08.802767","last_peered":"2024-04-17 22:29:08.802767","last_clean":"2024-04-17 22:29:08.802767","last_became_active":"2024-04-17 22:29:07.471085","last_became_peered":"2024-04-17 22:29:07.471085","last_unstale":"2024-04-17 22:29:08.802767","last_undegraded":"2024-04-17 22:29:08.802767","last_fullsized":"2024-04-17 22:29:08.802767","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":"10","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:09.398266","last_change":"2024-04-17 22:29:07.829288","last_active":"2024-04-17 22:29:09.398266","last_peered":"2024-04-17 22:29:09.398266","last_clean":"2024-04-17 22:29:09.398266","last_became_active":"2024-04-17 22:29:07.828994","last_became_peered":"2024-04-17 22:29:07.828994","last_unstale":"2024-04-17 22:29:09.398266","last_undegraded":"2024-04-17 22:29:09.398266","last_fullsized":"2024-04-17 22:29:09.398266","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":"11","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:09.397936","last_change":"2024-04-17 22:29:07.480194","last_active":"2024-04-17 22:29:09.397936","last_peered":"2024-04-17 22:29:09.397936","last_clean":"2024-04-17 22:29:09.397936","last_became_active":"2024-04-17 22:29:07.479728","last_became_peered":"2024-04-17 22:29:07.479728","last_unstale":"2024-04-17 22:29:09.397936","last_undegraded":"2024-04-17 22:29:09.397936","last_fullsized":"2024-04-17 22:29:09.397936","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":"10","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:08.799216","last_change":"2024-04-17 22:29:07.815790","last_active":"2024-04-17 22:29:08.799216","last_peered":"2024-04-17 22:29:08.799216","last_clean":"2024-04-17 22:29:08.799216","last_became_active":"2024-04-17 22:29:07.815632","last_became_peered":"2024-04-17 22:29:07.815632","last_unstale":"2024-04-17 22:29:08.799216","last_undegraded":"2024-04-17 22:29:08.799216","last_fullsized":"2024-04-17 22:29:08.799216","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":"11","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:08.802288","last_change":"2024-04-17 22:29:07.475830","last_active":"2024-04-17 22:29:08.802288","last_peered":"2024-04-17 22:29:08.802288","last_clean":"2024-04-17 22:29:08.802288","last_became_active":"2024-04-17 22:29:07.474411","last_became_peered":"2024-04-17 22:29:07.474411","last_unstale":"2024-04-17 22:29:08.802288","last_undegraded":"2024-04-17 22:29:08.802288","last_fullsized":"2024-04-17 22:29:08.802288","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":"10","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:09.397680","last_change":"2024-04-17 22:29:07.826101","last_active":"2024-04-17 22:29:09.397680","last_peered":"2024-04-17 22:29:09.397680","last_clean":"2024-04-17 22:29:09.397680","last_became_active":"2024-04-17 22:29:07.822917","last_became_peered":"2024-04-17 22:29:07.822917","last_unstale":"2024-04-17 22:29:09.397680","last_undegraded":"2024-04-17 22:29:09.397680","last_fullsized":"2024-04-17 22:29:09.397680","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.10","version":"15'4","reported_seq":"15","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:08.800588","last_change":"2024-04-17 22:29:07.457017","last_active":"2024-04-17 22:29:08.800588","last_peered":"2024-04-17 22:29:08.800588","last_clean":"2024-04-17 22:29:08.800588","last_became_active":"2024-04-17 22:29:07.456705","last_became_peered":"2024-04-17 22:29:07.456705","last_unstale":"2024-04-17 22:29:08.800588","last_undegraded":"2024-04-17 22:29:08.800588","last_fullsized":"2024-04-17 22:29:08.800588","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":"10","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:08.800899","last_change":"2024-04-17 22:29:07.820464","last_active":"2024-04-17 22:29:08.800899","last_peered":"2024-04-17 22:29:08.800899","last_clean":"2024-04-17 22:29:08.800899","last_became_active":"2024-04-17 22:29:07.820151","last_became_peered":"2024-04-17 22:29:07.820151","last_unstale":"2024-04-17 22:29:08.800899","last_undegraded":"2024-04-17 22:29:08.800899","last_fullsized":"2024-04-17 22:29:08.800899","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.f","version":"15'1","reported_seq":"12","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:08.806028","last_change":"2024-04-17 22:29:07.478464","last_active":"2024-04-17 22:29:08.806028","last_peered":"2024-04-17 22:29:08.806028","last_clean":"2024-04-17 22:29:08.806028","last_became_active":"2024-04-17 22:29:07.478144","last_became_peered":"2024-04-17 22:29:07.478144","last_unstale":"2024-04-17 22:29:08.806028","last_undegraded":"2024-04-17 22:29:08.806028","last_fullsized":"2024-04-17 22:29:08.806028","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":"10","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:08.805853","last_change":"2024-04-17 22:29:07.815171","last_active":"2024-04-17 22:29:08.805853","last_peered":"2024-04-17 22:29:08.805853","last_clean":"2024-04-17 22:29:08.805853","last_became_active":"2024-04-17 22:29:07.814830","last_became_peered":"2024-04-17 22:29:07.814830","last_unstale":"2024-04-17 22:29:08.805853","last_undegraded":"2024-04-17 22:29:08.805853","last_fullsized":"2024-04-17 22:29:08.805853","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"15'2","reported_seq":"13","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:08.801038","last_change":"2024-04-17 22:29:07.481865","last_active":"2024-04-17 22:29:08.801038","last_peered":"2024-04-17 22:29:08.801038","last_clean":"2024-04-17 22:29:08.801038","last_became_active":"2024-04-17 22:29:07.481506","last_became_peered":"2024-04-17 22:29:07.481506","last_unstale":"2024-04-17 22:29:08.801038","last_undegraded":"2024-04-17 22:29:08.801038","last_fullsized":"2024-04-17 22:29:08.801038","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":"10","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:08.802658","last_change":"2024-04-17 22:29:07.818238","last_active":"2024-04-17 22:29:08.802658","last_peered":"2024-04-17 22:29:08.802658","last_clean":"2024-04-17 22:29:08.802658","last_became_active":"2024-04-17 22:29:07.818060","last_became_peered":"2024-04-17 22:29:07.818060","last_unstale":"2024-04-17 22:29:08.802658","last_undegraded":"2024-04-17 22:29:08.802658","last_fullsized":"2024-04-17 22:29:08.802658","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":"11","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:08.800285","last_change":"2024-04-17 22:29:07.484999","last_active":"2024-04-17 22:29:08.800285","last_peered":"2024-04-17 22:29:08.800285","last_clean":"2024-04-17 22:29:08.800285","last_became_active":"2024-04-17 22:29:07.484668","last_became_peered":"2024-04-17 22:29:07.484668","last_unstale":"2024-04-17 22:29:08.800285","last_undegraded":"2024-04-17 22:29:08.800285","last_fullsized":"2024-04-17 22:29:08.800285","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":"10","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:09.398470","last_change":"2024-04-17 22:29:07.826426","last_active":"2024-04-17 22:29:09.398470","last_peered":"2024-04-17 22:29:09.398470","last_clean":"2024-04-17 22:29:09.398470","last_became_active":"2024-04-17 22:29:07.826109","last_became_peered":"2024-04-17 22:29:07.826109","last_unstale":"2024-04-17 22:29:09.398470","last_undegraded":"2024-04-17 22:29:09.398470","last_fullsized":"2024-04-17 22:29:09.398470","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":"11","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:08.800646","last_change":"2024-04-17 22:29:07.462218","last_active":"2024-04-17 22:29:08.800646","last_peered":"2024-04-17 22:29:08.800646","last_clean":"2024-04-17 22:29:08.800646","last_became_active":"2024-04-17 22:29:07.461842","last_became_peered":"2024-04-17 22:29:07.461842","last_unstale":"2024-04-17 22:29:08.800646","last_undegraded":"2024-04-17 22:29:08.800646","last_fullsized":"2024-04-17 22:29:08.800646","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":"10","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:09.397712","last_change":"2024-04-17 22:29:07.817144","last_active":"2024-04-17 22:29:09.397712","last_peered":"2024-04-17 22:29:09.397712","last_clean":"2024-04-17 22:29:09.397712","last_became_active":"2024-04-17 22:29:07.816859","last_became_peered":"2024-04-17 22:29:07.816859","last_unstale":"2024-04-17 22:29:09.397712","last_undegraded":"2024-04-17 22:29:09.397712","last_fullsized":"2024-04-17 22:29:09.397712","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"15'1","reported_seq":"12","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:09.398046","last_change":"2024-04-17 22:29:07.479816","last_active":"2024-04-17 22:29:09.398046","last_peered":"2024-04-17 22:29:09.398046","last_clean":"2024-04-17 22:29:09.398046","last_became_active":"2024-04-17 22:29:07.479251","last_became_peered":"2024-04-17 22:29:07.479251","last_unstale":"2024-04-17 22:29:09.398046","last_undegraded":"2024-04-17 22:29:09.398046","last_fullsized":"2024-04-17 22:29:09.398046","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":"10","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:09.398161","last_change":"2024-04-17 22:29:07.829215","last_active":"2024-04-17 22:29:09.398161","last_peered":"2024-04-17 22:29:09.398161","last_clean":"2024-04-17 22:29:09.398161","last_became_active":"2024-04-17 22:29:07.828803","last_became_peered":"2024-04-17 22:29:07.828803","last_unstale":"2024-04-17 22:29:09.398161","last_undegraded":"2024-04-17 22:29:09.398161","last_fullsized":"2024-04-17 22:29:09.398161","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":"11","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:08.799648","last_change":"2024-04-17 22:29:07.473128","last_active":"2024-04-17 22:29:08.799648","last_peered":"2024-04-17 22:29:08.799648","last_clean":"2024-04-17 22:29:08.799648","last_became_active":"2024-04-17 22:29:07.472806","last_became_peered":"2024-04-17 22:29:07.472806","last_unstale":"2024-04-17 22:29:08.799648","last_undegraded":"2024-04-17 22:29:08.799648","last_fullsized":"2024-04-17 22:29:08.799648","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":"10","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:09.397925","last_change":"2024-04-17 22:29:07.817630","last_active":"2024-04-17 22:29:09.397925","last_peered":"2024-04-17 22:29:09.397925","last_clean":"2024-04-17 22:29:09.397925","last_became_active":"2024-04-17 22:29:07.817333","last_became_peered":"2024-04-17 22:29:07.817333","last_unstale":"2024-04-17 22:29:09.397925","last_undegraded":"2024-04-17 22:29:09.397925","last_fullsized":"2024-04-17 22:29:09.397925","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":"11","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:08.799500","last_change":"2024-04-17 22:29:07.465053","last_active":"2024-04-17 22:29:08.799500","last_peered":"2024-04-17 22:29:08.799500","last_clean":"2024-04-17 22:29:08.799500","last_became_active":"2024-04-17 22:29:07.464766","last_became_peered":"2024-04-17 22:29:07.464766","last_unstale":"2024-04-17 22:29:08.799500","last_undegraded":"2024-04-17 22:29:08.799500","last_fullsized":"2024-04-17 22:29:08.799500","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":"10","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:09.398181","last_change":"2024-04-17 22:29:07.823514","last_active":"2024-04-17 22:29:09.398181","last_peered":"2024-04-17 22:29:09.398181","last_clean":"2024-04-17 22:29:09.398181","last_became_active":"2024-04-17 22:29:07.823007","last_became_peered":"2024-04-17 22:29:07.823007","last_unstale":"2024-04-17 22:29:09.398181","last_undegraded":"2024-04-17 22:29:09.398181","last_fullsized":"2024-04-17 22:29:09.398181","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":"11","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:08.802037","last_change":"2024-04-17 22:29:07.461568","last_active":"2024-04-17 22:29:08.802037","last_peered":"2024-04-17 22:29:08.802037","last_clean":"2024-04-17 22:29:08.802037","last_became_active":"2024-04-17 22:29:07.461249","last_became_peered":"2024-04-17 22:29:07.461249","last_unstale":"2024-04-17 22:29:08.802037","last_undegraded":"2024-04-17 22:29:08.802037","last_fullsized":"2024-04-17 22:29:08.802037","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":"10","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:09.398949","last_change":"2024-04-17 22:29:07.826342","last_active":"2024-04-17 22:29:09.398949","last_peered":"2024-04-17 22:29:09.398949","last_clean":"2024-04-17 22:29:09.398949","last_became_active":"2024-04-17 22:29:07.826029","last_became_peered":"2024-04-17 22:29:07.826029","last_unstale":"2024-04-17 22:29:09.398949","last_undegraded":"2024-04-17 22:29:09.398949","last_fullsized":"2024-04-17 22:29:09.398949","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":"14","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:08.801226","last_change":"2024-04-17 22:28:59.253977","last_active":"2024-04-17 22:29:08.801226","last_peered":"2024-04-17 22:29:08.801226","last_clean":"2024-04-17 22:29:08.801226","last_became_active":"2024-04-17 22:28:59.253641","last_became_peered":"2024-04-17 22:28:59.253641","last_unstale":"2024-04-17 22:29:08.801226","last_undegraded":"2024-04-17 22:29:08.801226","last_fullsized":"2024-04-17 22:29:08.801226","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:28:58.239187","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:28:58.239187","last_clean_scrub_stamp":"2024-04-17 22:28:58.239187","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":"11","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:08.801068","last_change":"2024-04-17 22:29:07.466907","last_active":"2024-04-17 22:29:08.801068","last_peered":"2024-04-17 22:29:08.801068","last_clean":"2024-04-17 22:29:08.801068","last_became_active":"2024-04-17 22:29:07.466583","last_became_peered":"2024-04-17 22:29:07.466583","last_unstale":"2024-04-17 22:29:08.801068","last_undegraded":"2024-04-17 22:29:08.801068","last_fullsized":"2024-04-17 22:29:08.801068","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":"10","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:08.802634","last_change":"2024-04-17 22:29:07.824875","last_active":"2024-04-17 22:29:08.802634","last_peered":"2024-04-17 22:29:08.802634","last_clean":"2024-04-17 22:29:08.802634","last_became_active":"2024-04-17 22:29:07.824377","last_became_peered":"2024-04-17 22:29:07.824377","last_unstale":"2024-04-17 22:29:08.802634","last_undegraded":"2024-04-17 22:29:08.802634","last_fullsized":"2024-04-17 22:29:08.802634","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":"14","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:09.398292","last_change":"2024-04-17 22:28:59.255733","last_active":"2024-04-17 22:29:09.398292","last_peered":"2024-04-17 22:29:09.398292","last_clean":"2024-04-17 22:29:09.398292","last_became_active":"2024-04-17 22:28:59.255411","last_became_peered":"2024-04-17 22:28:59.255411","last_unstale":"2024-04-17 22:29:09.398292","last_undegraded":"2024-04-17 22:29:09.398292","last_fullsized":"2024-04-17 22:29:09.398292","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:28:58.239187","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:28:58.239187","last_clean_scrub_stamp":"2024-04-17 22:28:58.239187","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"15'2","reported_seq":"13","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:09.398794","last_change":"2024-04-17 22:29:07.478196","last_active":"2024-04-17 22:29:09.398794","last_peered":"2024-04-17 22:29:09.398794","last_clean":"2024-04-17 22:29:09.398794","last_became_active":"2024-04-17 22:29:07.477887","last_became_peered":"2024-04-17 22:29:07.477887","last_unstale":"2024-04-17 22:29:09.398794","last_undegraded":"2024-04-17 22:29:09.398794","last_fullsized":"2024-04-17 22:29:09.398794","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":"14","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:08.799327","last_change":"2024-04-17 22:28:59.252728","last_active":"2024-04-17 22:29:08.799327","last_peered":"2024-04-17 22:29:08.799327","last_clean":"2024-04-17 22:29:08.799327","last_became_active":"2024-04-17 22:28:59.252345","last_became_peered":"2024-04-17 22:28:59.252345","last_unstale":"2024-04-17 22:29:08.799327","last_undegraded":"2024-04-17 22:29:08.799327","last_fullsized":"2024-04-17 22:29:08.799327","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:28:58.239187","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:28:58.239187","last_clean_scrub_stamp":"2024-04-17 22:28:58.239187","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":"11","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:09.397593","last_change":"2024-04-17 22:29:07.474520","last_active":"2024-04-17 22:29:09.397593","last_peered":"2024-04-17 22:29:09.397593","last_clean":"2024-04-17 22:29:09.397593","last_became_active":"2024-04-17 22:29:07.474029","last_became_peered":"2024-04-17 22:29:07.474029","last_unstale":"2024-04-17 22:29:09.397593","last_undegraded":"2024-04-17 22:29:09.397593","last_fullsized":"2024-04-17 22:29:09.397593","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":"10","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:08.802181","last_change":"2024-04-17 22:29:07.825091","last_active":"2024-04-17 22:29:08.802181","last_peered":"2024-04-17 22:29:08.802181","last_clean":"2024-04-17 22:29:08.802181","last_became_active":"2024-04-17 22:29:07.824832","last_became_peered":"2024-04-17 22:29:07.824832","last_unstale":"2024-04-17 22:29:08.802181","last_undegraded":"2024-04-17 22:29:08.802181","last_fullsized":"2024-04-17 22:29:08.802181","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":"14","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:08.802193","last_change":"2024-04-17 22:28:59.253392","last_active":"2024-04-17 22:29:08.802193","last_peered":"2024-04-17 22:29:08.802193","last_clean":"2024-04-17 22:29:08.802193","last_became_active":"2024-04-17 22:28:59.253049","last_became_peered":"2024-04-17 22:28:59.253049","last_unstale":"2024-04-17 22:29:08.802193","last_undegraded":"2024-04-17 22:29:08.802193","last_fullsized":"2024-04-17 22:29:08.802193","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:28:58.239187","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:28:58.239187","last_clean_scrub_stamp":"2024-04-17 22:28:58.239187","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":"11","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:08.800158","last_change":"2024-04-17 22:29:07.471913","last_active":"2024-04-17 22:29:08.800158","last_peered":"2024-04-17 22:29:08.800158","last_clean":"2024-04-17 22:29:08.800158","last_became_active":"2024-04-17 22:29:07.471686","last_became_peered":"2024-04-17 22:29:07.471686","last_unstale":"2024-04-17 22:29:08.800158","last_undegraded":"2024-04-17 22:29:08.800158","last_fullsized":"2024-04-17 22:29:08.800158","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":"10","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:08.799248","last_change":"2024-04-17 22:29:07.812999","last_active":"2024-04-17 22:29:08.799248","last_peered":"2024-04-17 22:29:08.799248","last_clean":"2024-04-17 22:29:08.799248","last_became_active":"2024-04-17 22:29:07.812701","last_became_peered":"2024-04-17 22:29:07.812701","last_unstale":"2024-04-17 22:29:08.799248","last_undegraded":"2024-04-17 22:29:08.799248","last_fullsized":"2024-04-17 22:29:08.799248","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":"14","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:08.799327","last_change":"2024-04-17 22:28:59.254907","last_active":"2024-04-17 22:29:08.799327","last_peered":"2024-04-17 22:29:08.799327","last_clean":"2024-04-17 22:29:08.799327","last_became_active":"2024-04-17 22:28:59.254633","last_became_peered":"2024-04-17 22:28:59.254633","last_unstale":"2024-04-17 22:29:08.799327","last_undegraded":"2024-04-17 22:29:08.799327","last_fullsized":"2024-04-17 22:29:08.799327","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:28:58.239187","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:28:58.239187","last_clean_scrub_stamp":"2024-04-17 22:28:58.239187","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":"11","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:09.398108","last_change":"2024-04-17 22:29:07.480020","last_active":"2024-04-17 22:29:09.398108","last_peered":"2024-04-17 22:29:09.398108","last_clean":"2024-04-17 22:29:09.398108","last_became_active":"2024-04-17 22:29:07.479492","last_became_peered":"2024-04-17 22:29:07.479492","last_unstale":"2024-04-17 22:29:09.398108","last_undegraded":"2024-04-17 22:29:09.398108","last_fullsized":"2024-04-17 22:29:09.398108","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":"10","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:09.397806","last_change":"2024-04-17 22:29:07.821883","last_active":"2024-04-17 22:29:09.397806","last_peered":"2024-04-17 22:29:09.397806","last_clean":"2024-04-17 22:29:09.397806","last_became_active":"2024-04-17 22:29:07.821569","last_became_peered":"2024-04-17 22:29:07.821569","last_unstale":"2024-04-17 22:29:09.397806","last_undegraded":"2024-04-17 22:29:09.397806","last_fullsized":"2024-04-17 22:29:09.397806","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":"14","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:09.397706","last_change":"2024-04-17 22:28:59.254217","last_active":"2024-04-17 22:29:09.397706","last_peered":"2024-04-17 22:29:09.397706","last_clean":"2024-04-17 22:29:09.397706","last_became_active":"2024-04-17 22:28:59.253877","last_became_peered":"2024-04-17 22:28:59.253877","last_unstale":"2024-04-17 22:29:09.397706","last_undegraded":"2024-04-17 22:29:09.397706","last_fullsized":"2024-04-17 22:29:09.397706","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:28:58.239187","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:28:58.239187","last_clean_scrub_stamp":"2024-04-17 22:28:58.239187","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":"11","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:09.398009","last_change":"2024-04-17 22:29:07.480378","last_active":"2024-04-17 22:29:09.398009","last_peered":"2024-04-17 22:29:09.398009","last_clean":"2024-04-17 22:29:09.398009","last_became_active":"2024-04-17 22:29:07.479866","last_became_peered":"2024-04-17 22:29:07.479866","last_unstale":"2024-04-17 22:29:09.398009","last_undegraded":"2024-04-17 22:29:09.398009","last_fullsized":"2024-04-17 22:29:09.398009","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":"10","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:08.802861","last_change":"2024-04-17 22:29:07.821315","last_active":"2024-04-17 22:29:08.802861","last_peered":"2024-04-17 22:29:08.802861","last_clean":"2024-04-17 22:29:08.802861","last_became_active":"2024-04-17 22:29:07.820937","last_became_peered":"2024-04-17 22:29:07.820937","last_unstale":"2024-04-17 22:29:08.802861","last_undegraded":"2024-04-17 22:29:08.802861","last_fullsized":"2024-04-17 22:29:08.802861","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":"14","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:09.398670","last_change":"2024-04-17 22:28:59.254830","last_active":"2024-04-17 22:29:09.398670","last_peered":"2024-04-17 22:29:09.398670","last_clean":"2024-04-17 22:29:09.398670","last_became_active":"2024-04-17 22:28:59.254505","last_became_peered":"2024-04-17 22:28:59.254505","last_unstale":"2024-04-17 22:29:09.398670","last_undegraded":"2024-04-17 22:29:09.398670","last_fullsized":"2024-04-17 22:29:09.398670","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:28:58.239187","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:28:58.239187","last_clean_scrub_stamp":"2024-04-17 22:28:58.239187","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":"11","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:09.398561","last_change":"2024-04-17 22:29:07.475017","last_active":"2024-04-17 22:29:09.398561","last_peered":"2024-04-17 22:29:09.398561","last_clean":"2024-04-17 22:29:09.398561","last_became_active":"2024-04-17 22:29:07.473559","last_became_peered":"2024-04-17 22:29:07.473559","last_unstale":"2024-04-17 22:29:09.398561","last_undegraded":"2024-04-17 22:29:09.398561","last_fullsized":"2024-04-17 22:29:09.398561","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":"10","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:08.801429","last_change":"2024-04-17 22:29:07.808988","last_active":"2024-04-17 22:29:08.801429","last_peered":"2024-04-17 22:29:08.801429","last_clean":"2024-04-17 22:29:08.801429","last_became_active":"2024-04-17 22:29:07.808663","last_became_peered":"2024-04-17 22:29:07.808663","last_unstale":"2024-04-17 22:29:08.801429","last_undegraded":"2024-04-17 22:29:08.801429","last_fullsized":"2024-04-17 22:29:08.801429","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":"14","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:08.800951","last_change":"2024-04-17 22:28:59.254055","last_active":"2024-04-17 22:29:08.800951","last_peered":"2024-04-17 22:29:08.800951","last_clean":"2024-04-17 22:29:08.800951","last_became_active":"2024-04-17 22:28:59.253750","last_became_peered":"2024-04-17 22:28:59.253750","last_unstale":"2024-04-17 22:29:08.800951","last_undegraded":"2024-04-17 22:29:08.800951","last_fullsized":"2024-04-17 22:29:08.800951","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:28:58.239187","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:28:58.239187","last_clean_scrub_stamp":"2024-04-17 22:28:58.239187","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":"11","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:08.801348","last_change":"2024-04-17 22:29:07.466238","last_active":"2024-04-17 22:29:08.801348","last_peered":"2024-04-17 22:29:08.801348","last_clean":"2024-04-17 22:29:08.801348","last_became_active":"2024-04-17 22:29:07.465806","last_became_peered":"2024-04-17 22:29:07.465806","last_unstale":"2024-04-17 22:29:08.801348","last_undegraded":"2024-04-17 22:29:08.801348","last_fullsized":"2024-04-17 22:29:08.801348","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":"10","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:08.800621","last_change":"2024-04-17 22:29:07.809155","last_active":"2024-04-17 22:29:08.800621","last_peered":"2024-04-17 22:29:08.800621","last_clean":"2024-04-17 22:29:08.800621","last_became_active":"2024-04-17 22:29:07.808850","last_became_peered":"2024-04-17 22:29:07.808850","last_unstale":"2024-04-17 22:29:08.800621","last_undegraded":"2024-04-17 22:29:08.800621","last_fullsized":"2024-04-17 22:29:08.800621","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":"10","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:09.398583","last_change":"2024-04-17 22:29:07.832794","last_active":"2024-04-17 22:29:09.398583","last_peered":"2024-04-17 22:29:09.398583","last_clean":"2024-04-17 22:29:09.398583","last_became_active":"2024-04-17 22:29:07.832467","last_became_peered":"2024-04-17 22:29:07.832467","last_unstale":"2024-04-17 22:29:09.398583","last_undegraded":"2024-04-17 22:29:09.398583","last_fullsized":"2024-04-17 22:29:09.398583","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":"11","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:08.802745","last_change":"2024-04-17 22:29:07.477743","last_active":"2024-04-17 22:29:08.802745","last_peered":"2024-04-17 22:29:08.802745","last_clean":"2024-04-17 22:29:08.802745","last_became_active":"2024-04-17 22:29:07.477503","last_became_peered":"2024-04-17 22:29:07.477503","last_unstale":"2024-04-17 22:29:08.802745","last_undegraded":"2024-04-17 22:29:08.802745","last_fullsized":"2024-04-17 22:29:08.802745","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":"10","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:08.801269","last_change":"2024-04-17 22:29:07.828401","last_active":"2024-04-17 22:29:08.801269","last_peered":"2024-04-17 22:29:08.801269","last_clean":"2024-04-17 22:29:08.801269","last_became_active":"2024-04-17 22:29:07.828093","last_became_peered":"2024-04-17 22:29:07.828093","last_unstale":"2024-04-17 22:29:08.801269","last_undegraded":"2024-04-17 22:29:08.801269","last_fullsized":"2024-04-17 22:29:08.801269","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"15'2","reported_seq":"13","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:08.803325","last_change":"2024-04-17 22:29:07.478361","last_active":"2024-04-17 22:29:08.803325","last_peered":"2024-04-17 22:29:08.803325","last_clean":"2024-04-17 22:29:08.803325","last_became_active":"2024-04-17 22:29:07.477793","last_became_peered":"2024-04-17 22:29:07.477793","last_unstale":"2024-04-17 22:29:08.803325","last_undegraded":"2024-04-17 22:29:08.803325","last_fullsized":"2024-04-17 22:29:08.803325","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":"10","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:08.805677","last_change":"2024-04-17 22:29:07.815548","last_active":"2024-04-17 22:29:08.805677","last_peered":"2024-04-17 22:29:08.805677","last_clean":"2024-04-17 22:29:08.805677","last_became_active":"2024-04-17 22:29:07.815263","last_became_peered":"2024-04-17 22:29:07.815263","last_unstale":"2024-04-17 22:29:08.805677","last_undegraded":"2024-04-17 22:29:08.805677","last_fullsized":"2024-04-17 22:29:08.805677","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"15'1","reported_seq":"12","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:09.398690","last_change":"2024-04-17 22:29:07.482507","last_active":"2024-04-17 22:29:09.398690","last_peered":"2024-04-17 22:29:09.398690","last_clean":"2024-04-17 22:29:09.398690","last_became_active":"2024-04-17 22:29:07.481981","last_became_peered":"2024-04-17 22:29:07.481981","last_unstale":"2024-04-17 22:29:09.398690","last_undegraded":"2024-04-17 22:29:09.398690","last_fullsized":"2024-04-17 22:29:09.398690","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":"10","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:09.398811","last_change":"2024-04-17 22:29:07.828459","last_active":"2024-04-17 22:29:09.398811","last_peered":"2024-04-17 22:29:09.398811","last_clean":"2024-04-17 22:29:09.398811","last_became_active":"2024-04-17 22:29:07.828157","last_became_peered":"2024-04-17 22:29:07.828157","last_unstale":"2024-04-17 22:29:09.398811","last_undegraded":"2024-04-17 22:29:09.398811","last_fullsized":"2024-04-17 22:29:09.398811","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"15'1","reported_seq":"12","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:08.805937","last_change":"2024-04-17 22:29:07.471751","last_active":"2024-04-17 22:29:08.805937","last_peered":"2024-04-17 22:29:08.805937","last_clean":"2024-04-17 22:29:08.805937","last_became_active":"2024-04-17 22:29:07.471385","last_became_peered":"2024-04-17 22:29:07.471385","last_unstale":"2024-04-17 22:29:08.805937","last_undegraded":"2024-04-17 22:29:08.805937","last_fullsized":"2024-04-17 22:29:08.805937","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":"10","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:08.800362","last_change":"2024-04-17 22:29:07.812125","last_active":"2024-04-17 22:29:08.800362","last_peered":"2024-04-17 22:29:08.800362","last_clean":"2024-04-17 22:29:08.800362","last_became_active":"2024-04-17 22:29:07.811846","last_became_peered":"2024-04-17 22:29:07.811846","last_unstale":"2024-04-17 22:29:08.800362","last_undegraded":"2024-04-17 22:29:08.800362","last_fullsized":"2024-04-17 22:29:08.800362","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.20","version":"0'0","reported_seq":"11","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:09.397688","last_change":"2024-04-17 22:29:07.477127","last_active":"2024-04-17 22:29:09.397688","last_peered":"2024-04-17 22:29:09.397688","last_clean":"2024-04-17 22:29:09.397688","last_became_active":"2024-04-17 22:29:07.476773","last_became_peered":"2024-04-17 22:29:07.476773","last_unstale":"2024-04-17 22:29:09.397688","last_undegraded":"2024-04-17 22:29:09.397688","last_fullsized":"2024-04-17 22:29:09.397688","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":"10","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:08.800687","last_change":"2024-04-17 22:29:07.821132","last_active":"2024-04-17 22:29:08.800687","last_peered":"2024-04-17 22:29:08.800687","last_clean":"2024-04-17 22:29:08.800687","last_became_active":"2024-04-17 22:29:07.820838","last_became_peered":"2024-04-17 22:29:07.820838","last_unstale":"2024-04-17 22:29:08.800687","last_undegraded":"2024-04-17 22:29:08.800687","last_fullsized":"2024-04-17 22:29:08.800687","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.21","version":"0'0","reported_seq":"11","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:08.800070","last_change":"2024-04-17 22:29:07.460110","last_active":"2024-04-17 22:29:08.800070","last_peered":"2024-04-17 22:29:08.800070","last_clean":"2024-04-17 22:29:08.800070","last_became_active":"2024-04-17 22:29:07.459611","last_became_peered":"2024-04-17 22:29:07.459611","last_unstale":"2024-04-17 22:29:08.800070","last_undegraded":"2024-04-17 22:29:08.800070","last_fullsized":"2024-04-17 22:29:08.800070","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":"10","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:08.802908","last_change":"2024-04-17 22:29:07.818496","last_active":"2024-04-17 22:29:08.802908","last_peered":"2024-04-17 22:29:08.802908","last_clean":"2024-04-17 22:29:08.802908","last_became_active":"2024-04-17 22:29:07.818184","last_became_peered":"2024-04-17 22:29:07.818184","last_unstale":"2024-04-17 22:29:08.802908","last_undegraded":"2024-04-17 22:29:08.802908","last_fullsized":"2024-04-17 22:29:08.802908","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.22","version":"0'0","reported_seq":"11","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:09.398098","last_change":"2024-04-17 22:29:07.481695","last_active":"2024-04-17 22:29:09.398098","last_peered":"2024-04-17 22:29:09.398098","last_clean":"2024-04-17 22:29:09.398098","last_became_active":"2024-04-17 22:29:07.479982","last_became_peered":"2024-04-17 22:29:07.479982","last_unstale":"2024-04-17 22:29:09.398098","last_undegraded":"2024-04-17 22:29:09.398098","last_fullsized":"2024-04-17 22:29:09.398098","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":"10","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:09.397819","last_change":"2024-04-17 22:29:07.833317","last_active":"2024-04-17 22:29:09.397819","last_peered":"2024-04-17 22:29:09.397819","last_clean":"2024-04-17 22:29:09.397819","last_became_active":"2024-04-17 22:29:07.827050","last_became_peered":"2024-04-17 22:29:07.827050","last_unstale":"2024-04-17 22:29:09.397819","last_undegraded":"2024-04-17 22:29:09.397819","last_fullsized":"2024-04-17 22:29:09.397819","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.23","version":"0'0","reported_seq":"11","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:09.398251","last_change":"2024-04-17 22:29:07.477449","last_active":"2024-04-17 22:29:09.398251","last_peered":"2024-04-17 22:29:09.398251","last_clean":"2024-04-17 22:29:09.398251","last_became_active":"2024-04-17 22:29:07.477155","last_became_peered":"2024-04-17 22:29:07.477155","last_unstale":"2024-04-17 22:29:09.398251","last_undegraded":"2024-04-17 22:29:09.398251","last_fullsized":"2024-04-17 22:29:09.398251","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":"10","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:08.800540","last_change":"2024-04-17 22:29:07.808974","last_active":"2024-04-17 22:29:08.800540","last_peered":"2024-04-17 22:29:08.800540","last_clean":"2024-04-17 22:29:08.800540","last_became_active":"2024-04-17 22:29:07.808747","last_became_peered":"2024-04-17 22:29:07.808747","last_unstale":"2024-04-17 22:29:08.800540","last_undegraded":"2024-04-17 22:29:08.800540","last_fullsized":"2024-04-17 22:29:08.800540","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.24","version":"0'0","reported_seq":"11","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:08.799263","last_change":"2024-04-17 22:29:07.466052","last_active":"2024-04-17 22:29:08.799263","last_peered":"2024-04-17 22:29:08.799263","last_clean":"2024-04-17 22:29:08.799263","last_became_active":"2024-04-17 22:29:07.465576","last_became_peered":"2024-04-17 22:29:07.465576","last_unstale":"2024-04-17 22:29:08.799263","last_undegraded":"2024-04-17 22:29:08.799263","last_fullsized":"2024-04-17 22:29:08.799263","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":"10","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:09.398346","last_change":"2024-04-17 22:29:07.829066","last_active":"2024-04-17 22:29:09.398346","last_peered":"2024-04-17 22:29:09.398346","last_clean":"2024-04-17 22:29:09.398346","last_became_active":"2024-04-17 22:29:07.828735","last_became_peered":"2024-04-17 22:29:07.828735","last_unstale":"2024-04-17 22:29:09.398346","last_undegraded":"2024-04-17 22:29:09.398346","last_fullsized":"2024-04-17 22:29:09.398346","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.25","version":"0'0","reported_seq":"11","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:08.800709","last_change":"2024-04-17 22:29:07.462011","last_active":"2024-04-17 22:29:08.800709","last_peered":"2024-04-17 22:29:08.800709","last_clean":"2024-04-17 22:29:08.800709","last_became_active":"2024-04-17 22:29:07.461693","last_became_peered":"2024-04-17 22:29:07.461693","last_unstale":"2024-04-17 22:29:08.800709","last_undegraded":"2024-04-17 22:29:08.800709","last_fullsized":"2024-04-17 22:29:08.800709","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":"10","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:09.398722","last_change":"2024-04-17 22:29:07.826655","last_active":"2024-04-17 22:29:09.398722","last_peered":"2024-04-17 22:29:09.398722","last_clean":"2024-04-17 22:29:09.398722","last_became_active":"2024-04-17 22:29:07.823665","last_became_peered":"2024-04-17 22:29:07.823665","last_unstale":"2024-04-17 22:29:09.398722","last_undegraded":"2024-04-17 22:29:09.398722","last_fullsized":"2024-04-17 22:29:09.398722","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.26","version":"16'1","reported_seq":"12","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:09.396977","last_change":"2024-04-17 22:29:07.480854","last_active":"2024-04-17 22:29:09.396977","last_peered":"2024-04-17 22:29:09.396977","last_clean":"2024-04-17 22:29:09.396977","last_became_active":"2024-04-17 22:29:07.479975","last_became_peered":"2024-04-17 22:29:07.479975","last_unstale":"2024-04-17 22:29:09.396977","last_undegraded":"2024-04-17 22:29:09.396977","last_fullsized":"2024-04-17 22:29:09.396977","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":"10","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:08.801236","last_change":"2024-04-17 22:29:07.813744","last_active":"2024-04-17 22:29:08.801236","last_peered":"2024-04-17 22:29:08.801236","last_clean":"2024-04-17 22:29:08.801236","last_became_active":"2024-04-17 22:29:07.810646","last_became_peered":"2024-04-17 22:29:07.810646","last_unstale":"2024-04-17 22:29:08.801236","last_undegraded":"2024-04-17 22:29:08.801236","last_fullsized":"2024-04-17 22:29:08.801236","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.27","version":"0'0","reported_seq":"11","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:09.398895","last_change":"2024-04-17 22:29:07.476428","last_active":"2024-04-17 22:29:09.398895","last_peered":"2024-04-17 22:29:09.398895","last_clean":"2024-04-17 22:29:09.398895","last_became_active":"2024-04-17 22:29:07.476128","last_became_peered":"2024-04-17 22:29:07.476128","last_unstale":"2024-04-17 22:29:09.398895","last_undegraded":"2024-04-17 22:29:09.398895","last_fullsized":"2024-04-17 22:29:09.398895","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":"10","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:09.397602","last_change":"2024-04-17 22:29:07.823340","last_active":"2024-04-17 22:29:09.397602","last_peered":"2024-04-17 22:29:09.397602","last_clean":"2024-04-17 22:29:09.397602","last_became_active":"2024-04-17 22:29:07.822764","last_became_peered":"2024-04-17 22:29:07.822764","last_unstale":"2024-04-17 22:29:09.397602","last_undegraded":"2024-04-17 22:29:09.397602","last_fullsized":"2024-04-17 22:29:09.397602","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.28","version":"0'0","reported_seq":"11","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:09.397875","last_change":"2024-04-17 22:29:07.477137","last_active":"2024-04-17 22:29:09.397875","last_peered":"2024-04-17 22:29:09.397875","last_clean":"2024-04-17 22:29:09.397875","last_became_active":"2024-04-17 22:29:07.476796","last_became_peered":"2024-04-17 22:29:07.476796","last_unstale":"2024-04-17 22:29:09.397875","last_undegraded":"2024-04-17 22:29:09.397875","last_fullsized":"2024-04-17 22:29:09.397875","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":"10","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:09.397527","last_change":"2024-04-17 22:29:07.823119","last_active":"2024-04-17 22:29:09.397527","last_peered":"2024-04-17 22:29:09.397527","last_clean":"2024-04-17 22:29:09.397527","last_became_active":"2024-04-17 22:29:07.822640","last_became_peered":"2024-04-17 22:29:07.822640","last_unstale":"2024-04-17 22:29:09.397527","last_undegraded":"2024-04-17 22:29:09.397527","last_fullsized":"2024-04-17 22:29:09.397527","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.29","version":"0'0","reported_seq":"11","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:08.799663","last_change":"2024-04-17 22:29:07.473467","last_active":"2024-04-17 22:29:08.799663","last_peered":"2024-04-17 22:29:08.799663","last_clean":"2024-04-17 22:29:08.799663","last_became_active":"2024-04-17 22:29:07.473028","last_became_peered":"2024-04-17 22:29:07.473028","last_unstale":"2024-04-17 22:29:08.799663","last_undegraded":"2024-04-17 22:29:08.799663","last_fullsized":"2024-04-17 22:29:08.799663","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":"10","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:08.802996","last_change":"2024-04-17 22:29:07.817097","last_active":"2024-04-17 22:29:08.802996","last_peered":"2024-04-17 22:29:08.802996","last_clean":"2024-04-17 22:29:08.802996","last_became_active":"2024-04-17 22:29:07.816813","last_became_peered":"2024-04-17 22:29:07.816813","last_unstale":"2024-04-17 22:29:08.802996","last_undegraded":"2024-04-17 22:29:08.802996","last_fullsized":"2024-04-17 22:29:08.802996","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2a","version":"0'0","reported_seq":"11","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:08.799564","last_change":"2024-04-17 22:29:07.464330","last_active":"2024-04-17 22:29:08.799564","last_peered":"2024-04-17 22:29:08.799564","last_clean":"2024-04-17 22:29:08.799564","last_became_active":"2024-04-17 22:29:07.464002","last_became_peered":"2024-04-17 22:29:07.464002","last_unstale":"2024-04-17 22:29:08.799564","last_undegraded":"2024-04-17 22:29:08.799564","last_fullsized":"2024-04-17 22:29:08.799564","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":"10","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:08.799834","last_change":"2024-04-17 22:29:07.828690","last_active":"2024-04-17 22:29:08.799834","last_peered":"2024-04-17 22:29:08.799834","last_clean":"2024-04-17 22:29:08.799834","last_became_active":"2024-04-17 22:29:07.828407","last_became_peered":"2024-04-17 22:29:07.828407","last_unstale":"2024-04-17 22:29:08.799834","last_undegraded":"2024-04-17 22:29:08.799834","last_fullsized":"2024-04-17 22:29:08.799834","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2b","version":"15'1","reported_seq":"12","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:08.800356","last_change":"2024-04-17 22:29:07.471300","last_active":"2024-04-17 22:29:08.800356","last_peered":"2024-04-17 22:29:08.800356","last_clean":"2024-04-17 22:29:08.800356","last_became_active":"2024-04-17 22:29:07.470886","last_became_peered":"2024-04-17 22:29:07.470886","last_unstale":"2024-04-17 22:29:08.800356","last_undegraded":"2024-04-17 22:29:08.800356","last_fullsized":"2024-04-17 22:29:08.800356","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":"10","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:09.398460","last_change":"2024-04-17 22:29:07.826555","last_active":"2024-04-17 22:29:09.398460","last_peered":"2024-04-17 22:29:09.398460","last_clean":"2024-04-17 22:29:09.398460","last_became_active":"2024-04-17 22:29:07.826262","last_became_peered":"2024-04-17 22:29:07.826262","last_unstale":"2024-04-17 22:29:09.398460","last_undegraded":"2024-04-17 22:29:09.398460","last_fullsized":"2024-04-17 22:29:09.398460","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2c","version":"0'0","reported_seq":"11","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:08.801315","last_change":"2024-04-17 22:29:07.462176","last_active":"2024-04-17 22:29:08.801315","last_peered":"2024-04-17 22:29:08.801315","last_clean":"2024-04-17 22:29:08.801315","last_became_active":"2024-04-17 22:29:07.461594","last_became_peered":"2024-04-17 22:29:07.461594","last_unstale":"2024-04-17 22:29:08.801315","last_undegraded":"2024-04-17 22:29:08.801315","last_fullsized":"2024-04-17 22:29:08.801315","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":"10","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:08.802743","last_change":"2024-04-17 22:29:07.816896","last_active":"2024-04-17 22:29:08.802743","last_peered":"2024-04-17 22:29:08.802743","last_clean":"2024-04-17 22:29:08.802743","last_became_active":"2024-04-17 22:29:07.816538","last_became_peered":"2024-04-17 22:29:07.816538","last_unstale":"2024-04-17 22:29:08.802743","last_undegraded":"2024-04-17 22:29:08.802743","last_fullsized":"2024-04-17 22:29:08.802743","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2d","version":"15'1","reported_seq":"12","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:09.398663","last_change":"2024-04-17 22:29:07.479214","last_active":"2024-04-17 22:29:09.398663","last_peered":"2024-04-17 22:29:09.398663","last_clean":"2024-04-17 22:29:09.398663","last_became_active":"2024-04-17 22:29:07.478790","last_became_peered":"2024-04-17 22:29:07.478790","last_unstale":"2024-04-17 22:29:09.398663","last_undegraded":"2024-04-17 22:29:09.398663","last_fullsized":"2024-04-17 22:29:09.398663","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":"10","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:09.398672","last_change":"2024-04-17 22:29:07.832349","last_active":"2024-04-17 22:29:09.398672","last_peered":"2024-04-17 22:29:09.398672","last_clean":"2024-04-17 22:29:09.398672","last_became_active":"2024-04-17 22:29:07.831999","last_became_peered":"2024-04-17 22:29:07.831999","last_unstale":"2024-04-17 22:29:09.398672","last_undegraded":"2024-04-17 22:29:09.398672","last_fullsized":"2024-04-17 22:29:09.398672","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2e","version":"0'0","reported_seq":"11","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:08.801154","last_change":"2024-04-17 22:29:07.483244","last_active":"2024-04-17 22:29:08.801154","last_peered":"2024-04-17 22:29:08.801154","last_clean":"2024-04-17 22:29:08.801154","last_became_active":"2024-04-17 22:29:07.482893","last_became_peered":"2024-04-17 22:29:07.482893","last_unstale":"2024-04-17 22:29:08.801154","last_undegraded":"2024-04-17 22:29:08.801154","last_fullsized":"2024-04-17 22:29:08.801154","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":"10","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:08.801323","last_change":"2024-04-17 22:29:07.806512","last_active":"2024-04-17 22:29:08.801323","last_peered":"2024-04-17 22:29:08.801323","last_clean":"2024-04-17 22:29:08.801323","last_became_active":"2024-04-17 22:29:07.806182","last_became_peered":"2024-04-17 22:29:07.806182","last_unstale":"2024-04-17 22:29:08.801323","last_undegraded":"2024-04-17 22:29:08.801323","last_fullsized":"2024-04-17 22:29:08.801323","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2f","version":"0'0","reported_seq":"11","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:09.398900","last_change":"2024-04-17 22:29:07.478655","last_active":"2024-04-17 22:29:09.398900","last_peered":"2024-04-17 22:29:09.398900","last_clean":"2024-04-17 22:29:09.398900","last_became_active":"2024-04-17 22:29:07.478346","last_became_peered":"2024-04-17 22:29:07.478346","last_unstale":"2024-04-17 22:29:09.398900","last_undegraded":"2024-04-17 22:29:09.398900","last_fullsized":"2024-04-17 22:29:09.398900","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":"10","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:08.799410","last_change":"2024-04-17 22:29:07.825658","last_active":"2024-04-17 22:29:08.799410","last_peered":"2024-04-17 22:29:08.799410","last_clean":"2024-04-17 22:29:08.799410","last_became_active":"2024-04-17 22:29:07.825349","last_became_peered":"2024-04-17 22:29:07.825349","last_unstale":"2024-04-17 22:29:08.799410","last_undegraded":"2024-04-17 22:29:08.799410","last_fullsized":"2024-04-17 22:29:08.799410","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.30","version":"15'4","reported_seq":"15","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:08.802113","last_change":"2024-04-17 22:29:07.475470","last_active":"2024-04-17 22:29:08.802113","last_peered":"2024-04-17 22:29:08.802113","last_clean":"2024-04-17 22:29:08.802113","last_became_active":"2024-04-17 22:29:07.474917","last_became_peered":"2024-04-17 22:29:07.474917","last_unstale":"2024-04-17 22:29:08.802113","last_undegraded":"2024-04-17 22:29:08.802113","last_fullsized":"2024-04-17 22:29:08.802113","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":"10","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:09.397646","last_change":"2024-04-17 22:29:07.823624","last_active":"2024-04-17 22:29:09.397646","last_peered":"2024-04-17 22:29:09.397646","last_clean":"2024-04-17 22:29:09.397646","last_became_active":"2024-04-17 22:29:07.822815","last_became_peered":"2024-04-17 22:29:07.822815","last_unstale":"2024-04-17 22:29:09.397646","last_undegraded":"2024-04-17 22:29:09.397646","last_fullsized":"2024-04-17 22:29:09.397646","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.31","version":"0'0","reported_seq":"11","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:08.802386","last_change":"2024-04-17 22:29:07.474318","last_active":"2024-04-17 22:29:08.802386","last_peered":"2024-04-17 22:29:08.802386","last_clean":"2024-04-17 22:29:08.802386","last_became_active":"2024-04-17 22:29:07.473754","last_became_peered":"2024-04-17 22:29:07.473754","last_unstale":"2024-04-17 22:29:08.802386","last_undegraded":"2024-04-17 22:29:08.802386","last_fullsized":"2024-04-17 22:29:08.802386","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":"10","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:09.398038","last_change":"2024-04-17 22:29:07.829098","last_active":"2024-04-17 22:29:09.398038","last_peered":"2024-04-17 22:29:09.398038","last_clean":"2024-04-17 22:29:09.398038","last_became_active":"2024-04-17 22:29:07.828773","last_became_peered":"2024-04-17 22:29:07.828773","last_unstale":"2024-04-17 22:29:09.398038","last_undegraded":"2024-04-17 22:29:09.398038","last_fullsized":"2024-04-17 22:29:09.398038","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.32","version":"0'0","reported_seq":"11","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:08.800366","last_change":"2024-04-17 22:29:07.470666","last_active":"2024-04-17 22:29:08.800366","last_peered":"2024-04-17 22:29:08.800366","last_clean":"2024-04-17 22:29:08.800366","last_became_active":"2024-04-17 22:29:07.470363","last_became_peered":"2024-04-17 22:29:07.470363","last_unstale":"2024-04-17 22:29:08.800366","last_undegraded":"2024-04-17 22:29:08.800366","last_fullsized":"2024-04-17 22:29:08.800366","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":"10","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:08.802294","last_change":"2024-04-17 22:29:07.827580","last_active":"2024-04-17 22:29:08.802294","last_peered":"2024-04-17 22:29:08.802294","last_clean":"2024-04-17 22:29:08.802294","last_became_active":"2024-04-17 22:29:07.827233","last_became_peered":"2024-04-17 22:29:07.827233","last_unstale":"2024-04-17 22:29:08.802294","last_undegraded":"2024-04-17 22:29:08.802294","last_fullsized":"2024-04-17 22:29:08.802294","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.33","version":"15'1","reported_seq":"12","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:08.799425","last_change":"2024-04-17 22:29:07.460978","last_active":"2024-04-17 22:29:08.799425","last_peered":"2024-04-17 22:29:08.799425","last_clean":"2024-04-17 22:29:08.799425","last_became_active":"2024-04-17 22:29:07.460659","last_became_peered":"2024-04-17 22:29:07.460659","last_unstale":"2024-04-17 22:29:08.799425","last_undegraded":"2024-04-17 22:29:08.799425","last_fullsized":"2024-04-17 22:29:08.799425","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":"10","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:08.803388","last_change":"2024-04-17 22:29:07.816335","last_active":"2024-04-17 22:29:08.803388","last_peered":"2024-04-17 22:29:08.803388","last_clean":"2024-04-17 22:29:08.803388","last_became_active":"2024-04-17 22:29:07.816090","last_became_peered":"2024-04-17 22:29:07.816090","last_unstale":"2024-04-17 22:29:08.803388","last_undegraded":"2024-04-17 22:29:08.803388","last_fullsized":"2024-04-17 22:29:08.803388","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.34","version":"17'7","reported_seq":"18","reported_epoch":"17","state":"active+clean","last_fresh":"2024-04-17 22:29:29.339150","last_change":"2024-04-17 22:29:07.475702","last_active":"2024-04-17 22:29:29.339150","last_peered":"2024-04-17 22:29:29.339150","last_clean":"2024-04-17 22:29:29.339150","last_became_active":"2024-04-17 22:29:07.475395","last_became_peered":"2024-04-17 22:29:07.475395","last_unstale":"2024-04-17 22:29:29.339150","last_undegraded":"2024-04-17 22:29:29.339150","last_fullsized":"2024-04-17 22:29:29.339150","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":2203,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":"10","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:08.802632","last_change":"2024-04-17 22:29:07.818538","last_active":"2024-04-17 22:29:08.802632","last_peered":"2024-04-17 22:29:08.802632","last_clean":"2024-04-17 22:29:08.802632","last_became_active":"2024-04-17 22:29:07.817164","last_became_peered":"2024-04-17 22:29:07.817164","last_unstale":"2024-04-17 22:29:08.802632","last_undegraded":"2024-04-17 22:29:08.802632","last_fullsized":"2024-04-17 22:29:08.802632","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.35","version":"0'0","reported_seq":"11","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:08.800453","last_change":"2024-04-17 22:29:07.467651","last_active":"2024-04-17 22:29:08.800453","last_peered":"2024-04-17 22:29:08.800453","last_clean":"2024-04-17 22:29:08.800453","last_became_active":"2024-04-17 22:29:07.467327","last_became_peered":"2024-04-17 22:29:07.467327","last_unstale":"2024-04-17 22:29:08.800453","last_undegraded":"2024-04-17 22:29:08.800453","last_fullsized":"2024-04-17 22:29:08.800453","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":"10","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:08.801055","last_change":"2024-04-17 22:29:07.809693","last_active":"2024-04-17 22:29:08.801055","last_peered":"2024-04-17 22:29:08.801055","last_clean":"2024-04-17 22:29:08.801055","last_became_active":"2024-04-17 22:29:07.809415","last_became_peered":"2024-04-17 22:29:07.809415","last_unstale":"2024-04-17 22:29:08.801055","last_undegraded":"2024-04-17 22:29:08.801055","last_fullsized":"2024-04-17 22:29:08.801055","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.36","version":"0'0","reported_seq":"11","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:08.800982","last_change":"2024-04-17 22:29:07.483402","last_active":"2024-04-17 22:29:08.800982","last_peered":"2024-04-17 22:29:08.800982","last_clean":"2024-04-17 22:29:08.800982","last_became_active":"2024-04-17 22:29:07.482914","last_became_peered":"2024-04-17 22:29:07.482914","last_unstale":"2024-04-17 22:29:08.800982","last_undegraded":"2024-04-17 22:29:08.800982","last_fullsized":"2024-04-17 22:29:08.800982","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":"10","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:08.802739","last_change":"2024-04-17 22:29:07.827623","last_active":"2024-04-17 22:29:08.802739","last_peered":"2024-04-17 22:29:08.802739","last_clean":"2024-04-17 22:29:08.802739","last_became_active":"2024-04-17 22:29:07.827337","last_became_peered":"2024-04-17 22:29:07.827337","last_unstale":"2024-04-17 22:29:08.802739","last_undegraded":"2024-04-17 22:29:08.802739","last_fullsized":"2024-04-17 22:29:08.802739","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.37","version":"0'0","reported_seq":"11","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:08.806112","last_change":"2024-04-17 22:29:07.463449","last_active":"2024-04-17 22:29:08.806112","last_peered":"2024-04-17 22:29:08.806112","last_clean":"2024-04-17 22:29:08.806112","last_became_active":"2024-04-17 22:29:07.463110","last_became_peered":"2024-04-17 22:29:07.463110","last_unstale":"2024-04-17 22:29:08.806112","last_undegraded":"2024-04-17 22:29:08.806112","last_fullsized":"2024-04-17 22:29:08.806112","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":"10","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:08.802179","last_change":"2024-04-17 22:29:07.815214","last_active":"2024-04-17 22:29:08.802179","last_peered":"2024-04-17 22:29:08.802179","last_clean":"2024-04-17 22:29:08.802179","last_became_active":"2024-04-17 22:29:07.814938","last_became_peered":"2024-04-17 22:29:07.814938","last_unstale":"2024-04-17 22:29:08.802179","last_undegraded":"2024-04-17 22:29:08.802179","last_fullsized":"2024-04-17 22:29:08.802179","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.38","version":"0'0","reported_seq":"11","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:08.802121","last_change":"2024-04-17 22:29:07.474682","last_active":"2024-04-17 22:29:08.802121","last_peered":"2024-04-17 22:29:08.802121","last_clean":"2024-04-17 22:29:08.802121","last_became_active":"2024-04-17 22:29:07.474305","last_became_peered":"2024-04-17 22:29:07.474305","last_unstale":"2024-04-17 22:29:08.802121","last_undegraded":"2024-04-17 22:29:08.802121","last_fullsized":"2024-04-17 22:29:08.802121","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":"10","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:09.397814","last_change":"2024-04-17 22:29:07.829769","last_active":"2024-04-17 22:29:09.397814","last_peered":"2024-04-17 22:29:09.397814","last_clean":"2024-04-17 22:29:09.397814","last_became_active":"2024-04-17 22:29:07.829484","last_became_peered":"2024-04-17 22:29:07.829484","last_unstale":"2024-04-17 22:29:09.397814","last_undegraded":"2024-04-17 22:29:09.397814","last_fullsized":"2024-04-17 22:29:09.397814","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.39","version":"0'0","reported_seq":"11","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:08.799584","last_change":"2024-04-17 22:29:07.458386","last_active":"2024-04-17 22:29:08.799584","last_peered":"2024-04-17 22:29:08.799584","last_clean":"2024-04-17 22:29:08.799584","last_became_active":"2024-04-17 22:29:07.456695","last_became_peered":"2024-04-17 22:29:07.456695","last_unstale":"2024-04-17 22:29:08.799584","last_undegraded":"2024-04-17 22:29:08.799584","last_fullsized":"2024-04-17 22:29:08.799584","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":"10","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:08.802578","last_change":"2024-04-17 22:29:07.821490","last_active":"2024-04-17 22:29:08.802578","last_peered":"2024-04-17 22:29:08.802578","last_clean":"2024-04-17 22:29:08.802578","last_became_active":"2024-04-17 22:29:07.821103","last_became_peered":"2024-04-17 22:29:07.821103","last_unstale":"2024-04-17 22:29:08.802578","last_undegraded":"2024-04-17 22:29:08.802578","last_fullsized":"2024-04-17 22:29:08.802578","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3a","version":"15'1","reported_seq":"12","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:08.800480","last_change":"2024-04-17 22:29:07.472610","last_active":"2024-04-17 22:29:08.800480","last_peered":"2024-04-17 22:29:08.800480","last_clean":"2024-04-17 22:29:08.800480","last_became_active":"2024-04-17 22:29:07.472297","last_became_peered":"2024-04-17 22:29:07.472297","last_unstale":"2024-04-17 22:29:08.800480","last_undegraded":"2024-04-17 22:29:08.800480","last_fullsized":"2024-04-17 22:29:08.800480","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":"10","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:08.800273","last_change":"2024-04-17 22:29:07.820738","last_active":"2024-04-17 22:29:08.800273","last_peered":"2024-04-17 22:29:08.800273","last_clean":"2024-04-17 22:29:08.800273","last_became_active":"2024-04-17 22:29:07.820354","last_became_peered":"2024-04-17 22:29:07.820354","last_unstale":"2024-04-17 22:29:08.800273","last_undegraded":"2024-04-17 22:29:08.800273","last_fullsized":"2024-04-17 22:29:08.800273","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3b","version":"15'1","reported_seq":"12","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:08.802950","last_change":"2024-04-17 22:29:07.473963","last_active":"2024-04-17 22:29:08.802950","last_peered":"2024-04-17 22:29:08.802950","last_clean":"2024-04-17 22:29:08.802950","last_became_active":"2024-04-17 22:29:07.472018","last_became_peered":"2024-04-17 22:29:07.472018","last_unstale":"2024-04-17 22:29:08.802950","last_undegraded":"2024-04-17 22:29:08.802950","last_fullsized":"2024-04-17 22:29:08.802950","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":"10","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:08.803375","last_change":"2024-04-17 22:29:07.824919","last_active":"2024-04-17 22:29:08.803375","last_peered":"2024-04-17 22:29:08.803375","last_clean":"2024-04-17 22:29:08.803375","last_became_active":"2024-04-17 22:29:07.824600","last_became_peered":"2024-04-17 22:29:07.824600","last_unstale":"2024-04-17 22:29:08.803375","last_undegraded":"2024-04-17 22:29:08.803375","last_fullsized":"2024-04-17 22:29:08.803375","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3c","version":"15'2","reported_seq":"13","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:08.802864","last_change":"2024-04-17 22:29:07.475485","last_active":"2024-04-17 22:29:08.802864","last_peered":"2024-04-17 22:29:08.802864","last_clean":"2024-04-17 22:29:08.802864","last_became_active":"2024-04-17 22:29:07.475182","last_became_peered":"2024-04-17 22:29:07.475182","last_unstale":"2024-04-17 22:29:08.802864","last_undegraded":"2024-04-17 22:29:08.802864","last_fullsized":"2024-04-17 22:29:08.802864","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":"10","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:08.800557","last_change":"2024-04-17 22:29:07.810455","last_active":"2024-04-17 22:29:08.800557","last_peered":"2024-04-17 22:29:08.800557","last_clean":"2024-04-17 22:29:08.800557","last_became_active":"2024-04-17 22:29:07.810143","last_became_peered":"2024-04-17 22:29:07.810143","last_unstale":"2024-04-17 22:29:08.800557","last_undegraded":"2024-04-17 22:29:08.800557","last_fullsized":"2024-04-17 22:29:08.800557","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3d","version":"0'0","reported_seq":"11","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:08.800361","last_change":"2024-04-17 22:29:07.468187","last_active":"2024-04-17 22:29:08.800361","last_peered":"2024-04-17 22:29:08.800361","last_clean":"2024-04-17 22:29:08.800361","last_became_active":"2024-04-17 22:29:07.467849","last_became_peered":"2024-04-17 22:29:07.467849","last_unstale":"2024-04-17 22:29:08.800361","last_undegraded":"2024-04-17 22:29:08.800361","last_fullsized":"2024-04-17 22:29:08.800361","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":"10","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:08.805405","last_change":"2024-04-17 22:29:07.822173","last_active":"2024-04-17 22:29:08.805405","last_peered":"2024-04-17 22:29:08.805405","last_clean":"2024-04-17 22:29:08.805405","last_became_active":"2024-04-17 22:29:07.821875","last_became_peered":"2024-04-17 22:29:07.821875","last_unstale":"2024-04-17 22:29:08.805405","last_undegraded":"2024-04-17 22:29:08.805405","last_fullsized":"2024-04-17 22:29:08.805405","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3e","version":"0'0","reported_seq":"11","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:08.802827","last_change":"2024-04-17 22:29:07.479547","last_active":"2024-04-17 22:29:08.802827","last_peered":"2024-04-17 22:29:08.802827","last_clean":"2024-04-17 22:29:08.802827","last_became_active":"2024-04-17 22:29:07.479004","last_became_peered":"2024-04-17 22:29:07.479004","last_unstale":"2024-04-17 22:29:08.802827","last_undegraded":"2024-04-17 22:29:08.802827","last_fullsized":"2024-04-17 22:29:08.802827","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":"10","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:08.800895","last_change":"2024-04-17 22:29:07.819432","last_active":"2024-04-17 22:29:08.800895","last_peered":"2024-04-17 22:29:08.800895","last_clean":"2024-04-17 22:29:08.800895","last_became_active":"2024-04-17 22:29:07.819001","last_became_peered":"2024-04-17 22:29:07.819001","last_unstale":"2024-04-17 22:29:08.800895","last_undegraded":"2024-04-17 22:29:08.800895","last_fullsized":"2024-04-17 22:29:08.800895","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:07.435832","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:07.435832","last_clean_scrub_stamp":"2024-04-17 22:29:07.435832","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3f","version":"15'3","reported_seq":"14","reported_epoch":"16","state":"active+clean","last_fresh":"2024-04-17 22:29:08.801001","last_change":"2024-04-17 22:29:07.466765","last_active":"2024-04-17 22:29:08.801001","last_peered":"2024-04-17 22:29:08.801001","last_clean":"2024-04-17 22:29:08.801001","last_became_active":"2024-04-17 22:29:07.463177","last_became_peered":"2024-04-17 22:29:07.463177","last_unstale":"2024-04-17 22:29:08.801001","last_undegraded":"2024-04-17 22:29:08.801001","last_fullsized":"2024-04-17 22:29:08.801001","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:29:06.425872","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:29:06.425872","last_clean_scrub_stamp":"2024-04-17 22:29:06.425872","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":544,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":4},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":4},{"poolid":2,"num_pg":64,"stat_sum":{"num_bytes":3597,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":7194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":43,"ondisk_log_size":43,"up":128,"acting":128,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705692,"num_pgs":82,"num_osds":1,"num_per_pool_osds":1,"kb":94371840,"kb_used":1048752,"kb_used_data":168,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323088,"statfs":{"total":96636764160,"available":95562842112,"internally_reserved":1073741824,"allocated":172032,"data_stored":45697,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 17 22:29:57 2024","interfaces":[{"interface":"back","average":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"min":{"1min":0.19900000000000001,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.443,"5min":0.443,"15min":0.443},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.317,"5min":0.317,"15min":0.317},"min":{"1min":0.214,"5min":0.214,"15min":0.214},"max":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"last":0.34599999999999997}]},{"osd":1,"last update":"Wed Apr 17 22:29:57 2024","interfaces":[{"interface":"back","average":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"min":{"1min":0.19600000000000001,"5min":0.19600000000000001,"15min":0.19600000000000001},"max":{"1min":0.44,"5min":0.44,"15min":0.44},"last":0.376},{"interface":"front","average":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"last":0.36899999999999999}]},{"osd":2,"last update":"Wed Apr 17 22:29:57 2024","interfaces":[{"interface":"back","average":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"min":{"1min":0.20699999999999999,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"last":0.40699999999999997}]}]},{"osd":2,"up_from":9,"seq":38654705692,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"kb":94371840,"kb_used":1048740,"kb_used_data":156,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323100,"statfs":{"total":96636764160,"available":95562854400,"internally_reserved":1073741824,"allocated":159744,"data_stored":43370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 17 22:29:58 2024","interfaces":[{"interface":"back","average":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"min":{"1min":0.112,"5min":0.112,"15min":0.112},"max":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"last":0.27400000000000002},{"interface":"front","average":{"1min":0.309,"5min":0.309,"15min":0.309},"min":{"1min":0.155,"5min":0.155,"15min":0.155},"max":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"last":0.34399999999999997}]},{"osd":1,"last update":"Wed Apr 17 22:29:58 2024","interfaces":[{"interface":"back","average":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"min":{"1min":0.17899999999999999,"5min":0.17899999999999999,"15min":0.17899999999999999},"max":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"last":0.26400000000000001},{"interface":"front","average":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"min":{"1min":0.192,"5min":0.192,"15min":0.192},"max":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"last":0.30399999999999999}]},{"osd":3,"last update":"Wed Apr 17 22:29:58 2024","interfaces":[{"interface":"back","average":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"min":{"1min":0.191,"5min":0.191,"15min":0.191},"max":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"last":0.318},{"interface":"front","average":{"1min":0.31,"5min":0.31,"15min":0.31},"min":{"1min":0.214,"5min":0.214,"15min":0.214},"max":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"last":0.28799999999999998}]}]},{"osd":0,"up_from":9,"seq":38654705692,"num_pgs":78,"num_osds":1,"num_per_pool_osds":1,"kb":94371840,"kb_used":1048744,"kb_used_data":160,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323096,"statfs":{"total":96636764160,"available":95562850304,"internally_reserved":1073741824,"allocated":163840,"data_stored":45469,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Apr 17 22:29:55 2024","interfaces":[{"interface":"back","average":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"min":{"1min":0.17299999999999999,"5min":0.17299999999999999,"15min":0.17299999999999999},"max":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"min":{"1min":0.16600000000000001,"5min":0.16600000000000001,"15min":0.16600000000000001},"max":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"last":0.35299999999999998}]},{"osd":2,"last update":"Wed Apr 17 22:29:55 2024","interfaces":[{"interface":"back","average":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"min":{"1min":0.20300000000000001,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"min":{"1min":0.20799999999999999,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"last":0.371}]},{"osd":3,"last update":"Wed Apr 17 22:29:55 2024","interfaces":[{"interface":"back","average":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.505,"5min":0.505,"15min":0.505},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"min":{"1min":0.20300000000000001,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"last":0.38300000000000001}]}]},{"osd":1,"up_from":9,"seq":38654705691,"num_pgs":63,"num_osds":1,"num_per_pool_osds":1,"kb":94371840,"kb_used":1048740,"kb_used_data":156,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323100,"statfs":{"total":96636764160,"available":95562854400,"internally_reserved":1073741824,"allocated":159744,"data_stored":43322,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 17 22:29:58 2024","interfaces":[{"interface":"back","average":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"min":{"1min":0.16200000000000001,"5min":0.16200000000000001,"15min":0.16200000000000001},"max":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"last":0.27400000000000002},{"interface":"front","average":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"min":{"1min":0.191,"5min":0.191,"15min":0.191},"max":{"1min":0.376,"5min":0.376,"15min":0.376},"last":0.20899999999999999}]},{"osd":2,"last update":"Wed Apr 17 22:29:58 2024","interfaces":[{"interface":"back","average":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"min":{"1min":0.20499999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"last":0.216},{"interface":"front","average":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"min":{"1min":0.19800000000000001,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"last":0.25700000000000001}]},{"osd":3,"last update":"Wed Apr 17 22:29:58 2024","interfaces":[{"interface":"back","average":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"min":{"1min":0.218,"5min":0.218,"15min":0.218},"max":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"last":0.23699999999999999},{"interface":"front","average":{"1min":0.317,"5min":0.317,"15min":0.317},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"last":0.245}]}]}]}} 2024-04-17T22:30:57.292 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-17T22:30:57.292 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-17T22:30:57.292 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-04-17T22:30:57.292 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-17T22:30:57.454 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.466 7f8957ab6700 1 Processor -- start 2024-04-17T22:30:57.454 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.466 7f8957ab6700 1 -- start start 2024-04-17T22:30:57.454 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.466 7f8957ab6700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f895010b830 0x7f89501a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:30:57.454 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.466 7f8957ab6700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f895010bc80 0x7f89501a7920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:30:57.454 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.466 7f8957ab6700 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8950103b60 con 0x7f89500f5860 2024-04-17T22:30:57.454 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.466 7f8957ab6700 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89500fa390 con 0x7f895010b830 2024-04-17T22:30:57.454 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.466 7f8957ab6700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89501a0b20 con 0x7f895010bc80 2024-04-17T22:30:57.454 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.466 7f89550b2700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f895010bc80 0x7f89501a7920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:30:57.455 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.466 7f89548b1700 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f89500f5860 0x7f895010b030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:40950/0 (socket says 172.21.15.102:40950) 2024-04-17T22:30:57.455 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.466 7f894ffff700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f895010b830 0x7f89501a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:30:57.455 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.466 7f89548b1700 1 -- 172.21.15.102:0/3076163806 learned_addr learned my addr 172.21.15.102:0/3076163806 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-17T22:30:57.455 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.467 7f894ffff700 1 -- 172.21.15.102:0/3076163806 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f895010bc80 msgr2=0x7f89501a7920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:30:57.455 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.467 7f894ffff700 1 --2- 172.21.15.102:0/3076163806 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f895010bc80 0x7f89501a7920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:30:57.455 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.467 7f894ffff700 1 -- 172.21.15.102:0/3076163806 >> v1:172.21.15.102:6789/0 conn(0x7f89500f5860 legacy=0x7f895010b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:30:57.455 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.467 7f894ffff700 1 -- 172.21.15.102:0/3076163806 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89501a8f60 con 0x7f895010b830 2024-04-17T22:30:57.455 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.467 7f89550b2700 1 --2- 172.21.15.102:0/3076163806 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f895010bc80 0x7f89501a7920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:30:57.455 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.467 7f894ffff700 1 --2- 172.21.15.102:0/3076163806 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f895010b830 0x7f89501a52e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 rx=0x7f893c002f70 tx=0x7f893c00b9f0).ready entity=mon.0 client_cookie=32f30cc019c8a70f server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:30:57.455 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.467 7f894f7fe700 1 -- 172.21.15.102:0/3076163806 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f89501a8f60 con 0x7f895010b830 2024-04-17T22:30:57.455 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.467 7f894f7fe700 1 -- 172.21.15.102:0/3076163806 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f893c00d5c0 con 0x7f895010b830 2024-04-17T22:30:57.456 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.467 7f894f7fe700 1 -- 172.21.15.102:0/3076163806 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f893c00d990 con 0x7f895010b830 2024-04-17T22:30:57.456 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.467 7f8957ab6700 1 -- 172.21.15.102:0/3076163806 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f895010b830 msgr2=0x7f89501a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:30:57.456 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.467 7f8957ab6700 1 --2- 172.21.15.102:0/3076163806 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f895010b830 0x7f89501a52e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 rx=0x7f893c002f70 tx=0x7f893c00b9f0).stop 2024-04-17T22:30:57.456 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.467 7f8957ab6700 1 -- 172.21.15.102:0/3076163806 shutdown_connections 2024-04-17T22:30:57.456 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.467 7f8957ab6700 1 --2- 172.21.15.102:0/3076163806 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f895010bc80 0x7f89501a7920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:30:57.456 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.467 7f8957ab6700 1 --2- 172.21.15.102:0/3076163806 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f895010b830 0x7f89501a52e0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:30:57.456 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.467 7f8957ab6700 1 -- 172.21.15.102:0/3076163806 >> 172.21.15.102:0/3076163806 conn(0x7f89501138c0 msgr2=0x7f89500f4f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:30:57.456 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.467 7f8957ab6700 1 -- 172.21.15.102:0/3076163806 shutdown_connections 2024-04-17T22:30:57.456 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.467 7f8957ab6700 1 -- 172.21.15.102:0/3076163806 wait complete. 2024-04-17T22:30:57.476 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.490 7f8957ab6700 1 Processor -- start 2024-04-17T22:30:57.476 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.490 7f8957ab6700 1 -- start start 2024-04-17T22:30:57.478 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.490 7f8957ab6700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f89500d60a0 0x7f89500d64f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:30:57.478 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.490 7f8957ab6700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f89500d6b20 0x7f89500c7200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:30:57.478 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.490 7f8957ab6700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f89500c7830 0x7f89501b85e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:30:57.478 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.490 7f8957ab6700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89500c4580 con 0x7f89500d60a0 2024-04-17T22:30:57.479 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.490 7f8957ab6700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89500da9d0 con 0x7f89500d6b20 2024-04-17T22:30:57.479 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.490 7f8957ab6700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f89500cf750 con 0x7f89500c7830 2024-04-17T22:30:57.479 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.491 7f89548b1700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f89500d60a0 0x7f89500d64f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:30:57.479 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.491 7f894ffff700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f89500d6b20 0x7f89500c7200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:30:57.479 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.491 7f89550b2700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f89500c7830 0x7f89501b85e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:30:57.479 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.491 7f89548b1700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f89500d60a0 0x7f89500d64f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:40392/0 (socket says 172.21.15.102:40392) 2024-04-17T22:30:57.479 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.491 7f89548b1700 1 -- 172.21.15.102:0/1005220855 learned_addr learned my addr 172.21.15.102:0/1005220855 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:30:57.479 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.491 7f894ffff700 1 -- 172.21.15.102:0/1005220855 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f89500c7830 msgr2=0x7f89501b85e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:30:57.479 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.491 7f894ffff700 1 --2- 172.21.15.102:0/1005220855 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f89500c7830 0x7f89501b85e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:30:57.479 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.491 7f894ffff700 1 -- 172.21.15.102:0/1005220855 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f89500d60a0 msgr2=0x7f89500d64f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:30:57.479 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.491 7f894ffff700 1 --2- 172.21.15.102:0/1005220855 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f89500d60a0 0x7f89500d64f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:30:57.479 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.491 7f894ffff700 1 -- 172.21.15.102:0/1005220855 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89500da5b0 con 0x7f89500d6b20 2024-04-17T22:30:57.479 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.491 7f89548b1700 1 --2- 172.21.15.102:0/1005220855 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f89500d60a0 0x7f89500d64f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:30:57.480 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.491 7f894ffff700 1 --2- 172.21.15.102:0/1005220855 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f89500d6b20 0x7f89500c7200 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 rx=0x7f893c002f70 tx=0x7f893c0068c0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:30:57.480 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.491 7f894dffb700 1 -- 172.21.15.102:0/1005220855 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f89500da5b0 con 0x7f89500d6b20 2024-04-17T22:30:57.480 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.491 7f894dffb700 1 -- 172.21.15.102:0/1005220855 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f893c011a00 con 0x7f89500d6b20 2024-04-17T22:30:57.480 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.491 7f894dffb700 1 -- 172.21.15.102:0/1005220855 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f893c017040 con 0x7f89500d6b20 2024-04-17T22:30:57.480 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.491 7f8957ab6700 1 -- 172.21.15.102:0/1005220855 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89500c7c80 con 0x7f89500d6b20 2024-04-17T22:30:57.480 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.491 7f8957ab6700 1 -- 172.21.15.102:0/1005220855 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89501b8c10 con 0x7f89500d6b20 2024-04-17T22:30:57.480 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.492 7f894dffb700 1 -- 172.21.15.102:0/1005220855 <== mon.1 v2:172.21.15.102:3301/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7f893c0263f0 con 0x7f89500d6b20 2024-04-17T22:30:57.480 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.492 7f894dffb700 1 -- 172.21.15.102:0/1005220855 <== mon.1 v2:172.21.15.102:3301/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3194+0+0 (secure 0 0 0) 0x7f893c018070 con 0x7f89500d6b20 2024-04-17T22:30:57.480 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.492 7f8957ab6700 1 -- 172.21.15.102:0/1005220855 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89500f0820 con 0x7f89500d6b20 2024-04-17T22:30:57.481 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.495 7f894dffb700 1 -- 172.21.15.102:0/1005220855 <== mon.1 v2:172.21.15.102:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+110455 (secure 0 0 0) 0x7f893c0117f0 con 0x7f89500d6b20 2024-04-17T22:30:57.619 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.632 7f8957ab6700 1 -- 172.21.15.102:0/1005220855 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f89500dace0 con 0x7f89500d6b20 2024-04-17T22:30:57.619 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.633 7f894dffb700 1 -- 172.21.15.102:0/1005220855 <== mon.1 v2:172.21.15.102:3301/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+34 (secure 0 0 0) 0x7f893c00d3b0 con 0x7f89500d6b20 2024-04-17T22:30:57.619 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-17T22:30:57.621 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.634 7f8957ab6700 1 -- 172.21.15.102:0/1005220855 >> v1:172.21.15.102:6816/42360 conn(0x7f89380145a0 legacy=0x7f8938016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:30:57.621 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.634 7f8957ab6700 1 -- 172.21.15.102:0/1005220855 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f89500d6b20 msgr2=0x7f89500c7200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:30:57.621 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.634 7f8957ab6700 1 --2- 172.21.15.102:0/1005220855 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f89500d6b20 0x7f89500c7200 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 rx=0x7f893c002f70 tx=0x7f893c0068c0).stop 2024-04-17T22:30:57.621 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.634 7f8957ab6700 1 -- 172.21.15.102:0/1005220855 shutdown_connections 2024-04-17T22:30:57.621 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.634 7f8957ab6700 1 --2- 172.21.15.102:0/1005220855 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f89500c7830 0x7f89501b85e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:30:57.621 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.634 7f8957ab6700 1 --2- 172.21.15.102:0/1005220855 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f89500d60a0 0x7f89500d64f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:30:57.621 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.634 7f8957ab6700 1 --2- 172.21.15.102:0/1005220855 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f89500d6b20 0x7f89500c7200 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:30:57.621 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.634 7f8957ab6700 1 -- 172.21.15.102:0/1005220855 >> 172.21.15.102:0/1005220855 conn(0x7f89500c6a60 msgr2=0x7f89500c3050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:30:57.621 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.634 7f8957ab6700 1 -- 172.21.15.102:0/1005220855 shutdown_connections 2024-04-17T22:30:57.622 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.634 7f8957ab6700 1 -- 172.21.15.102:0/1005220855 wait complete. 2024-04-17T22:30:57.631 INFO:teuthology.orchestra.run.smithi102.stdout:{"checks":{},"status":"HEALTH_OK"} 2024-04-17T22:30:57.631 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-04-17T22:30:57.631 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T22:30:57.755 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.769 7f8205f9f700 1 Processor -- start 2024-04-17T22:30:57.755 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.769 7f8205f9f700 1 -- start start 2024-04-17T22:30:57.758 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.769 7f8205f9f700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f82000f57d0 0x7f820010afa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:30:57.758 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.769 7f8205f9f700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f820010b5d0 0x7f820019fa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:30:57.758 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.770 7f8205f9f700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f820010ba20 0x7f82001a00b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:30:57.758 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.770 7f8205f9f700 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f820010be70 con 0x7f820010b5d0 2024-04-17T22:30:57.758 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.770 7f8205f9f700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82000fa300 con 0x7f82000f57d0 2024-04-17T22:30:57.758 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.770 7f8205f9f700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8200103ad0 con 0x7f820010ba20 2024-04-17T22:30:57.758 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.770 7f81fed72700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f82000f57d0 0x7f820010afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:30:57.758 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.770 7f81fe571700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f820010b5d0 0x7f820019fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:30:57.758 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.770 7f81ff573700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f820010ba20 0x7f82001a00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:30:57.759 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.770 7f81fe571700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f820010b5d0 0x7f820019fa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:40406/0 (socket says 172.21.15.102:40406) 2024-04-17T22:30:57.759 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.770 7f81fed72700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f82000f57d0 0x7f820010afa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:48802/0 (socket says 172.21.15.102:48802) 2024-04-17T22:30:57.759 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.770 7f81fe571700 1 -- 172.21.15.102:0/490813379 learned_addr learned my addr 172.21.15.102:0/490813379 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:30:57.759 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.770 7f81ff573700 1 -- 172.21.15.102:0/490813379 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f82000f57d0 msgr2=0x7f820010afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:30:57.759 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.770 7f81ff573700 1 --2- 172.21.15.102:0/490813379 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f82000f57d0 0x7f820010afa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:30:57.759 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.770 7f81ff573700 1 -- 172.21.15.102:0/490813379 >> v2:172.21.15.102:3300/0 conn(0x7f820010b5d0 msgr2=0x7f820019fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:30:57.759 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.770 7f81ff573700 1 --2- 172.21.15.102:0/490813379 >> v2:172.21.15.102:3300/0 conn(0x7f820010b5d0 0x7f820019fa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:30:57.759 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.770 7f81ff573700 1 -- 172.21.15.102:0/490813379 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82001a06e0 con 0x7f820010ba20 2024-04-17T22:30:57.759 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.770 7f81fe571700 1 --2- 172.21.15.102:0/490813379 >> v2:172.21.15.102:3300/0 conn(0x7f820010b5d0 0x7f820019fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:30:57.759 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.770 7f81ff573700 1 --2- 172.21.15.102:0/490813379 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f820010ba20 0x7f82001a00b0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0x7f82000f4630 tx=0x7f81f8009cf0).ready entity=mon.2 client_cookie=a0156a8d3bb9a527 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:30:57.759 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.770 7f81fdd70700 1 -- 172.21.15.102:0/490813379 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f82001a06e0 con 0x7f820010ba20 2024-04-17T22:30:57.759 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.770 7f81fdd70700 1 -- 172.21.15.102:0/490813379 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81f800b800 con 0x7f820010ba20 2024-04-17T22:30:57.759 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.770 7f81fdd70700 1 -- 172.21.15.102:0/490813379 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f81f800b9e0 con 0x7f820010ba20 2024-04-17T22:30:57.760 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.770 7f8205f9f700 1 -- 172.21.15.102:0/490813379 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f820010ba20 msgr2=0x7f82001a00b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:30:57.760 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.770 7f8205f9f700 1 --2- 172.21.15.102:0/490813379 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f820010ba20 0x7f82001a00b0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0x7f82000f4630 tx=0x7f81f8009cf0).stop 2024-04-17T22:30:57.760 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.771 7f8205f9f700 1 -- 172.21.15.102:0/490813379 shutdown_connections 2024-04-17T22:30:57.760 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.771 7f8205f9f700 1 --2- 172.21.15.102:0/490813379 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f820010ba20 0x7f82001a00b0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:30:57.760 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.771 7f8205f9f700 1 --2- 172.21.15.102:0/490813379 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f82000f57d0 0x7f820010afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:30:57.760 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.771 7f8205f9f700 1 --2- 172.21.15.102:0/490813379 >> v2:172.21.15.102:3300/0 conn(0x7f820010b5d0 0x7f820019fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:30:57.760 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.771 7f8205f9f700 1 -- 172.21.15.102:0/490813379 >> 172.21.15.102:0/490813379 conn(0x7f8200113830 msgr2=0x7f82000f4ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:30:57.760 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.771 7f8205f9f700 1 -- 172.21.15.102:0/490813379 shutdown_connections 2024-04-17T22:30:57.760 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.771 7f8205f9f700 1 -- 172.21.15.102:0/490813379 wait complete. 2024-04-17T22:30:57.781 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.793 7f8205f9f700 1 Processor -- start 2024-04-17T22:30:57.781 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.793 7f8205f9f700 1 -- start start 2024-04-17T22:30:57.781 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.793 7f8205f9f700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f82001ab060 0x7f82001ab4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:30:57.781 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.793 7f8205f9f700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f82001abae0 0x7f82001b1930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:30:57.781 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.793 7f8205f9f700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f82001b1f60 0x7f82001b23b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:30:57.781 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.793 7f8205f9f700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8200089940 con 0x7f82001b1f60 2024-04-17T22:30:57.781 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.793 7f8205f9f700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82000c74a0 con 0x7f82001abae0 2024-04-17T22:30:57.781 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.793 7f8205f9f700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f82000d5c80 con 0x7f82001ab060 2024-04-17T22:30:57.781 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.794 7f81fe571700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f82001abae0 0x7f82001b1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:30:57.782 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.794 7f81fed72700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f82001ab060 0x7f82001ab4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:30:57.782 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.794 7f81ff573700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f82001b1f60 0x7f82001b23b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:30:57.782 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.794 7f81fe571700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f82001abae0 0x7f82001b1930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:48818/0 (socket says 172.21.15.102:48818) 2024-04-17T22:30:57.782 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.794 7f81fed72700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f82001ab060 0x7f82001ab4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:59202/0 (socket says 172.21.15.102:59202) 2024-04-17T22:30:57.782 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.794 7f81fe571700 1 -- 172.21.15.102:0/1334280203 learned_addr learned my addr 172.21.15.102:0/1334280203 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:30:57.782 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.794 7f81fed72700 1 -- 172.21.15.102:0/1334280203 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f82001abae0 msgr2=0x7f82001b1930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:30:57.782 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.794 7f81fed72700 1 --2- 172.21.15.102:0/1334280203 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f82001abae0 0x7f82001b1930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:30:57.782 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.794 7f81fed72700 1 -- 172.21.15.102:0/1334280203 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f82001b1f60 msgr2=0x7f82001b23b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:30:57.782 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.794 7f81fed72700 1 --2- 172.21.15.102:0/1334280203 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f82001b1f60 0x7f82001b23b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:30:57.782 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.794 7f81fed72700 1 -- 172.21.15.102:0/1334280203 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82000c7150 con 0x7f82001ab060 2024-04-17T22:30:57.782 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.794 7f81fe571700 1 --2- 172.21.15.102:0/1334280203 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f82001abae0 0x7f82001b1930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:30:57.782 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.794 7f81ff573700 1 --2- 172.21.15.102:0/1334280203 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f82001b1f60 0x7f82001b23b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:30:57.782 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.794 7f81fed72700 1 --2- 172.21.15.102:0/1334280203 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f82001ab060 0x7f82001ab4b0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0x7f82001a6b40 tx=0x7f81f400ec80).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:30:57.782 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.794 7f81effff700 1 -- 172.21.15.102:0/1334280203 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f82000c7150 con 0x7f82001ab060 2024-04-17T22:30:57.783 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.794 7f81effff700 1 -- 172.21.15.102:0/1334280203 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81f400c9c0 con 0x7f82001ab060 2024-04-17T22:30:57.783 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.794 7f81effff700 1 -- 172.21.15.102:0/1334280203 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f81f400f040 con 0x7f82001ab060 2024-04-17T22:30:57.783 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.794 7f8205f9f700 1 -- 172.21.15.102:0/1334280203 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82000c6d70 con 0x7f82001ab060 2024-04-17T22:30:57.783 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.794 7f8205f9f700 1 -- 172.21.15.102:0/1334280203 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82000c2b80 con 0x7f82001ab060 2024-04-17T22:30:57.783 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.795 7f81effff700 1 -- 172.21.15.102:0/1334280203 <== mon.2 v2:172.21.15.102:3302/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7f81f4011070 con 0x7f82001ab060 2024-04-17T22:30:57.783 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.795 7f81effff700 1 -- 172.21.15.102:0/1334280203 <== mon.2 v2:172.21.15.102:3302/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3194+0+0 (secure 0 0 0) 0x7f81f4010040 con 0x7f82001ab060 2024-04-17T22:30:57.783 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.795 7f8205f9f700 1 -- 172.21.15.102:0/1334280203 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82000f0790 con 0x7f82001ab060 2024-04-17T22:30:57.784 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.798 7f81effff700 1 -- 172.21.15.102:0/1334280203 <== mon.2 v2:172.21.15.102:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+110455 (secure 0 0 0) 0x7f81f400c5c0 con 0x7f82001ab060 2024-04-17T22:30:57.890 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.904 7f8205f9f700 1 -- 172.21.15.102:0/1334280203 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f82001ac090 con 0x7f82001ab060 2024-04-17T22:30:57.891 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.905 7f81effff700 1 -- 172.21.15.102:0/1334280203 <== mon.2 v2:172.21.15.102:3302/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) v1 ==== 74+0+8764 (secure 0 0 0) 0x7f81f4000c30 con 0x7f82001ab060 2024-04-17T22:30:57.891 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-17T22:30:57.891 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":17,"fsid":"cb63e789-091d-49b1-b120-55f9db526871","created":"2024-04-17 22:28:48.209635","modified":"2024-04-17 22:29:23.257387","last_up_change":"2024-04-17 22:28:54.236215","last_in_change":"2024-04-17 22:28:54.236215","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":4,"require_min_compat_client":"nautilus","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-04-17 22:28:57.478545","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-17 22:29:06.144708","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-17 22:29:06.748924","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3351ef85-ade4-4b90-a8f4-213f4e00ba01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6812","nonce":42955}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6813","nonce":42955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6815","nonce":42955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6814","nonce":42955}]},"public_addr":"172.21.15.102:6812/42955","cluster_addr":"172.21.15.102:6813/42955","heartbeat_back_addr":"172.21.15.102:6815/42955","heartbeat_front_addr":"172.21.15.102:6814/42955","state":["exists","up"]},{"osd":1,"uuid":"04f67eac-f36b-48f5-982d-4cfc776596d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6808","nonce":42958}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6809","nonce":42958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6811","nonce":42958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6810","nonce":42958}]},"public_addr":"172.21.15.102:6808/42958","cluster_addr":"172.21.15.102:6809/42958","heartbeat_back_addr":"172.21.15.102:6811/42958","heartbeat_front_addr":"172.21.15.102:6810/42958","state":["exists","up"]},{"osd":2,"uuid":"d524204b-70b2-40a8-8f73-f9e7d44318af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6804","nonce":42954}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6805","nonce":42954}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6807","nonce":42954}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6806","nonce":42954}]},"public_addr":"172.21.15.102:6804/42954","cluster_addr":"172.21.15.102:6805/42954","heartbeat_back_addr":"172.21.15.102:6807/42954","heartbeat_front_addr":"172.21.15.102:6806/42954","state":["exists","up"]},{"osd":3,"uuid":"4fc9d20c-1ad0-484f-aafa-2c9a7574245b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6800","nonce":42953}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6801","nonce":42953}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6803","nonce":42953}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6802","nonce":42953}]},"public_addr":"172.21.15.102:6800/42953","cluster_addr":"172.21.15.102:6801/42953","heartbeat_back_addr":"172.21.15.102:6803/42953","heartbeat_front_addr":"172.21.15.102:6802/42953","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2024-04-17T22:30:57.892 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.906 7f8205f9f700 1 -- 172.21.15.102:0/1334280203 >> v1:172.21.15.102:6816/42360 conn(0x7f81dc0145a0 legacy=0x7f81dc016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:30:57.892 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.906 7f8205f9f700 1 -- 172.21.15.102:0/1334280203 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f82001ab060 msgr2=0x7f82001ab4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:30:57.892 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.906 7f8205f9f700 1 --2- 172.21.15.102:0/1334280203 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f82001ab060 0x7f82001ab4b0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0x7f82001a6b40 tx=0x7f81f400ec80).stop 2024-04-17T22:30:57.893 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.906 7f8205f9f700 1 -- 172.21.15.102:0/1334280203 shutdown_connections 2024-04-17T22:30:57.893 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.906 7f8205f9f700 1 --2- 172.21.15.102:0/1334280203 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f82001b1f60 0x7f82001b23b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:30:57.893 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.906 7f8205f9f700 1 --2- 172.21.15.102:0/1334280203 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f82001ab060 0x7f82001ab4b0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:30:57.893 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.906 7f8205f9f700 1 --2- 172.21.15.102:0/1334280203 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f82001abae0 0x7f82001b1930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:30:57.893 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.906 7f8205f9f700 1 -- 172.21.15.102:0/1334280203 >> 172.21.15.102:0/1334280203 conn(0x7f82000f17f0 msgr2=0x7f82000c9010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:30:57.893 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.906 7f8205f9f700 1 -- 172.21.15.102:0/1334280203 shutdown_connections 2024-04-17T22:30:57.893 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:57.906 7f8205f9f700 1 -- 172.21.15.102:0/1334280203 wait complete. 2024-04-17T22:30:57.903 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2024-04-17 22:28:57.478545', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '12', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-17 22:29:06.144708', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-17 22:29:06.748924', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-17T22:30:57.903 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2024-04-17T22:30:58.034 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.044 7f2a38210700 1 Processor -- start 2024-04-17T22:30:58.034 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.045 7f2a38210700 1 -- start start 2024-04-17T22:30:58.034 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.045 7f2a38210700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2a300f5750 0x7f2a3010c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:30:58.034 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.045 7f2a38210700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f2a3010cb70 0x7f2a301a13e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:30:58.035 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.045 7f2a38210700 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a301a1a10 con 0x7f2a3010c720 2024-04-17T22:30:58.035 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.045 7f2a38210700 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a300f93c0 con 0x7f2a3010cb70 2024-04-17T22:30:58.035 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.045 7f2a38210700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a301a1c30 con 0x7f2a300f5750 2024-04-17T22:30:58.035 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.045 7f2a3580c700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f2a3010cb70 0x7f2a301a13e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:30:58.035 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.045 7f2a3500b700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2a300f5750 0x7f2a3010c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:30:58.035 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.045 7f2a3480a700 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f2a3010c720 0x7f2a301a0be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:40962/0 (socket says 172.21.15.102:40962) 2024-04-17T22:30:58.036 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.045 7f2a3580c700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f2a3010cb70 0x7f2a301a13e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:40430/0 (socket says 172.21.15.102:40430) 2024-04-17T22:30:58.036 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.045 7f2a3480a700 1 -- 172.21.15.102:0/3345934761 learned_addr learned my addr 172.21.15.102:0/3345934761 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-17T22:30:58.036 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.045 7f2a3580c700 1 -- 172.21.15.102:0/3345934761 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2a300f5750 msgr2=0x7f2a3010c0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:30:58.036 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.045 7f2a3580c700 1 --2- 172.21.15.102:0/3345934761 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2a300f5750 0x7f2a3010c0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:30:58.036 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.045 7f2a3580c700 1 -- 172.21.15.102:0/3345934761 >> v1:172.21.15.102:6789/0 conn(0x7f2a3010c720 legacy=0x7f2a301a0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:30:58.036 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.045 7f2a3580c700 1 -- 172.21.15.102:0/3345934761 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a301aa310 con 0x7f2a3010cb70 2024-04-17T22:30:58.037 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.045 7f2a2ffff700 1 -- 172.21.15.102:0/3345934761 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3995759403 0 0) 0x7f2a24001490 con 0x7f2a3010c720 2024-04-17T22:30:58.037 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.045 7f2a3500b700 1 --2- 172.21.15.102:0/3345934761 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2a300f5750 0x7f2a3010c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:30:58.037 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.046 7f2a3580c700 1 --2- 172.21.15.102:0/3345934761 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2a3010cb70 0x7f2a301a13e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 rx=0x7f2a300f9570 tx=0x7f2a2000ba20).ready entity=mon.0 client_cookie=dce2e7cef63961e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:30:58.037 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.046 7f2a2ffff700 1 -- 172.21.15.102:0/3345934761 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f2a301aa310 con 0x7f2a3010cb70 2024-04-17T22:30:58.037 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.046 7f2a2ffff700 1 -- 172.21.15.102:0/3345934761 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a2000d5c0 con 0x7f2a3010cb70 2024-04-17T22:30:58.037 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.046 7f2a2ffff700 1 -- 172.21.15.102:0/3345934761 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f2a2000d9e0 con 0x7f2a3010cb70 2024-04-17T22:30:58.038 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.046 7f2a38210700 1 -- 172.21.15.102:0/3345934761 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2a3010cb70 msgr2=0x7f2a301a13e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:30:58.038 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.046 7f2a38210700 1 --2- 172.21.15.102:0/3345934761 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2a3010cb70 0x7f2a301a13e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 rx=0x7f2a300f9570 tx=0x7f2a2000ba20).stop 2024-04-17T22:30:58.038 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.046 7f2a38210700 1 -- 172.21.15.102:0/3345934761 shutdown_connections 2024-04-17T22:30:58.038 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.046 7f2a38210700 1 --2- 172.21.15.102:0/3345934761 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2a300f5750 0x7f2a3010c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:30:58.038 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.046 7f2a38210700 1 --2- 172.21.15.102:0/3345934761 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2a3010cb70 0x7f2a301a13e0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:30:58.038 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.046 7f2a38210700 1 -- 172.21.15.102:0/3345934761 >> 172.21.15.102:0/3345934761 conn(0x7f2a301138a0 msgr2=0x7f2a300f4f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:30:58.039 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.046 7f2a38210700 1 -- 172.21.15.102:0/3345934761 shutdown_connections 2024-04-17T22:30:58.039 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.046 7f2a38210700 1 -- 172.21.15.102:0/3345934761 wait complete. 2024-04-17T22:30:58.058 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.069 7f2a38210700 1 Processor -- start 2024-04-17T22:30:58.059 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.069 7f2a38210700 1 -- start start 2024-04-17T22:30:58.059 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.069 7f2a38210700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2a300cf480 0x7f2a300d5de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:30:58.059 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.069 7f2a38210700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2a300d6410 0x7f2a301b1b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:30:58.059 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.069 7f2a38210700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f2a300d6860 0x7f2a301b21b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:30:58.059 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.069 7f2a38210700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a300e2b10 con 0x7f2a300cf480 2024-04-17T22:30:58.059 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.069 7f2a38210700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a301a6a90 con 0x7f2a300d6410 2024-04-17T22:30:58.060 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.069 7f2a38210700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2a300cf8d0 con 0x7f2a300d6860 2024-04-17T22:30:58.060 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.069 7f2a3500b700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2a300cf480 0x7f2a300d5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:30:58.060 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.069 7f2a3480a700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2a300d6410 0x7f2a301b1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:30:58.060 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.069 7f2a3580c700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f2a300d6860 0x7f2a301b21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:30:58.060 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.069 7f2a3500b700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2a300cf480 0x7f2a300d5de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:40432/0 (socket says 172.21.15.102:40432) 2024-04-17T22:30:58.060 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.069 7f2a3480a700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2a300d6410 0x7f2a301b1b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:48850/0 (socket says 172.21.15.102:48850) 2024-04-17T22:30:58.060 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.069 7f2a3500b700 1 -- 172.21.15.102:0/3998937814 learned_addr learned my addr 172.21.15.102:0/3998937814 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:30:58.060 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.070 7f2a3580c700 1 -- 172.21.15.102:0/3998937814 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2a300d6410 msgr2=0x7f2a301b1b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:30:58.060 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.070 7f2a3580c700 1 --2- 172.21.15.102:0/3998937814 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2a300d6410 0x7f2a301b1b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:30:58.060 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.070 7f2a3580c700 1 -- 172.21.15.102:0/3998937814 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2a300cf480 msgr2=0x7f2a300d5de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:30:58.060 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.070 7f2a3580c700 1 --2- 172.21.15.102:0/3998937814 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2a300cf480 0x7f2a300d5de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:30:58.060 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.070 7f2a3580c700 1 -- 172.21.15.102:0/3998937814 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a300ccea0 con 0x7f2a300d6860 2024-04-17T22:30:58.060 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.070 7f2a3500b700 1 --2- 172.21.15.102:0/3998937814 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2a300cf480 0x7f2a300d5de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:30:58.060 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.070 7f2a3580c700 1 --2- 172.21.15.102:0/3998937814 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f2a300d6860 0x7f2a301b21b0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0x7f2a300cd890 tx=0x7f2a200068c0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:30:58.061 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.070 7f2a2e7fc700 1 -- 172.21.15.102:0/3998937814 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f2a300ccea0 con 0x7f2a300d6860 2024-04-17T22:30:58.061 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.070 7f2a2e7fc700 1 -- 172.21.15.102:0/3998937814 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a20011940 con 0x7f2a300d6860 2024-04-17T22:30:58.061 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.070 7f2a38210700 1 -- 172.21.15.102:0/3998937814 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a301b27e0 con 0x7f2a300d6860 2024-04-17T22:30:58.061 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.070 7f2a2e7fc700 1 -- 172.21.15.102:0/3998937814 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f2a20017040 con 0x7f2a300d6860 2024-04-17T22:30:58.061 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.070 7f2a38210700 1 -- 172.21.15.102:0/3998937814 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a301b92f0 con 0x7f2a300d6860 2024-04-17T22:30:58.061 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.071 7f2a2e7fc700 1 -- 172.21.15.102:0/3998937814 <== mon.2 v2:172.21.15.102:3302/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7f2a20011d10 con 0x7f2a300d6860 2024-04-17T22:30:58.061 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.071 7f2a2e7fc700 1 -- 172.21.15.102:0/3998937814 <== mon.2 v2:172.21.15.102:3302/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3194+0+0 (secure 0 0 0) 0x7f2a20018070 con 0x7f2a300d6860 2024-04-17T22:30:58.061 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.071 7f2a38210700 1 -- 172.21.15.102:0/3998937814 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a300f0800 con 0x7f2a300d6860 2024-04-17T22:30:58.061 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.074 7f2a2e7fc700 1 -- 172.21.15.102:0/3998937814 <== mon.2 v2:172.21.15.102:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+110455 (secure 0 0 0) 0x7f2a20011730 con 0x7f2a300d6860 2024-04-17T22:30:58.165 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.179 7f2a38210700 1 -- 172.21.15.102:0/3998937814 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f2a301ac2e0 con 0x7f2a300d6860 2024-04-17T22:30:58.166 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.180 7f2a2e7fc700 1 -- 172.21.15.102:0/3998937814 <== mon.2 v2:172.21.15.102:3302/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v17) v1 ==== 92+0+10 (secure 0 0 0) 0x7f2a2000d3b0 con 0x7f2a300d6860 2024-04-17T22:30:58.166 INFO:teuthology.orchestra.run.smithi102.stdout:pg_num: 8 2024-04-17T22:30:58.168 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.181 7f2a38210700 1 -- 172.21.15.102:0/3998937814 >> v1:172.21.15.102:6816/42360 conn(0x7f2a100145a0 legacy=0x7f2a10016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:30:58.168 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.181 7f2a38210700 1 -- 172.21.15.102:0/3998937814 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f2a300d6860 msgr2=0x7f2a301b21b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:30:58.168 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.181 7f2a38210700 1 --2- 172.21.15.102:0/3998937814 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f2a300d6860 0x7f2a301b21b0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0x7f2a300cd890 tx=0x7f2a200068c0).stop 2024-04-17T22:30:58.168 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.181 7f2a38210700 1 -- 172.21.15.102:0/3998937814 shutdown_connections 2024-04-17T22:30:58.168 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.181 7f2a38210700 1 --2- 172.21.15.102:0/3998937814 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f2a300d6860 0x7f2a301b21b0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:30:58.168 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.181 7f2a38210700 1 --2- 172.21.15.102:0/3998937814 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2a300cf480 0x7f2a300d5de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:30:58.168 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.181 7f2a38210700 1 --2- 172.21.15.102:0/3998937814 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2a300d6410 0x7f2a301b1b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:30:58.168 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.181 7f2a38210700 1 -- 172.21.15.102:0/3998937814 >> 172.21.15.102:0/3998937814 conn(0x7f2a300e0f20 msgr2=0x7f2a300dac40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:30:58.168 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.181 7f2a38210700 1 -- 172.21.15.102:0/3998937814 shutdown_connections 2024-04-17T22:30:58.168 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.181 7f2a38210700 1 -- 172.21.15.102:0/3998937814 wait complete. 2024-04-17T22:30:58.178 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-17T22:30:58.307 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.318 7f566cbfa700 1 Processor -- start 2024-04-17T22:30:58.307 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.319 7f566cbfa700 1 -- start start 2024-04-17T22:30:58.307 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.319 7f566cbfa700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f56680f5840 0x7f566810b010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:30:58.307 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.319 7f566cbfa700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f566810b640 0x7f566819faf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:30:58.307 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.319 7f566cbfa700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f566810ba90 0x7f56681a0120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:30:58.307 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.319 7f566cbfa700 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f566810bee0 con 0x7f566810ba90 2024-04-17T22:30:58.307 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.319 7f566cbfa700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56680fa370 con 0x7f566810b640 2024-04-17T22:30:58.307 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.319 7f566cbfa700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5668103b40 con 0x7f56680f5840 2024-04-17T22:30:58.307 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.319 7f566504f700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f566810b640 0x7f566819faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:30:58.308 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.319 7f5665850700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f56680f5840 0x7f566810b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:30:58.308 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.319 7f5666051700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f566810ba90 0x7f56681a0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:30:58.308 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.319 7f566504f700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f566810b640 0x7f566819faf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:48864/0 (socket says 172.21.15.102:48864) 2024-04-17T22:30:58.308 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.319 7f566504f700 1 -- 172.21.15.102:0/3277775214 learned_addr learned my addr 172.21.15.102:0/3277775214 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:30:58.308 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.319 7f5666051700 1 -- 172.21.15.102:0/3277775214 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f56680f5840 msgr2=0x7f566810b010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:30:58.308 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.319 7f5666051700 1 --2- 172.21.15.102:0/3277775214 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f56680f5840 0x7f566810b010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:30:58.308 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.319 7f5666051700 1 -- 172.21.15.102:0/3277775214 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f566810b640 msgr2=0x7f566819faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:30:58.308 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.319 7f5666051700 1 --2- 172.21.15.102:0/3277775214 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f566810b640 0x7f566819faf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:30:58.308 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.319 7f5666051700 1 -- 172.21.15.102:0/3277775214 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56681a0750 con 0x7f566810ba90 2024-04-17T22:30:58.308 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.319 7f5665850700 1 --2- 172.21.15.102:0/3277775214 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f56680f5840 0x7f566810b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:30:58.308 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.319 7f566504f700 1 --2- 172.21.15.102:0/3277775214 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f566810b640 0x7f566819faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:30:58.308 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.320 7f5666051700 1 --2- 172.21.15.102:0/3277775214 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f566810ba90 0x7f56681a0120 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 rx=0x7f56680fa520 tx=0x7f5660009d40).ready entity=mon.0 client_cookie=49ca7758ef9a5d04 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:30:58.308 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.320 7f566484e700 1 -- 172.21.15.102:0/3277775214 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f56681a0750 con 0x7f566810ba90 2024-04-17T22:30:58.309 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.320 7f566484e700 1 -- 172.21.15.102:0/3277775214 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f566000b7c0 con 0x7f566810ba90 2024-04-17T22:30:58.309 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.320 7f566484e700 1 -- 172.21.15.102:0/3277775214 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f566000b9a0 con 0x7f566810ba90 2024-04-17T22:30:58.309 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.320 7f566cbfa700 1 -- 172.21.15.102:0/3277775214 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f566810ba90 msgr2=0x7f56681a0120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:30:58.309 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.320 7f566cbfa700 1 --2- 172.21.15.102:0/3277775214 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f566810ba90 0x7f56681a0120 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 rx=0x7f56680fa520 tx=0x7f5660009d40).stop 2024-04-17T22:30:58.309 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.320 7f566cbfa700 1 -- 172.21.15.102:0/3277775214 shutdown_connections 2024-04-17T22:30:58.309 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.320 7f566cbfa700 1 --2- 172.21.15.102:0/3277775214 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f566810ba90 0x7f56681a0120 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:30:58.309 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.320 7f566cbfa700 1 --2- 172.21.15.102:0/3277775214 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f56680f5840 0x7f566810b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:30:58.309 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.320 7f566cbfa700 1 --2- 172.21.15.102:0/3277775214 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f566810b640 0x7f566819faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:30:58.310 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.320 7f566cbfa700 1 -- 172.21.15.102:0/3277775214 >> 172.21.15.102:0/3277775214 conn(0x7f56681138a0 msgr2=0x7f56680f4f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:30:58.310 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.320 7f566cbfa700 1 -- 172.21.15.102:0/3277775214 shutdown_connections 2024-04-17T22:30:58.310 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.320 7f566cbfa700 1 -- 172.21.15.102:0/3277775214 wait complete. 2024-04-17T22:30:58.331 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.343 7f566cbfa700 1 Processor -- start 2024-04-17T22:30:58.331 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.343 7f566cbfa700 1 -- start start 2024-04-17T22:30:58.331 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.343 7f566cbfa700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f56681ad140 0x7f56681ad590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:30:58.331 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.343 7f566cbfa700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f56681adbc0 0x7f56681b3c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:30:58.331 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.343 7f566cbfa700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f56681b4230 0x7f56681b4680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:30:58.331 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.343 7f566cbfa700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56680e2790 con 0x7f56681b4230 2024-04-17T22:30:58.331 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.343 7f566cbfa700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56680c2e00 con 0x7f56681adbc0 2024-04-17T22:30:58.331 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.343 7f566cbfa700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f56681b4cb0 con 0x7f56681ad140 2024-04-17T22:30:58.331 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.344 7f5665850700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f56681ad140 0x7f56681ad590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:30:58.332 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.344 7f566504f700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f56681adbc0 0x7f56681b3c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:30:58.332 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.344 7f5666051700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f56681b4230 0x7f56681b4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:30:58.332 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.344 7f5665850700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f56681ad140 0x7f56681ad590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:59236/0 (socket says 172.21.15.102:59236) 2024-04-17T22:30:58.332 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.344 7f566504f700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f56681adbc0 0x7f56681b3c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:48868/0 (socket says 172.21.15.102:48868) 2024-04-17T22:30:58.332 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.344 7f5666051700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f56681b4230 0x7f56681b4680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:40456/0 (socket says 172.21.15.102:40456) 2024-04-17T22:30:58.332 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.344 7f5665850700 1 -- 172.21.15.102:0/3462863359 learned_addr learned my addr 172.21.15.102:0/3462863359 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:30:58.332 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.344 7f566504f700 1 -- 172.21.15.102:0/3462863359 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f56681ad140 msgr2=0x7f56681ad590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:30:58.332 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.344 7f566504f700 1 --2- 172.21.15.102:0/3462863359 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f56681ad140 0x7f56681ad590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:30:58.332 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.344 7f566504f700 1 -- 172.21.15.102:0/3462863359 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f56681b4230 msgr2=0x7f56681b4680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:30:58.332 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.344 7f566504f700 1 --2- 172.21.15.102:0/3462863359 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f56681b4230 0x7f56681b4680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:30:58.332 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.344 7f566504f700 1 -- 172.21.15.102:0/3462863359 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56680dec50 con 0x7f56681adbc0 2024-04-17T22:30:58.332 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.344 7f5666051700 1 --2- 172.21.15.102:0/3462863359 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f56681b4230 0x7f56681b4680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:30:58.332 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.344 7f5665850700 1 --2- 172.21.15.102:0/3462863359 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f56681ad140 0x7f56681ad590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:30:58.332 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.344 7f566504f700 1 --2- 172.21.15.102:0/3462863359 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f56681adbc0 0x7f56681b3c00 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 rx=0x7f56680e0160 tx=0x7f565800b1e0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:30:58.332 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.344 7f5656ffd700 1 -- 172.21.15.102:0/3462863359 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f56680dec50 con 0x7f56681adbc0 2024-04-17T22:30:58.333 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.344 7f5656ffd700 1 -- 172.21.15.102:0/3462863359 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f565800d9b0 con 0x7f56681adbc0 2024-04-17T22:30:58.333 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.344 7f566cbfa700 1 -- 172.21.15.102:0/3462863359 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56680d9b50 con 0x7f56681adbc0 2024-04-17T22:30:58.333 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.344 7f5656ffd700 1 -- 172.21.15.102:0/3462863359 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f565800dd80 con 0x7f56681adbc0 2024-04-17T22:30:58.333 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.344 7f566cbfa700 1 -- 172.21.15.102:0/3462863359 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56681bb550 con 0x7f56681adbc0 2024-04-17T22:30:58.333 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.345 7f5656ffd700 1 -- 172.21.15.102:0/3462863359 <== mon.1 v2:172.21.15.102:3301/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7f5658020060 con 0x7f56681adbc0 2024-04-17T22:30:58.333 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.345 7f5656ffd700 1 -- 172.21.15.102:0/3462863359 <== mon.1 v2:172.21.15.102:3301/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3194+0+0 (secure 0 0 0) 0x7f5658001bc0 con 0x7f56681adbc0 2024-04-17T22:30:58.333 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.345 7f566cbfa700 1 -- 172.21.15.102:0/3462863359 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56680f0800 con 0x7f56681adbc0 2024-04-17T22:30:58.334 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.348 7f5656ffd700 1 -- 172.21.15.102:0/3462863359 <== mon.1 v2:172.21.15.102:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+110455 (secure 0 0 0) 0x7f565800d7a0 con 0x7f56681adbc0 2024-04-17T22:30:58.440 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.454 7f566cbfa700 1 -- 172.21.15.102:0/3462863359 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f56681ae320 con 0x7f56681adbc0 2024-04-17T22:30:58.440 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.454 7f5656ffd700 1 -- 172.21.15.102:0/3462863359 <== mon.1 v2:172.21.15.102:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v17) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5658000c30 con 0x7f56681adbc0 2024-04-17T22:30:58.440 INFO:teuthology.orchestra.run.smithi102.stdout:pg_num: 64 2024-04-17T22:30:58.442 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.455 7f566cbfa700 1 -- 172.21.15.102:0/3462863359 >> v1:172.21.15.102:6816/42360 conn(0x7f56440145a0 legacy=0x7f5644016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:30:58.442 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.455 7f566cbfa700 1 -- 172.21.15.102:0/3462863359 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f56681adbc0 msgr2=0x7f56681b3c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:30:58.442 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.455 7f566cbfa700 1 --2- 172.21.15.102:0/3462863359 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f56681adbc0 0x7f56681b3c00 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 rx=0x7f56680e0160 tx=0x7f565800b1e0).stop 2024-04-17T22:30:58.442 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.455 7f566cbfa700 1 -- 172.21.15.102:0/3462863359 shutdown_connections 2024-04-17T22:30:58.442 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.455 7f566cbfa700 1 --2- 172.21.15.102:0/3462863359 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f56681b4230 0x7f56681b4680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:30:58.442 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.455 7f566cbfa700 1 --2- 172.21.15.102:0/3462863359 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f56681ad140 0x7f56681ad590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:30:58.442 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.455 7f566cbfa700 1 --2- 172.21.15.102:0/3462863359 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f56681adbc0 0x7f56681b3c00 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:30:58.442 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.455 7f566cbfa700 1 -- 172.21.15.102:0/3462863359 >> 172.21.15.102:0/3462863359 conn(0x7f56680c5840 msgr2=0x7f56680df070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:30:58.442 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.455 7f566cbfa700 1 -- 172.21.15.102:0/3462863359 shutdown_connections 2024-04-17T22:30:58.442 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.455 7f566cbfa700 1 -- 172.21.15.102:0/3462863359 wait complete. 2024-04-17T22:30:58.452 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-17T22:30:58.581 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.593 7f292ee06700 1 Processor -- start 2024-04-17T22:30:58.581 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.593 7f292ee06700 1 -- start start 2024-04-17T22:30:58.581 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.593 7f292ee06700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f29280f5750 0x7f292810c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:30:58.581 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.593 7f292ee06700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f292810c720 0x7f29281a0be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:30:58.581 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.593 7f292ee06700 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29281a1a10 con 0x7f292810cb70 2024-04-17T22:30:58.581 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.593 7f292ee06700 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29280f93c0 con 0x7f292810c720 2024-04-17T22:30:58.581 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.593 7f292ee06700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f29281a1c30 con 0x7f29280f5750 2024-04-17T22:30:58.581 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.593 7f2926ffd700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f292810c720 0x7f29281a0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:30:58.581 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.593 7f29277fe700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f29280f5750 0x7f292810c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:30:58.581 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.593 7f29277fe700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f29280f5750 0x7f292810c0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:59238/0 (socket says 172.21.15.102:59238) 2024-04-17T22:30:58.581 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.593 7f2926ffd700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f292810c720 0x7f29281a0be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:40458/0 (socket says 172.21.15.102:40458) 2024-04-17T22:30:58.582 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.593 7f29277fe700 1 -- 172.21.15.102:0/1826893814 learned_addr learned my addr 172.21.15.102:0/1826893814 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:30:58.582 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.593 7f2927fff700 1 --1- 172.21.15.102:0/1826893814 >> v1:172.21.15.102:6789/0 conn(0x7f292810cb70 0x7f29281a1210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:40966/0 (socket says 172.21.15.102:40966) 2024-04-17T22:30:58.582 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.594 7f2926ffd700 1 -- 172.21.15.102:0/1826893814 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f29280f5750 msgr2=0x7f292810c0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:30:58.582 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.594 7f2926ffd700 1 --2- 172.21.15.102:0/1826893814 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f29280f5750 0x7f292810c0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:30:58.582 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.594 7f2926ffd700 1 -- 172.21.15.102:0/1826893814 >> v1:172.21.15.102:6789/0 conn(0x7f292810cb70 legacy=0x7f29281a1210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:30:58.582 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.594 7f29267fc700 1 -- 172.21.15.102:0/1826893814 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2957378644 0 0) 0x7f2920001490 con 0x7f292810cb70 2024-04-17T22:30:58.582 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.594 7f2926ffd700 1 -- 172.21.15.102:0/1826893814 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29281aa310 con 0x7f292810c720 2024-04-17T22:30:58.582 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.594 7f29277fe700 1 --2- 172.21.15.102:0/1826893814 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f29280f5750 0x7f292810c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:30:58.582 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.594 7f2926ffd700 1 --2- 172.21.15.102:0/1826893814 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f292810c720 0x7f29281a0be0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 rx=0x7f29280f9570 tx=0x7f291800ba20).ready entity=mon.0 client_cookie=d5c7629bd31bc0de server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:30:58.582 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.594 7f29267fc700 1 -- 172.21.15.102:0/1826893814 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f29281aa310 con 0x7f292810c720 2024-04-17T22:30:58.582 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.594 7f29267fc700 1 -- 172.21.15.102:0/1826893814 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f291800d5c0 con 0x7f292810c720 2024-04-17T22:30:58.582 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.594 7f29267fc700 1 -- 172.21.15.102:0/1826893814 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f291800d9e0 con 0x7f292810c720 2024-04-17T22:30:58.582 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.594 7f292ee06700 1 -- 172.21.15.102:0/1826893814 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f292810c720 msgr2=0x7f29281a0be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:30:58.582 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.594 7f292ee06700 1 --2- 172.21.15.102:0/1826893814 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f292810c720 0x7f29281a0be0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 rx=0x7f29280f9570 tx=0x7f291800ba20).stop 2024-04-17T22:30:58.583 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.594 7f292ee06700 1 -- 172.21.15.102:0/1826893814 shutdown_connections 2024-04-17T22:30:58.583 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.594 7f292ee06700 1 --2- 172.21.15.102:0/1826893814 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f29280f5750 0x7f292810c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:30:58.583 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.594 7f292ee06700 1 --2- 172.21.15.102:0/1826893814 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f292810c720 0x7f29281a0be0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:30:58.583 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.595 7f292ee06700 1 -- 172.21.15.102:0/1826893814 >> 172.21.15.102:0/1826893814 conn(0x7f29281138a0 msgr2=0x7f29280f4f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:30:58.583 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.595 7f292ee06700 1 -- 172.21.15.102:0/1826893814 shutdown_connections 2024-04-17T22:30:58.583 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.595 7f292ee06700 1 -- 172.21.15.102:0/1826893814 wait complete. 2024-04-17T22:30:58.603 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.617 7f292ee06700 1 Processor -- start 2024-04-17T22:30:58.603 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.617 7f292ee06700 1 -- start start 2024-04-17T22:30:58.605 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.618 7f292ee06700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f29280cd6d0 0x7f29281ab0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:30:58.605 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.618 7f292ee06700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f29281ab6f0 0x7f29281b1aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:30:58.606 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.618 7f292ee06700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f29281abb40 0x7f29281b20d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:30:58.606 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.618 7f292ee06700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29280ccdb0 con 0x7f29280cd6d0 2024-04-17T22:30:58.606 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.618 7f292ee06700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29281b2700 con 0x7f29281abb40 2024-04-17T22:30:58.606 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.618 7f292ee06700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f29281b2930 con 0x7f29281ab6f0 2024-04-17T22:30:58.606 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.618 7f29277fe700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f29280cd6d0 0x7f29281ab0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:30:58.606 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.618 7f2926ffd700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f29281ab6f0 0x7f29281b1aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:30:58.606 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.618 7f29277fe700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f29280cd6d0 0x7f29281ab0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:40474/0 (socket says 172.21.15.102:40474) 2024-04-17T22:30:58.606 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.618 7f29277fe700 1 -- 172.21.15.102:0/928512794 learned_addr learned my addr 172.21.15.102:0/928512794 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:30:58.606 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.618 7f2927fff700 1 --2- 172.21.15.102:0/928512794 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f29281abb40 0x7f29281b20d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:30:58.606 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.618 7f29277fe700 1 -- 172.21.15.102:0/928512794 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f29281ab6f0 msgr2=0x7f29281b1aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:30:58.606 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.618 7f29277fe700 1 --2- 172.21.15.102:0/928512794 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f29281ab6f0 0x7f29281b1aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:30:58.606 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.618 7f29277fe700 1 -- 172.21.15.102:0/928512794 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f29281abb40 msgr2=0x7f29281b20d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:30:58.607 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.618 7f29277fe700 1 --2- 172.21.15.102:0/928512794 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f29281abb40 0x7f29281b20d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:30:58.607 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.618 7f29277fe700 1 -- 172.21.15.102:0/928512794 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29281b9210 con 0x7f29280cd6d0 2024-04-17T22:30:58.607 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.618 7f2926ffd700 1 --2- 172.21.15.102:0/928512794 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f29281ab6f0 0x7f29281b1aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:30:58.607 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.618 7f29277fe700 1 --2- 172.21.15.102:0/928512794 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f29280cd6d0 0x7f29281ab0c0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 rx=0x7f291c00c000 tx=0x7f291c00ecd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:30:58.607 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.618 7f2924ff9700 1 -- 172.21.15.102:0/928512794 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f29281b9210 con 0x7f29280cd6d0 2024-04-17T22:30:58.607 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.618 7f2924ff9700 1 -- 172.21.15.102:0/928512794 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f291c00ee10 con 0x7f29280cd6d0 2024-04-17T22:30:58.607 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.618 7f2924ff9700 1 -- 172.21.15.102:0/928512794 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f291c00f040 con 0x7f29280cd6d0 2024-04-17T22:30:58.607 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.618 7f292ee06700 1 -- 172.21.15.102:0/928512794 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29281b9410 con 0x7f29280cd6d0 2024-04-17T22:30:58.607 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.618 7f292ee06700 1 -- 172.21.15.102:0/928512794 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29281b9610 con 0x7f29280cd6d0 2024-04-17T22:30:58.607 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.619 7f2924ff9700 1 -- 172.21.15.102:0/928512794 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7f291c011070 con 0x7f29280cd6d0 2024-04-17T22:30:58.607 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.619 7f2924ff9700 1 -- 172.21.15.102:0/928512794 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3194+0+0 (secure 0 0 0) 0x7f291c010040 con 0x7f29280cd6d0 2024-04-17T22:30:58.607 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.619 7f292ee06700 1 -- 172.21.15.102:0/928512794 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29280f0800 con 0x7f29280cd6d0 2024-04-17T22:30:58.608 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.622 7f2924ff9700 1 -- 172.21.15.102:0/928512794 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+110455 (secure 0 0 0) 0x7f291c00c5a0 con 0x7f29280cd6d0 2024-04-17T22:30:58.713 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.727 7f292ee06700 1 -- 172.21.15.102:0/928512794 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f29280cd300 con 0x7f29280cd6d0 2024-04-17T22:30:58.713 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.727 7f2924ff9700 1 -- 172.21.15.102:0/928512794 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v17) v1 ==== 100+0+11 (secure 0 0 0) 0x7f291c000c30 con 0x7f29280cd6d0 2024-04-17T22:30:58.713 INFO:teuthology.orchestra.run.smithi102.stdout:pg_num: 64 2024-04-17T22:30:58.715 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.728 7f292ee06700 1 -- 172.21.15.102:0/928512794 >> v1:172.21.15.102:6816/42360 conn(0x7f29080145a0 legacy=0x7f2908016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:30:58.715 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.728 7f292ee06700 1 -- 172.21.15.102:0/928512794 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f29280cd6d0 msgr2=0x7f29281ab0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:30:58.715 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.728 7f292ee06700 1 --2- 172.21.15.102:0/928512794 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f29280cd6d0 0x7f29281ab0c0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 rx=0x7f291c00c000 tx=0x7f291c00ecd0).stop 2024-04-17T22:30:58.715 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.728 7f292ee06700 1 -- 172.21.15.102:0/928512794 shutdown_connections 2024-04-17T22:30:58.715 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.728 7f292ee06700 1 --2- 172.21.15.102:0/928512794 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f29281abb40 0x7f29281b20d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:30:58.715 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.728 7f292ee06700 1 --2- 172.21.15.102:0/928512794 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f29280cd6d0 0x7f29281ab0c0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:30:58.715 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.728 7f292ee06700 1 --2- 172.21.15.102:0/928512794 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f29281ab6f0 0x7f29281b1aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:30:58.715 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.728 7f292ee06700 1 -- 172.21.15.102:0/928512794 >> 172.21.15.102:0/928512794 conn(0x7f29280e0da0 msgr2=0x7f29280daac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:30:58.715 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.728 7f292ee06700 1 -- 172.21.15.102:0/928512794 shutdown_connections 2024-04-17T22:30:58.715 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.728 7f292ee06700 1 -- 172.21.15.102:0/928512794 wait complete. 2024-04-17T22:30:58.725 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T22:30:58.852 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.864 7f2b377d1700 1 Processor -- start 2024-04-17T22:30:58.852 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.864 7f2b377d1700 1 -- start start 2024-04-17T22:30:58.853 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.864 7f2b377d1700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2b3010b7a0 0x7f2b301a5250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:30:58.853 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.864 7f2b377d1700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f2b3010bbf0 0x7f2b301a7890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:30:58.853 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.864 7f2b377d1700 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b30103ad0 con 0x7f2b300f57d0 2024-04-17T22:30:58.853 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.864 7f2b377d1700 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b300fa300 con 0x7f2b3010bbf0 2024-04-17T22:30:58.853 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.864 7f2b377d1700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b301a0a90 con 0x7f2b3010b7a0 2024-04-17T22:30:58.853 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.864 7f2b34dcd700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f2b3010bbf0 0x7f2b301a7890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:30:58.853 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.864 7f2b2f7fe700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2b3010b7a0 0x7f2b301a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:30:58.853 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.865 7f2b34dcd700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f2b3010bbf0 0x7f2b301a7890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:40486/0 (socket says 172.21.15.102:40486) 2024-04-17T22:30:58.853 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.865 7f2b2ffff700 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f2b300f57d0 0x7f2b3010afa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:40982/0 (socket says 172.21.15.102:40982) 2024-04-17T22:30:58.853 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.865 7f2b34dcd700 1 -- 172.21.15.102:0/1623636469 learned_addr learned my addr 172.21.15.102:0/1623636469 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:30:58.853 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.865 7f2b2f7fe700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2b3010b7a0 0x7f2b301a5250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:48896/0 (socket says 172.21.15.102:48896) 2024-04-17T22:30:58.854 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.865 7f2b2f7fe700 1 -- 172.21.15.102:0/1623636469 >> v2:172.21.15.102:3300/0 conn(0x7f2b3010bbf0 msgr2=0x7f2b301a7890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:30:58.854 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.865 7f2b2f7fe700 1 --2- 172.21.15.102:0/1623636469 >> v2:172.21.15.102:3300/0 conn(0x7f2b3010bbf0 0x7f2b301a7890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:30:58.854 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.865 7f2b2f7fe700 1 -- 172.21.15.102:0/1623636469 >> v1:172.21.15.102:6789/0 conn(0x7f2b300f57d0 legacy=0x7f2b3010afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:30:58.854 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.865 7f2b2f7fe700 1 -- 172.21.15.102:0/1623636469 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b301a8ed0 con 0x7f2b3010b7a0 2024-04-17T22:30:58.854 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.865 7f2b2effd700 1 -- 172.21.15.102:0/1623636469 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2870267454 0 0) 0x7f2b24001110 con 0x7f2b300f57d0 2024-04-17T22:30:58.854 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.865 7f2b34dcd700 1 --2- 172.21.15.102:0/1623636469 >> v2:172.21.15.102:3300/0 conn(0x7f2b3010bbf0 0x7f2b301a7890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:30:58.854 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.865 7f2b2f7fe700 1 --2- 172.21.15.102:0/1623636469 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2b3010b7a0 0x7f2b301a5250 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 rx=0x7f2b20002f70 tx=0x7f2b2000b970).ready entity=mon.1 client_cookie=ada734d4c260c23d server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:30:58.854 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.865 7f2b2effd700 1 -- 172.21.15.102:0/1623636469 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f2b301a8ed0 con 0x7f2b3010b7a0 2024-04-17T22:30:58.854 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.865 7f2b2effd700 1 -- 172.21.15.102:0/1623636469 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b2000d5c0 con 0x7f2b3010b7a0 2024-04-17T22:30:58.854 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.865 7f2b2effd700 1 -- 172.21.15.102:0/1623636469 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f2b2000d990 con 0x7f2b3010b7a0 2024-04-17T22:30:58.854 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.865 7f2b377d1700 1 -- 172.21.15.102:0/1623636469 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2b3010b7a0 msgr2=0x7f2b301a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:30:58.854 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.865 7f2b377d1700 1 --2- 172.21.15.102:0/1623636469 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2b3010b7a0 0x7f2b301a5250 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 rx=0x7f2b20002f70 tx=0x7f2b2000b970).stop 2024-04-17T22:30:58.854 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.866 7f2b377d1700 1 -- 172.21.15.102:0/1623636469 shutdown_connections 2024-04-17T22:30:58.854 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.866 7f2b377d1700 1 --2- 172.21.15.102:0/1623636469 >> v2:172.21.15.102:3300/0 conn(0x7f2b3010bbf0 0x7f2b301a7890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:30:58.855 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.866 7f2b377d1700 1 --2- 172.21.15.102:0/1623636469 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2b3010b7a0 0x7f2b301a5250 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:30:58.855 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.866 7f2b377d1700 1 -- 172.21.15.102:0/1623636469 >> 172.21.15.102:0/1623636469 conn(0x7f2b30113830 msgr2=0x7f2b300f4ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:30:58.855 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.866 7f2b377d1700 1 -- 172.21.15.102:0/1623636469 shutdown_connections 2024-04-17T22:30:58.855 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.866 7f2b377d1700 1 -- 172.21.15.102:0/1623636469 wait complete. 2024-04-17T22:30:58.874 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.888 7f2b377d1700 1 Processor -- start 2024-04-17T22:30:58.874 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.888 7f2b377d1700 1 -- start start 2024-04-17T22:30:58.877 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.889 7f2b377d1700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f2b301a9300 0x7f2b301a9750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:30:58.877 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.889 7f2b377d1700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2b301a9d80 0x7f2b301afd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:30:58.877 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.889 7f2b377d1700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2b301b0370 0x7f2b301b07c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:30:58.877 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.889 7f2b377d1700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b300e2d40 con 0x7f2b301a9d80 2024-04-17T22:30:58.877 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.889 7f2b377d1700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b300dc7c0 con 0x7f2b301b0370 2024-04-17T22:30:58.877 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.889 7f2b377d1700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2b300c8da0 con 0x7f2b301a9300 2024-04-17T22:30:58.877 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.889 7f2b34dcd700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2b301b0370 0x7f2b301b07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:30:58.877 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.889 7f2b2f7fe700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2b301a9d80 0x7f2b301afd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:30:58.877 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.889 7f2b2ffff700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f2b301a9300 0x7f2b301a9750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:30:58.877 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.889 7f2b34dcd700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2b301b0370 0x7f2b301b07c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:48910/0 (socket says 172.21.15.102:48910) 2024-04-17T22:30:58.877 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.889 7f2b2f7fe700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2b301a9d80 0x7f2b301afd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:40500/0 (socket says 172.21.15.102:40500) 2024-04-17T22:30:58.877 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.889 7f2b34dcd700 1 -- 172.21.15.102:0/1520153360 learned_addr learned my addr 172.21.15.102:0/1520153360 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:30:58.877 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.889 7f2b2f7fe700 1 -- 172.21.15.102:0/1520153360 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f2b301a9300 msgr2=0x7f2b301a9750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:30:58.877 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.889 7f2b2f7fe700 1 --2- 172.21.15.102:0/1520153360 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f2b301a9300 0x7f2b301a9750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:30:58.878 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.889 7f2b2f7fe700 1 -- 172.21.15.102:0/1520153360 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2b301b0370 msgr2=0x7f2b301b07c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:30:58.878 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.889 7f2b2f7fe700 1 --2- 172.21.15.102:0/1520153360 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2b301b0370 0x7f2b301b07c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:30:58.878 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.889 7f2b2f7fe700 1 -- 172.21.15.102:0/1520153360 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b300cb880 con 0x7f2b301a9d80 2024-04-17T22:30:58.878 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.889 7f2b34dcd700 1 --2- 172.21.15.102:0/1520153360 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2b301b0370 0x7f2b301b07c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:30:58.878 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.889 7f2b2f7fe700 1 --2- 172.21.15.102:0/1520153360 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2b301a9d80 0x7f2b301afd40 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 rx=0x7f2b300c44f0 tx=0x7f2b200068c0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:30:58.878 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.889 7f2b2d7fa700 1 -- 172.21.15.102:0/1520153360 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f2b300cb880 con 0x7f2b301a9d80 2024-04-17T22:30:58.878 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.890 7f2b2d7fa700 1 -- 172.21.15.102:0/1520153360 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b200119c0 con 0x7f2b301a9d80 2024-04-17T22:30:58.878 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.890 7f2b2d7fa700 1 -- 172.21.15.102:0/1520153360 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f2b20017040 con 0x7f2b301a9d80 2024-04-17T22:30:58.878 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.890 7f2b377d1700 1 -- 172.21.15.102:0/1520153360 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b300dc2b0 con 0x7f2b301a9d80 2024-04-17T22:30:58.878 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.890 7f2b377d1700 1 -- 172.21.15.102:0/1520153360 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b301b7690 con 0x7f2b301a9d80 2024-04-17T22:30:58.878 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.890 7f2b2d7fa700 1 -- 172.21.15.102:0/1520153360 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7f2b20026360 con 0x7f2b301a9d80 2024-04-17T22:30:58.879 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.890 7f2b2d7fa700 1 -- 172.21.15.102:0/1520153360 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3194+0+0 (secure 0 0 0) 0x7f2b20018070 con 0x7f2b301a9d80 2024-04-17T22:30:58.879 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.891 7f2b377d1700 1 -- 172.21.15.102:0/1520153360 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b300f0790 con 0x7f2b301a9d80 2024-04-17T22:30:58.880 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:58.894 7f2b2d7fa700 1 -- 172.21.15.102:0/1520153360 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+110455 (secure 0 0 0) 0x7f2b200117b0 con 0x7f2b301a9d80 2024-04-17T22:30:59.013 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.027 7f2b377d1700 1 -- 172.21.15.102:0/1520153360 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2b300abea0 con 0x7f2b301a9d80 2024-04-17T22:30:59.014 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.027 7f2b2d7fa700 1 -- 172.21.15.102:0/1520153360 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) v1 ==== 93+0+1704 (secure 0 0 0) 0x7f2b2000d3b0 con 0x7f2b301a9d80 2024-04-17T22:30:59.014 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-17T22:30:59.014 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":5,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[],"filesystems":[{"mdsmap":{"epoch":5,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17 22:29:07.787730","modified":"2024-04-17 22:29:09.795249","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"-1 (unspecified)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4177},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4177":{"gid":4177,"name":"a","rank":0,"incarnation":3,"state":"up:active","state_seq":4,"addr":"172.21.15.102:6817/1944542598","addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6817","nonce":1944542598}]},"export_targets":[],"features":4611087854035861503,"flags":0}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-04-17T22:30:59.015 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.029 7f2b377d1700 1 -- 172.21.15.102:0/1520153360 >> v1:172.21.15.102:6816/42360 conn(0x7f2b100145a0 legacy=0x7f2b10016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:30:59.015 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.029 7f2b377d1700 1 -- 172.21.15.102:0/1520153360 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2b301a9d80 msgr2=0x7f2b301afd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:30:59.015 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.029 7f2b377d1700 1 --2- 172.21.15.102:0/1520153360 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2b301a9d80 0x7f2b301afd40 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 rx=0x7f2b300c44f0 tx=0x7f2b200068c0).stop 2024-04-17T22:30:59.015 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.029 7f2b377d1700 1 -- 172.21.15.102:0/1520153360 shutdown_connections 2024-04-17T22:30:59.016 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.029 7f2b377d1700 1 --2- 172.21.15.102:0/1520153360 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2b301b0370 0x7f2b301b07c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:30:59.016 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.029 7f2b377d1700 1 --2- 172.21.15.102:0/1520153360 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f2b301a9300 0x7f2b301a9750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:30:59.016 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.029 7f2b377d1700 1 --2- 172.21.15.102:0/1520153360 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2b301a9d80 0x7f2b301afd40 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:30:59.016 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.029 7f2b377d1700 1 -- 172.21.15.102:0/1520153360 >> 172.21.15.102:0/1520153360 conn(0x7f2b300e0eb0 msgr2=0x7f2b300c9360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:30:59.016 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.029 7f2b377d1700 1 -- 172.21.15.102:0/1520153360 shutdown_connections 2024-04-17T22:30:59.016 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.029 7f2b377d1700 1 -- 172.21.15.102:0/1520153360 wait complete. 2024-04-17T22:30:59.016 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 5 2024-04-17T22:30:59.026 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T22:30:59.153 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.164 7ffa4e01f700 1 Processor -- start 2024-04-17T22:30:59.153 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.164 7ffa4e01f700 1 -- start start 2024-04-17T22:30:59.153 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.165 7ffa4e01f700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7ffa4810e260 0x7ffa4810e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:30:59.153 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.165 7ffa4e01f700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7ffa480f1f50 0x7ffa481ab500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:30:59.153 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.165 7ffa4e01f700 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa480f23a0 con 0x7ffa4810ece0 2024-04-17T22:30:59.153 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.165 7ffa4e01f700 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa480f9f30 con 0x7ffa480f1f50 2024-04-17T22:30:59.153 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.165 7ffa4e01f700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ffa480f9050 con 0x7ffa4810e260 2024-04-17T22:30:59.153 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.165 7ffa475c7700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7ffa480f1f50 0x7ffa481ab500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:30:59.153 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.165 7ffa46dc6700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7ffa4810e260 0x7ffa4810e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:30:59.153 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.165 7ffa465c5700 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7ffa4810ece0 0x7ffa480f1750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:40992/0 (socket says 172.21.15.102:40992) 2024-04-17T22:30:59.154 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.165 7ffa465c5700 1 -- 172.21.15.102:0/1634381736 learned_addr learned my addr 172.21.15.102:0/1634381736 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-17T22:30:59.154 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.165 7ffa45dc4700 1 -- 172.21.15.102:0/1634381736 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 918090346 0 0) 0x7ffa3c001110 con 0x7ffa4810ece0 2024-04-17T22:30:59.154 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.165 7ffa45dc4700 1 -- 172.21.15.102:0/1634381736 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffa2c003270 con 0x7ffa4810ece0 2024-04-17T22:30:59.154 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.165 7ffa475c7700 1 -- 172.21.15.102:0/1634381736 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7ffa4810e260 msgr2=0x7ffa4810e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:30:59.154 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.165 7ffa475c7700 1 --2- 172.21.15.102:0/1634381736 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7ffa4810e260 0x7ffa4810e6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:30:59.154 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.165 7ffa475c7700 1 -- 172.21.15.102:0/1634381736 >> v1:172.21.15.102:6789/0 conn(0x7ffa4810ece0 legacy=0x7ffa480f1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:30:59.154 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.165 7ffa475c7700 1 -- 172.21.15.102:0/1634381736 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa480f25c0 con 0x7ffa480f1f50 2024-04-17T22:30:59.154 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.165 7ffa46dc6700 1 --2- 172.21.15.102:0/1634381736 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7ffa4810e260 0x7ffa4810e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:30:59.154 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.165 7ffa475c7700 1 --2- 172.21.15.102:0/1634381736 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffa480f1f50 0x7ffa481ab500 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 rx=0x7ffa481035a0 tx=0x7ffa40009d40).ready entity=mon.0 client_cookie=608d151e1a981ff4 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:30:59.154 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.165 7ffa45dc4700 1 -- 172.21.15.102:0/1634381736 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ffa480f25c0 con 0x7ffa480f1f50 2024-04-17T22:30:59.154 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.165 7ffa45dc4700 1 -- 172.21.15.102:0/1634381736 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa4000b7c0 con 0x7ffa480f1f50 2024-04-17T22:30:59.154 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.165 7ffa45dc4700 1 -- 172.21.15.102:0/1634381736 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ffa4000b9f0 con 0x7ffa480f1f50 2024-04-17T22:30:59.155 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.165 7ffa4e01f700 1 -- 172.21.15.102:0/1634381736 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffa480f1f50 msgr2=0x7ffa481ab500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:30:59.155 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.165 7ffa4e01f700 1 --2- 172.21.15.102:0/1634381736 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffa480f1f50 0x7ffa481ab500 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 rx=0x7ffa481035a0 tx=0x7ffa40009d40).stop 2024-04-17T22:30:59.155 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.166 7ffa4e01f700 1 -- 172.21.15.102:0/1634381736 shutdown_connections 2024-04-17T22:30:59.155 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.166 7ffa4e01f700 1 --2- 172.21.15.102:0/1634381736 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7ffa4810e260 0x7ffa4810e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:30:59.155 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.166 7ffa4e01f700 1 --2- 172.21.15.102:0/1634381736 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffa480f1f50 0x7ffa481ab500 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:30:59.155 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.166 7ffa4e01f700 1 -- 172.21.15.102:0/1634381736 >> 172.21.15.102:0/1634381736 conn(0x7ffa481138c0 msgr2=0x7ffa480f4f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:30:59.155 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.166 7ffa4e01f700 1 -- 172.21.15.102:0/1634381736 shutdown_connections 2024-04-17T22:30:59.155 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.166 7ffa4e01f700 1 -- 172.21.15.102:0/1634381736 wait complete. 2024-04-17T22:30:59.177 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.189 7ffa4e01f700 1 Processor -- start 2024-04-17T22:30:59.177 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.189 7ffa4e01f700 1 -- start start 2024-04-17T22:30:59.177 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.189 7ffa4e01f700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ffa480ce620 0x7ffa480d51d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:30:59.177 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.189 7ffa4e01f700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffa480d5800 0x7ffa481b4090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:30:59.177 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.189 7ffa4e01f700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7ffa480d5c50 0x7ffa481b46c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:30:59.177 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.189 7ffa4e01f700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa480fa1d0 con 0x7ffa480d5800 2024-04-17T22:30:59.177 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.189 7ffa4e01f700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa480ddfe0 con 0x7ffa480ce620 2024-04-17T22:30:59.177 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.189 7ffa4e01f700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ffa481b4cf0 con 0x7ffa480d5c50 2024-04-17T22:30:59.177 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.189 7ffa465c5700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffa480d5800 0x7ffa481b4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:30:59.177 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.189 7ffa475c7700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7ffa480d5c50 0x7ffa481b46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:30:59.177 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.189 7ffa465c5700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffa480d5800 0x7ffa481b4090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:40524/0 (socket says 172.21.15.102:40524) 2024-04-17T22:30:59.178 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.189 7ffa465c5700 1 -- 172.21.15.102:0/3573207738 learned_addr learned my addr 172.21.15.102:0/3573207738 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:30:59.178 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.189 7ffa46dc6700 1 --2- 172.21.15.102:0/3573207738 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ffa480ce620 0x7ffa480d51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:30:59.178 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.189 7ffa465c5700 1 -- 172.21.15.102:0/3573207738 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7ffa480d5c50 msgr2=0x7ffa481b46c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:30:59.178 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.189 7ffa465c5700 1 --2- 172.21.15.102:0/3573207738 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7ffa480d5c50 0x7ffa481b46c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:30:59.178 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.189 7ffa465c5700 1 -- 172.21.15.102:0/3573207738 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ffa480ce620 msgr2=0x7ffa480d51d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:30:59.178 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.189 7ffa465c5700 1 --2- 172.21.15.102:0/3573207738 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ffa480ce620 0x7ffa480d51d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:30:59.178 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.189 7ffa465c5700 1 -- 172.21.15.102:0/3573207738 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa481b4ef0 con 0x7ffa480d5800 2024-04-17T22:30:59.178 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.189 7ffa475c7700 1 --2- 172.21.15.102:0/3573207738 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7ffa480d5c50 0x7ffa481b46c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:30:59.178 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.189 7ffa46dc6700 1 --2- 172.21.15.102:0/3573207738 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ffa480ce620 0x7ffa480d51d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:30:59.178 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.190 7ffa465c5700 1 --2- 172.21.15.102:0/3573207738 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffa480d5800 0x7ffa481b4090 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 rx=0x7ffa3c000bc0 tx=0x7ffa3c00c7e0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:30:59.178 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.190 7ffa37fff700 1 -- 172.21.15.102:0/3573207738 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ffa481b4ef0 con 0x7ffa480d5800 2024-04-17T22:30:59.178 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.190 7ffa37fff700 1 -- 172.21.15.102:0/3573207738 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa3c0072e0 con 0x7ffa480d5800 2024-04-17T22:30:59.178 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.190 7ffa4e01f700 1 -- 172.21.15.102:0/3573207738 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa480ff020 con 0x7ffa480d5800 2024-04-17T22:30:59.179 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.190 7ffa37fff700 1 -- 172.21.15.102:0/3573207738 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ffa3c005810 con 0x7ffa480d5800 2024-04-17T22:30:59.179 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.190 7ffa4e01f700 1 -- 172.21.15.102:0/3573207738 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa480ff340 con 0x7ffa480d5800 2024-04-17T22:30:59.179 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.190 7ffa37fff700 1 -- 172.21.15.102:0/3573207738 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7ffa3c005a10 con 0x7ffa480d5800 2024-04-17T22:30:59.179 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.190 7ffa37fff700 1 -- 172.21.15.102:0/3573207738 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3194+0+0 (secure 0 0 0) 0x7ffa3c0148d0 con 0x7ffa480d5800 2024-04-17T22:30:59.179 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.191 7ffa4e01f700 1 -- 172.21.15.102:0/3573207738 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa480f0820 con 0x7ffa480d5800 2024-04-17T22:30:59.180 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.194 7ffa37fff700 1 -- 172.21.15.102:0/3573207738 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+110455 (secure 0 0 0) 0x7ffa3c0070d0 con 0x7ffa480d5800 2024-04-17T22:30:59.285 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.299 7ffa4e01f700 1 -- 172.21.15.102:0/3573207738 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ffa481ae790 con 0x7ffa480d5800 2024-04-17T22:30:59.286 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.299 7ffa37fff700 1 -- 172.21.15.102:0/3573207738 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) v1 ==== 74+0+8764 (secure 0 0 0) 0x7ffa480f1460 con 0x7ffa480d5800 2024-04-17T22:30:59.286 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-17T22:30:59.286 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":17,"fsid":"cb63e789-091d-49b1-b120-55f9db526871","created":"2024-04-17 22:28:48.209635","modified":"2024-04-17 22:29:23.257387","last_up_change":"2024-04-17 22:28:54.236215","last_in_change":"2024-04-17 22:28:54.236215","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":4,"require_min_compat_client":"nautilus","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-04-17 22:28:57.478545","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-17 22:29:06.144708","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-17 22:29:06.748924","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3351ef85-ade4-4b90-a8f4-213f4e00ba01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6812","nonce":42955}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6813","nonce":42955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6815","nonce":42955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6814","nonce":42955}]},"public_addr":"172.21.15.102:6812/42955","cluster_addr":"172.21.15.102:6813/42955","heartbeat_back_addr":"172.21.15.102:6815/42955","heartbeat_front_addr":"172.21.15.102:6814/42955","state":["exists","up"]},{"osd":1,"uuid":"04f67eac-f36b-48f5-982d-4cfc776596d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6808","nonce":42958}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6809","nonce":42958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6811","nonce":42958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6810","nonce":42958}]},"public_addr":"172.21.15.102:6808/42958","cluster_addr":"172.21.15.102:6809/42958","heartbeat_back_addr":"172.21.15.102:6811/42958","heartbeat_front_addr":"172.21.15.102:6810/42958","state":["exists","up"]},{"osd":2,"uuid":"d524204b-70b2-40a8-8f73-f9e7d44318af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6804","nonce":42954}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6805","nonce":42954}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6807","nonce":42954}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6806","nonce":42954}]},"public_addr":"172.21.15.102:6804/42954","cluster_addr":"172.21.15.102:6805/42954","heartbeat_back_addr":"172.21.15.102:6807/42954","heartbeat_front_addr":"172.21.15.102:6806/42954","state":["exists","up"]},{"osd":3,"uuid":"4fc9d20c-1ad0-484f-aafa-2c9a7574245b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6800","nonce":42953}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6801","nonce":42953}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6803","nonce":42953}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6802","nonce":42953}]},"public_addr":"172.21.15.102:6800/42953","cluster_addr":"172.21.15.102:6801/42953","heartbeat_back_addr":"172.21.15.102:6803/42953","heartbeat_front_addr":"172.21.15.102:6802/42953","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2024-04-17T22:30:59.287 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.301 7ffa4e01f700 1 -- 172.21.15.102:0/3573207738 >> v1:172.21.15.102:6816/42360 conn(0x7ffa240145a0 legacy=0x7ffa24016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:30:59.287 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.301 7ffa4e01f700 1 -- 172.21.15.102:0/3573207738 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffa480d5800 msgr2=0x7ffa481b4090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:30:59.287 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.301 7ffa4e01f700 1 --2- 172.21.15.102:0/3573207738 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffa480d5800 0x7ffa481b4090 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 rx=0x7ffa3c000bc0 tx=0x7ffa3c00c7e0).stop 2024-04-17T22:30:59.287 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.301 7ffa4e01f700 1 -- 172.21.15.102:0/3573207738 shutdown_connections 2024-04-17T22:30:59.287 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.301 7ffa4e01f700 1 --2- 172.21.15.102:0/3573207738 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7ffa480d5c50 0x7ffa481b46c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:30:59.287 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.301 7ffa4e01f700 1 --2- 172.21.15.102:0/3573207738 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ffa480ce620 0x7ffa480d51d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:30:59.288 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.301 7ffa4e01f700 1 --2- 172.21.15.102:0/3573207738 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffa480d5800 0x7ffa481b4090 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:30:59.288 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.301 7ffa4e01f700 1 -- 172.21.15.102:0/3573207738 >> 172.21.15.102:0/3573207738 conn(0x7ffa480c6af0 msgr2=0x7ffa481a8300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:30:59.288 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.301 7ffa4e01f700 1 -- 172.21.15.102:0/3573207738 shutdown_connections 2024-04-17T22:30:59.288 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.301 7ffa4e01f700 1 -- 172.21.15.102:0/3573207738 wait complete. 2024-04-17T22:30:59.298 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2024-04-17 22:28:57.478545', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '12', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-17 22:29:06.144708', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-17 22:29:06.748924', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-17T22:30:59.298 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2024-04-17T22:30:59.429 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.441 7fc72d36b700 1 Processor -- start 2024-04-17T22:30:59.429 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.441 7fc72d36b700 1 -- start start 2024-04-17T22:30:59.430 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.441 7fc72d36b700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fc7280f5840 0x7fc72810b010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:30:59.430 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.441 7fc72d36b700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc72810b640 0x7fc72819faf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:30:59.430 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.441 7fc72d36b700 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc728103b40 con 0x7fc72810ba90 2024-04-17T22:30:59.430 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.441 7fc72d36b700 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc72810bee0 con 0x7fc7280f5840 2024-04-17T22:30:59.430 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.441 7fc72d36b700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7280fa370 con 0x7fc72810b640 2024-04-17T22:30:59.430 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.441 7fc71ffff700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc72810b640 0x7fc72819faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:30:59.430 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.441 7fc72605c700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fc7280f5840 0x7fc72810b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:30:59.430 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.441 7fc72605c700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fc7280f5840 0x7fc72810b010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:40538/0 (socket says 172.21.15.102:40538) 2024-04-17T22:30:59.430 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.441 7fc72605c700 1 -- 172.21.15.102:0/1789277373 learned_addr learned my addr 172.21.15.102:0/1789277373 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:30:59.430 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.441 7fc71ffff700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc72810b640 0x7fc72819faf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:48938/0 (socket says 172.21.15.102:48938) 2024-04-17T22:30:59.430 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.442 7fc72605c700 1 -- 172.21.15.102:0/1789277373 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc72810b640 msgr2=0x7fc72819faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:30:59.430 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.442 7fc72605c700 1 --2- 172.21.15.102:0/1789277373 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc72810b640 0x7fc72819faf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:30:59.431 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.442 7fc72605c700 1 -- 172.21.15.102:0/1789277373 >> v1:172.21.15.102:6789/0 conn(0x7fc72810ba90 legacy=0x7fc7281a0120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:30:59.431 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.442 7fc72605c700 1 -- 172.21.15.102:0/1789277373 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7281a0920 con 0x7fc7280f5840 2024-04-17T22:30:59.431 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.442 7fc72585b700 1 -- 172.21.15.102:0/1789277373 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3005031708 0 0) 0x7fc720001490 con 0x7fc72810ba90 2024-04-17T22:30:59.431 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.442 7fc71ffff700 1 --2- 172.21.15.102:0/1789277373 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc72810b640 0x7fc72819faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:30:59.431 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.442 7fc72605c700 1 --2- 172.21.15.102:0/1789277373 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc7280f5840 0x7fc72810b010 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 rx=0x7fc728103d60 tx=0x7fc718009a80).ready entity=mon.0 client_cookie=364fb9d1ff22e7b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:30:59.431 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.442 7fc72585b700 1 -- 172.21.15.102:0/1789277373 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc7281a0920 con 0x7fc7280f5840 2024-04-17T22:30:59.431 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.442 7fc72585b700 1 -- 172.21.15.102:0/1789277373 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc71800f510 con 0x7fc7280f5840 2024-04-17T22:30:59.431 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.442 7fc72585b700 1 -- 172.21.15.102:0/1789277373 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc71800f6f0 con 0x7fc7280f5840 2024-04-17T22:30:59.431 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.442 7fc72d36b700 1 -- 172.21.15.102:0/1789277373 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc7280f5840 msgr2=0x7fc72810b010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:30:59.431 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.442 7fc72d36b700 1 --2- 172.21.15.102:0/1789277373 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc7280f5840 0x7fc72810b010 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 rx=0x7fc728103d60 tx=0x7fc718009a80).stop 2024-04-17T22:30:59.431 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.443 7fc72d36b700 1 -- 172.21.15.102:0/1789277373 shutdown_connections 2024-04-17T22:30:59.431 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.443 7fc72d36b700 1 --2- 172.21.15.102:0/1789277373 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc7280f5840 0x7fc72810b010 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:30:59.431 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.443 7fc72d36b700 1 --2- 172.21.15.102:0/1789277373 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc72810b640 0x7fc72819faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:30:59.432 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.443 7fc72d36b700 1 -- 172.21.15.102:0/1789277373 >> 172.21.15.102:0/1789277373 conn(0x7fc7281138a0 msgr2=0x7fc7280f4f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:30:59.432 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.443 7fc72d36b700 1 -- 172.21.15.102:0/1789277373 shutdown_connections 2024-04-17T22:30:59.432 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.443 7fc72d36b700 1 -- 172.21.15.102:0/1789277373 wait complete. 2024-04-17T22:30:59.454 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.465 7fc72d36b700 1 Processor -- start 2024-04-17T22:30:59.454 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.465 7fc72d36b700 1 -- start start 2024-04-17T22:30:59.454 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.465 7fc72d36b700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc7281b43a0 0x7fc7281b47f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:30:59.454 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.465 7fc72d36b700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fc7281b4e20 0x7fc7280c38c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:30:59.454 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.466 7fc72d36b700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc7280c3ef0 0x7fc7281bb5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:30:59.454 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.466 7fc72d36b700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7280c4a30 con 0x7fc7280c3ef0 2024-04-17T22:30:59.455 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.466 7fc72d36b700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7280cde80 con 0x7fc7281b43a0 2024-04-17T22:30:59.455 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.466 7fc72d36b700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc7280e0490 con 0x7fc7281b4e20 2024-04-17T22:30:59.455 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.466 7fc72605c700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc7281b43a0 0x7fc7281b47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:30:59.455 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.466 7fc72605c700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc7281b43a0 0x7fc7281b47f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:48942/0 (socket says 172.21.15.102:48942) 2024-04-17T22:30:59.455 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.466 7fc72605c700 1 -- 172.21.15.102:0/1262436518 learned_addr learned my addr 172.21.15.102:0/1262436518 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:30:59.455 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.466 7fc72685d700 1 --2- 172.21.15.102:0/1262436518 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc7280c3ef0 0x7fc7281bb5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:30:59.455 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.466 7fc71ffff700 1 --2- 172.21.15.102:0/1262436518 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fc7281b4e20 0x7fc7280c38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:30:59.455 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.466 7fc72605c700 1 -- 172.21.15.102:0/1262436518 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fc7281b4e20 msgr2=0x7fc7280c38c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:30:59.455 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.466 7fc72605c700 1 --2- 172.21.15.102:0/1262436518 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fc7281b4e20 0x7fc7280c38c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:30:59.455 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.466 7fc72605c700 1 -- 172.21.15.102:0/1262436518 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc7280c3ef0 msgr2=0x7fc7281bb5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:30:59.456 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.466 7fc72605c700 1 --2- 172.21.15.102:0/1262436518 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc7280c3ef0 0x7fc7281bb5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:30:59.456 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.466 7fc72605c700 1 -- 172.21.15.102:0/1262436518 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7280c4340 con 0x7fc7281b43a0 2024-04-17T22:30:59.456 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.466 7fc72605c700 1 --2- 172.21.15.102:0/1262436518 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc7281b43a0 0x7fc7281b47f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 rx=0x7fc7280bc700 tx=0x7fc7180068c0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:30:59.456 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.466 7fc71f7fe700 1 -- 172.21.15.102:0/1262436518 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc7280c4340 con 0x7fc7281b43a0 2024-04-17T22:30:59.456 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.466 7fc71f7fe700 1 -- 172.21.15.102:0/1262436518 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7180117f0 con 0x7fc7281b43a0 2024-04-17T22:30:59.456 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.466 7fc72d36b700 1 -- 172.21.15.102:0/1262436518 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7280c4540 con 0x7fc7281b43a0 2024-04-17T22:30:59.456 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.466 7fc71f7fe700 1 -- 172.21.15.102:0/1262436518 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc718011bc0 con 0x7fc7281b43a0 2024-04-17T22:30:59.456 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.466 7fc72d36b700 1 -- 172.21.15.102:0/1262436518 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7281bbe00 con 0x7fc7281b43a0 2024-04-17T22:30:59.456 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.467 7fc71f7fe700 1 -- 172.21.15.102:0/1262436518 <== mon.1 v2:172.21.15.102:3301/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7fc718027460 con 0x7fc7281b43a0 2024-04-17T22:30:59.456 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.467 7fc71f7fe700 1 -- 172.21.15.102:0/1262436518 <== mon.1 v2:172.21.15.102:3301/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3194+0+0 (secure 0 0 0) 0x7fc718001a80 con 0x7fc7281b43a0 2024-04-17T22:30:59.456 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.467 7fc72d36b700 1 -- 172.21.15.102:0/1262436518 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7280f0800 con 0x7fc7281b43a0 2024-04-17T22:30:59.457 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.471 7fc71f7fe700 1 -- 172.21.15.102:0/1262436518 <== mon.1 v2:172.21.15.102:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+110455 (secure 0 0 0) 0x7fc7180115e0 con 0x7fc7281b43a0 2024-04-17T22:30:59.562 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.575 7fc72d36b700 1 -- 172.21.15.102:0/1262436518 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fc7280efed0 con 0x7fc7281b43a0 2024-04-17T22:30:59.562 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.576 7fc71f7fe700 1 -- 172.21.15.102:0/1262436518 <== mon.1 v2:172.21.15.102:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v17) v1 ==== 92+0+10 (secure 0 0 0) 0x7fc71800f300 con 0x7fc7281b43a0 2024-04-17T22:30:59.562 INFO:teuthology.orchestra.run.smithi102.stdout:pg_num: 8 2024-04-17T22:30:59.563 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.577 7fc72d36b700 1 -- 172.21.15.102:0/1262436518 >> v1:172.21.15.102:6816/42360 conn(0x7fc70c0145a0 legacy=0x7fc70c016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:30:59.563 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.577 7fc72d36b700 1 -- 172.21.15.102:0/1262436518 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc7281b43a0 msgr2=0x7fc7281b47f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:30:59.563 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.577 7fc72d36b700 1 --2- 172.21.15.102:0/1262436518 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc7281b43a0 0x7fc7281b47f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 rx=0x7fc7280bc700 tx=0x7fc7180068c0).stop 2024-04-17T22:30:59.564 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.577 7fc72d36b700 1 -- 172.21.15.102:0/1262436518 shutdown_connections 2024-04-17T22:30:59.564 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.577 7fc72d36b700 1 --2- 172.21.15.102:0/1262436518 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc7280c3ef0 0x7fc7281bb5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:30:59.564 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.577 7fc72d36b700 1 --2- 172.21.15.102:0/1262436518 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc7281b43a0 0x7fc7281b47f0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:30:59.564 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.577 7fc72d36b700 1 --2- 172.21.15.102:0/1262436518 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fc7281b4e20 0x7fc7280c38c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:30:59.564 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.577 7fc72d36b700 1 -- 172.21.15.102:0/1262436518 >> 172.21.15.102:0/1262436518 conn(0x7fc7280c59e0 msgr2=0x7fc7280bc900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:30:59.564 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.577 7fc72d36b700 1 -- 172.21.15.102:0/1262436518 shutdown_connections 2024-04-17T22:30:59.564 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.577 7fc72d36b700 1 -- 172.21.15.102:0/1262436518 wait complete. 2024-04-17T22:30:59.574 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-17T22:30:59.705 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.715 7f076ac56700 1 Processor -- start 2024-04-17T22:30:59.705 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.715 7f076ac56700 1 -- start start 2024-04-17T22:30:59.705 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.715 7f076ac56700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f076410b810 0x7f07641a52c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:30:59.705 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.715 7f076ac56700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f076410bc60 0x7f07641a7900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:30:59.705 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.715 7f076ac56700 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0764103b40 con 0x7f07640f5840 2024-04-17T22:30:59.705 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.715 7f076ac56700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07640fa370 con 0x7f076410bc60 2024-04-17T22:30:59.706 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.715 7f076ac56700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f07641a0b00 con 0x7f076410b810 2024-04-17T22:30:59.706 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.715 7f0762ffd700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f076410b810 0x7f07641a52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:30:59.706 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.715 7f0763fff700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f076410bc60 0x7f07641a7900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:30:59.706 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.715 7f07637fe700 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f07640f5840 0x7f076410b010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:41002/0 (socket says 172.21.15.102:41002) 2024-04-17T22:30:59.706 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.715 7f07637fe700 1 -- 172.21.15.102:0/2775397096 learned_addr learned my addr 172.21.15.102:0/2775397096 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-17T22:30:59.706 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.716 7f0762ffd700 1 -- 172.21.15.102:0/2775397096 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f076410bc60 msgr2=0x7f07641a7900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:30:59.706 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.716 7f0762ffd700 1 --2- 172.21.15.102:0/2775397096 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f076410bc60 0x7f07641a7900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:30:59.706 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.716 7f0762ffd700 1 -- 172.21.15.102:0/2775397096 >> v1:172.21.15.102:6789/0 conn(0x7f07640f5840 legacy=0x7f076410b010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:30:59.707 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.716 7f07627fc700 1 -- 172.21.15.102:0/2775397096 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1103850058 0 0) 0x7f0754001110 con 0x7f07640f5840 2024-04-17T22:30:59.707 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.716 7f0762ffd700 1 -- 172.21.15.102:0/2775397096 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07641a8f40 con 0x7f076410b810 2024-04-17T22:30:59.707 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.716 7f0763fff700 1 --2- 172.21.15.102:0/2775397096 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f076410bc60 0x7f07641a7900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:30:59.707 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.716 7f0762ffd700 1 --2- 172.21.15.102:0/2775397096 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f076410b810 0x7f07641a52c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 rx=0x7f07640f46a0 tx=0x7f074c00b9d0).ready entity=mon.2 client_cookie=76b4e0a8df412ddc server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:30:59.707 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.716 7f07627fc700 1 -- 172.21.15.102:0/2775397096 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f07641a8f40 con 0x7f076410b810 2024-04-17T22:30:59.707 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.716 7f07627fc700 1 -- 172.21.15.102:0/2775397096 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f074c00d570 con 0x7f076410b810 2024-04-17T22:30:59.708 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.716 7f07627fc700 1 -- 172.21.15.102:0/2775397096 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f074c00d940 con 0x7f076410b810 2024-04-17T22:30:59.708 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.716 7f076ac56700 1 -- 172.21.15.102:0/2775397096 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f076410b810 msgr2=0x7f07641a52c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:30:59.708 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.716 7f076ac56700 1 --2- 172.21.15.102:0/2775397096 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f076410b810 0x7f07641a52c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 rx=0x7f07640f46a0 tx=0x7f074c00b9d0).stop 2024-04-17T22:30:59.708 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.717 7f076ac56700 1 -- 172.21.15.102:0/2775397096 shutdown_connections 2024-04-17T22:30:59.708 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.717 7f076ac56700 1 --2- 172.21.15.102:0/2775397096 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f076410bc60 0x7f07641a7900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:30:59.708 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.717 7f076ac56700 1 --2- 172.21.15.102:0/2775397096 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f076410b810 0x7f07641a52c0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:30:59.709 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.717 7f076ac56700 1 -- 172.21.15.102:0/2775397096 >> 172.21.15.102:0/2775397096 conn(0x7f07641138a0 msgr2=0x7f07640f4f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:30:59.709 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.717 7f076ac56700 1 -- 172.21.15.102:0/2775397096 shutdown_connections 2024-04-17T22:30:59.709 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.717 7f076ac56700 1 -- 172.21.15.102:0/2775397096 wait complete. 2024-04-17T22:30:59.729 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.740 7f076ac56700 1 Processor -- start 2024-04-17T22:30:59.729 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.740 7f076ac56700 1 -- start start 2024-04-17T22:30:59.729 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.740 7f076ac56700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f07640e0420 0x7f07640db140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:30:59.730 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.740 7f076ac56700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f07640db770 0x7f07640d52b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:30:59.730 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.740 7f076ac56700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f07640dbbc0 0x7f07640d58e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:30:59.730 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.740 7f076ac56700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07640dd5c0 con 0x7f07640db770 2024-04-17T22:30:59.730 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.740 7f076ac56700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07640d5f10 con 0x7f07640dbbc0 2024-04-17T22:30:59.730 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.740 7f076ac56700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f07641b8750 con 0x7f07640e0420 2024-04-17T22:30:59.730 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.740 7f07637fe700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f07640e0420 0x7f07640db140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:30:59.730 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.740 7f0763fff700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f07640dbbc0 0x7f07640d58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:30:59.730 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.740 7f07637fe700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f07640e0420 0x7f07640db140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:59308/0 (socket says 172.21.15.102:59308) 2024-04-17T22:30:59.730 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.740 7f07637fe700 1 -- 172.21.15.102:0/1450461133 learned_addr learned my addr 172.21.15.102:0/1450461133 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:30:59.731 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.740 7f0762ffd700 1 --2- 172.21.15.102:0/1450461133 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f07640db770 0x7f07640d52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:30:59.731 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.741 7f0763fff700 1 -- 172.21.15.102:0/1450461133 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f07640e0420 msgr2=0x7f07640db140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:30:59.731 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.741 7f0763fff700 1 --2- 172.21.15.102:0/1450461133 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f07640e0420 0x7f07640db140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:30:59.731 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.741 7f0763fff700 1 -- 172.21.15.102:0/1450461133 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f07640db770 msgr2=0x7f07640d52b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:30:59.731 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.741 7f0763fff700 1 --2- 172.21.15.102:0/1450461133 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f07640db770 0x7f07640d52b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:30:59.731 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.741 7f0763fff700 1 -- 172.21.15.102:0/1450461133 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07641b8900 con 0x7f07640dbbc0 2024-04-17T22:30:59.732 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.741 7f0763fff700 1 --2- 172.21.15.102:0/1450461133 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f07640dbbc0 0x7f07640d58e0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 rx=0x7f07640c2580 tx=0x7f075c00eda0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:30:59.732 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.741 7f0760ff9700 1 -- 172.21.15.102:0/1450461133 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f07641b8900 con 0x7f07640dbbc0 2024-04-17T22:30:59.732 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.741 7f0760ff9700 1 -- 172.21.15.102:0/1450461133 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f075c00ca70 con 0x7f07640dbbc0 2024-04-17T22:30:59.732 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.741 7f076ac56700 1 -- 172.21.15.102:0/1450461133 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07640f17b0 con 0x7f07640dbbc0 2024-04-17T22:30:59.732 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.741 7f0760ff9700 1 -- 172.21.15.102:0/1450461133 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f075c00f040 con 0x7f07640dbbc0 2024-04-17T22:30:59.732 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.741 7f076ac56700 1 -- 172.21.15.102:0/1450461133 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07640f1cb0 con 0x7f07640dbbc0 2024-04-17T22:30:59.732 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.741 7f0760ff9700 1 -- 172.21.15.102:0/1450461133 <== mon.1 v2:172.21.15.102:3301/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7f075c011450 con 0x7f07640dbbc0 2024-04-17T22:30:59.733 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.741 7f0760ff9700 1 -- 172.21.15.102:0/1450461133 <== mon.1 v2:172.21.15.102:3301/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3194+0+0 (secure 0 0 0) 0x7f075c012d30 con 0x7f07640dbbc0 2024-04-17T22:30:59.733 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.742 7f076ac56700 1 -- 172.21.15.102:0/1450461133 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07640c47d0 con 0x7f07640dbbc0 2024-04-17T22:30:59.733 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.745 7f0760ff9700 1 -- 172.21.15.102:0/1450461133 <== mon.1 v2:172.21.15.102:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+110455 (secure 0 0 0) 0x7f075c00c670 con 0x7f07640dbbc0 2024-04-17T22:30:59.838 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.851 7f076ac56700 1 -- 172.21.15.102:0/1450461133 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f07640f0800 con 0x7f07640dbbc0 2024-04-17T22:30:59.838 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.852 7f0760ff9700 1 -- 172.21.15.102:0/1450461133 <== mon.1 v2:172.21.15.102:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v17) v1 ==== 104+0+11 (secure 0 0 0) 0x7f0764114420 con 0x7f07640dbbc0 2024-04-17T22:30:59.839 INFO:teuthology.orchestra.run.smithi102.stdout:pg_num: 64 2024-04-17T22:30:59.840 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.853 7f076ac56700 1 -- 172.21.15.102:0/1450461133 >> v1:172.21.15.102:6816/42360 conn(0x7f07480145a0 legacy=0x7f0748016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:30:59.840 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.853 7f076ac56700 1 -- 172.21.15.102:0/1450461133 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f07640dbbc0 msgr2=0x7f07640d58e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:30:59.841 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.853 7f076ac56700 1 --2- 172.21.15.102:0/1450461133 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f07640dbbc0 0x7f07640d58e0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 rx=0x7f07640c2580 tx=0x7f075c00eda0).stop 2024-04-17T22:30:59.841 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.853 7f076ac56700 1 -- 172.21.15.102:0/1450461133 shutdown_connections 2024-04-17T22:30:59.841 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.853 7f076ac56700 1 --2- 172.21.15.102:0/1450461133 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f07640dbbc0 0x7f07640d58e0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:30:59.841 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.853 7f076ac56700 1 --2- 172.21.15.102:0/1450461133 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f07640e0420 0x7f07640db140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:30:59.841 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.853 7f076ac56700 1 --2- 172.21.15.102:0/1450461133 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f07640db770 0x7f07640d52b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:30:59.841 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.853 7f076ac56700 1 -- 172.21.15.102:0/1450461133 >> 172.21.15.102:0/1450461133 conn(0x7f07640c6af0 msgr2=0x7f07640d65c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:30:59.841 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.853 7f076ac56700 1 -- 172.21.15.102:0/1450461133 shutdown_connections 2024-04-17T22:30:59.842 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.853 7f076ac56700 1 -- 172.21.15.102:0/1450461133 wait complete. 2024-04-17T22:30:59.850 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-17T22:30:59.983 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.992 7f52d8a6a700 1 Processor -- start 2024-04-17T22:30:59.983 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.992 7f52d8a6a700 1 -- start start 2024-04-17T22:30:59.984 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.992 7f52d8a6a700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f52d40f5840 0x7f52d410b010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:30:59.984 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.992 7f52d8a6a700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f52d410b640 0x7f52d419faf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:30:59.984 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.992 7f52d8a6a700 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52d4103b40 con 0x7f52d410ba90 2024-04-17T22:30:59.984 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.992 7f52d8a6a700 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52d410bee0 con 0x7f52d410b640 2024-04-17T22:30:59.984 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.992 7f52d8a6a700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f52d40fa370 con 0x7f52d40f5840 2024-04-17T22:30:59.984 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.993 7f52d1850700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f52d40f5840 0x7f52d410b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:30:59.984 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.993 7f52cbfff700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f52d410b640 0x7f52d419faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:30:59.985 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.993 7f52d1850700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f52d40f5840 0x7f52d410b010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:59316/0 (socket says 172.21.15.102:59316) 2024-04-17T22:30:59.985 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.993 7f52d1850700 1 -- 172.21.15.102:0/33000903 learned_addr learned my addr 172.21.15.102:0/33000903 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:30:59.985 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.993 7f52d2051700 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f52d410ba90 0x7f52d41a0120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:41016/0 (socket says 172.21.15.102:41016) 2024-04-17T22:30:59.985 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.993 7f52d1850700 1 -- 172.21.15.102:0/33000903 >> v2:172.21.15.102:3300/0 conn(0x7f52d410b640 msgr2=0x7f52d419faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:30:59.985 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.993 7f52d1850700 1 --2- 172.21.15.102:0/33000903 >> v2:172.21.15.102:3300/0 conn(0x7f52d410b640 0x7f52d419faf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:30:59.985 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.993 7f52d1850700 1 -- 172.21.15.102:0/33000903 >> v1:172.21.15.102:6789/0 conn(0x7f52d410ba90 legacy=0x7f52d41a0120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:30:59.986 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.993 7f52d104f700 1 -- 172.21.15.102:0/33000903 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2679746380 0 0) 0x7f52cc001490 con 0x7f52d410ba90 2024-04-17T22:30:59.986 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.993 7f52d1850700 1 -- 172.21.15.102:0/33000903 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52d41a0920 con 0x7f52d40f5840 2024-04-17T22:30:59.986 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.993 7f52cbfff700 1 --2- 172.21.15.102:0/33000903 >> v2:172.21.15.102:3300/0 conn(0x7f52d410b640 0x7f52d419faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:30:59.986 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.993 7f52d1850700 1 --2- 172.21.15.102:0/33000903 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f52d40f5840 0x7f52d410b010 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 rx=0x7f52c4002f70 tx=0x7f52c40099d0).ready entity=mon.2 client_cookie=9166e96c31341de8 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:30:59.986 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.993 7f52d104f700 1 -- 172.21.15.102:0/33000903 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f52d41a0920 con 0x7f52d40f5840 2024-04-17T22:30:59.986 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.993 7f52d104f700 1 -- 172.21.15.102:0/33000903 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52c400f510 con 0x7f52d40f5840 2024-04-17T22:30:59.987 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.993 7f52d104f700 1 -- 172.21.15.102:0/33000903 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f52c400f6f0 con 0x7f52d40f5840 2024-04-17T22:30:59.987 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.993 7f52d8a6a700 1 -- 172.21.15.102:0/33000903 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f52d40f5840 msgr2=0x7f52d410b010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:30:59.987 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.993 7f52d8a6a700 1 --2- 172.21.15.102:0/33000903 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f52d40f5840 0x7f52d410b010 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 rx=0x7f52c4002f70 tx=0x7f52c40099d0).stop 2024-04-17T22:30:59.987 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.994 7f52d8a6a700 1 -- 172.21.15.102:0/33000903 shutdown_connections 2024-04-17T22:30:59.987 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.994 7f52d8a6a700 1 --2- 172.21.15.102:0/33000903 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f52d40f5840 0x7f52d410b010 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:30:59.987 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.994 7f52d8a6a700 1 --2- 172.21.15.102:0/33000903 >> v2:172.21.15.102:3300/0 conn(0x7f52d410b640 0x7f52d419faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:30:59.988 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.994 7f52d8a6a700 1 -- 172.21.15.102:0/33000903 >> 172.21.15.102:0/33000903 conn(0x7f52d41138a0 msgr2=0x7f52d40f4f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:30:59.988 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.994 7f52d8a6a700 1 -- 172.21.15.102:0/33000903 shutdown_connections 2024-04-17T22:30:59.988 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:30:59.994 7f52d8a6a700 1 -- 172.21.15.102:0/33000903 wait complete. 2024-04-17T22:31:00.006 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.016 7f52d8a6a700 1 Processor -- start 2024-04-17T22:31:00.006 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.016 7f52d8a6a700 1 -- start start 2024-04-17T22:31:00.007 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.016 7f52d8a6a700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f52d41a1320 0x7f52d40d4f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:00.007 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.016 7f52d8a6a700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f52d40d5540 0x7f52d41afbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:00.007 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.016 7f52d8a6a700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f52d40d5990 0x7f52d41b0220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:00.007 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.016 7f52d8a6a700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52d41a1130 con 0x7f52d40d5990 2024-04-17T22:31:00.007 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.016 7f52d8a6a700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52d40c3350 con 0x7f52d41a1320 2024-04-17T22:31:00.007 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.016 7f52d8a6a700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f52d41a1770 con 0x7f52d40d5540 2024-04-17T22:31:00.008 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.017 7f52d1850700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f52d41a1320 0x7f52d40d4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:00.008 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.017 7f52cbfff700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f52d40d5540 0x7f52d41afbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:00.008 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.017 7f52d2051700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f52d40d5990 0x7f52d41b0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:00.008 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.017 7f52d1850700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f52d41a1320 0x7f52d40d4f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:48980/0 (socket says 172.21.15.102:48980) 2024-04-17T22:31:00.008 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.017 7f52cbfff700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f52d40d5540 0x7f52d41afbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:59326/0 (socket says 172.21.15.102:59326) 2024-04-17T22:31:00.008 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.017 7f52d1850700 1 -- 172.21.15.102:0/3145366995 learned_addr learned my addr 172.21.15.102:0/3145366995 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:31:00.009 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.017 7f52cbfff700 1 -- 172.21.15.102:0/3145366995 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f52d41a1320 msgr2=0x7f52d40d4f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:00.009 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.017 7f52cbfff700 1 --2- 172.21.15.102:0/3145366995 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f52d41a1320 0x7f52d40d4f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:00.009 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.017 7f52cbfff700 1 -- 172.21.15.102:0/3145366995 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f52d40d5990 msgr2=0x7f52d41b0220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:00.009 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.017 7f52cbfff700 1 --2- 172.21.15.102:0/3145366995 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f52d40d5990 0x7f52d41b0220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:00.009 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.017 7f52cbfff700 1 -- 172.21.15.102:0/3145366995 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52d41b0850 con 0x7f52d40d5540 2024-04-17T22:31:00.009 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.017 7f52d1850700 1 --2- 172.21.15.102:0/3145366995 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f52d41a1320 0x7f52d40d4f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:00.009 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.017 7f52cbfff700 1 --2- 172.21.15.102:0/3145366995 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f52d40d5540 0x7f52d41afbf0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 rx=0x7f52d40d63c0 tx=0x7f52bc010f00).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:00.010 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.017 7f52caffd700 1 -- 172.21.15.102:0/3145366995 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f52d41b0850 con 0x7f52d40d5540 2024-04-17T22:31:00.010 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.017 7f52caffd700 1 -- 172.21.15.102:0/3145366995 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52bc00ed50 con 0x7f52d40d5540 2024-04-17T22:31:00.010 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.017 7f52d8a6a700 1 -- 172.21.15.102:0/3145366995 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52d41b73f0 con 0x7f52d40d5540 2024-04-17T22:31:00.010 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.017 7f52caffd700 1 -- 172.21.15.102:0/3145366995 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f52bc01a660 con 0x7f52d40d5540 2024-04-17T22:31:00.010 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.017 7f52d8a6a700 1 -- 172.21.15.102:0/3145366995 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52d41b77d0 con 0x7f52d40d5540 2024-04-17T22:31:00.010 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.018 7f52caffd700 1 -- 172.21.15.102:0/3145366995 <== mon.2 v2:172.21.15.102:3302/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7f52bc011450 con 0x7f52d40d5540 2024-04-17T22:31:00.011 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.018 7f52caffd700 1 -- 172.21.15.102:0/3145366995 <== mon.2 v2:172.21.15.102:3302/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3194+0+0 (secure 0 0 0) 0x7f52bc0178c0 con 0x7f52d40d5540 2024-04-17T22:31:00.011 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.018 7f52d8a6a700 1 -- 172.21.15.102:0/3145366995 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52d40f0800 con 0x7f52d40d5540 2024-04-17T22:31:00.011 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.022 7f52caffd700 1 -- 172.21.15.102:0/3145366995 <== mon.2 v2:172.21.15.102:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+110455 (secure 0 0 0) 0x7f52bc00e950 con 0x7f52d40d5540 2024-04-17T22:31:00.114 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.127 7f52d8a6a700 1 -- 172.21.15.102:0/3145366995 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f52d41aa310 con 0x7f52d40d5540 2024-04-17T22:31:00.114 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.128 7f52caffd700 1 -- 172.21.15.102:0/3145366995 <== mon.2 v2:172.21.15.102:3302/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v17) v1 ==== 100+0+11 (secure 0 0 0) 0x7f52bc000c30 con 0x7f52d40d5540 2024-04-17T22:31:00.114 INFO:teuthology.orchestra.run.smithi102.stdout:pg_num: 64 2024-04-17T22:31:00.116 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.129 7f52d8a6a700 1 -- 172.21.15.102:0/3145366995 >> v1:172.21.15.102:6816/42360 conn(0x7f52b80145a0 legacy=0x7f52b8016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:00.116 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.129 7f52d8a6a700 1 -- 172.21.15.102:0/3145366995 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f52d40d5540 msgr2=0x7f52d41afbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:00.116 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.129 7f52d8a6a700 1 --2- 172.21.15.102:0/3145366995 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f52d40d5540 0x7f52d41afbf0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 rx=0x7f52d40d63c0 tx=0x7f52bc010f00).stop 2024-04-17T22:31:00.116 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.129 7f52d8a6a700 1 -- 172.21.15.102:0/3145366995 shutdown_connections 2024-04-17T22:31:00.117 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.129 7f52d8a6a700 1 --2- 172.21.15.102:0/3145366995 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f52d40d5990 0x7f52d41b0220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:00.117 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.129 7f52d8a6a700 1 --2- 172.21.15.102:0/3145366995 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f52d41a1320 0x7f52d40d4f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:00.117 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.129 7f52d8a6a700 1 --2- 172.21.15.102:0/3145366995 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f52d40d5540 0x7f52d41afbf0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:00.117 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.129 7f52d8a6a700 1 -- 172.21.15.102:0/3145366995 >> 172.21.15.102:0/3145366995 conn(0x7f52d40e0fb0 msgr2=0x7f52d40dff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:00.117 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.129 7f52d8a6a700 1 -- 172.21.15.102:0/3145366995 shutdown_connections 2024-04-17T22:31:00.117 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.129 7f52d8a6a700 1 -- 172.21.15.102:0/3145366995 wait complete. 2024-04-17T22:31:00.130 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T22:31:00.256 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.268 7fe9de97c700 1 Processor -- start 2024-04-17T22:31:00.257 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.268 7fe9de97c700 1 -- start start 2024-04-17T22:31:00.257 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.268 7fe9de97c700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fe9d80f57d0 0x7fe9d810afa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:00.257 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.268 7fe9de97c700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fe9d810ba20 0x7fe9d81a0280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:00.257 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.268 7fe9de97c700 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9d8103ad0 con 0x7fe9d810b5d0 2024-04-17T22:31:00.257 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.268 7fe9de97c700 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9d810be70 con 0x7fe9d80f57d0 2024-04-17T22:31:00.257 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.268 7fe9de97c700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe9d80fa300 con 0x7fe9d810ba20 2024-04-17T22:31:00.257 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.269 7fe9d75eb700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fe9d80f57d0 0x7fe9d810afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:00.257 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.269 7fe9d7dec700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fe9d810ba20 0x7fe9d81a0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:00.257 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.269 7fe9d6dea700 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7fe9d810b5d0 0x7fe9d819fa80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:41032/0 (socket says 172.21.15.102:41032) 2024-04-17T22:31:00.257 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.269 7fe9d6dea700 1 -- 172.21.15.102:0/2501535534 learned_addr learned my addr 172.21.15.102:0/2501535534 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-17T22:31:00.257 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.269 7fe9d75eb700 1 -- 172.21.15.102:0/2501535534 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fe9d810ba20 msgr2=0x7fe9d81a0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:31:00.257 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.269 7fe9d75eb700 1 --2- 172.21.15.102:0/2501535534 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fe9d810ba20 0x7fe9d81a0280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:00.257 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.269 7fe9d75eb700 1 -- 172.21.15.102:0/2501535534 >> v1:172.21.15.102:6789/0 conn(0x7fe9d810b5d0 legacy=0x7fe9d819fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:00.258 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.269 7fe9d75eb700 1 -- 172.21.15.102:0/2501535534 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9d81a08b0 con 0x7fe9d80f57d0 2024-04-17T22:31:00.258 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.269 7fe9d65e9700 1 -- 172.21.15.102:0/2501535534 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1898191837 0 0) 0x7fe9c8001110 con 0x7fe9d810b5d0 2024-04-17T22:31:00.258 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.269 7fe9d7dec700 1 --2- 172.21.15.102:0/2501535534 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fe9d810ba20 0x7fe9d81a0280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:00.258 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.269 7fe9d75eb700 1 --2- 172.21.15.102:0/2501535534 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe9d80f57d0 0x7fe9d810afa0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 rx=0x7fe9d8103cf0 tx=0x7fe9cc00ba20).ready entity=mon.0 client_cookie=cc23f65969786442 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:00.258 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.269 7fe9d65e9700 1 -- 172.21.15.102:0/2501535534 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe9d81a08b0 con 0x7fe9d80f57d0 2024-04-17T22:31:00.258 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.269 7fe9d65e9700 1 -- 172.21.15.102:0/2501535534 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9cc00d5c0 con 0x7fe9d80f57d0 2024-04-17T22:31:00.258 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.269 7fe9d65e9700 1 -- 172.21.15.102:0/2501535534 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe9cc00d990 con 0x7fe9d80f57d0 2024-04-17T22:31:00.258 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.269 7fe9de97c700 1 -- 172.21.15.102:0/2501535534 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe9d80f57d0 msgr2=0x7fe9d810afa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:00.258 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.269 7fe9de97c700 1 --2- 172.21.15.102:0/2501535534 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe9d80f57d0 0x7fe9d810afa0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 rx=0x7fe9d8103cf0 tx=0x7fe9cc00ba20).stop 2024-04-17T22:31:00.258 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.270 7fe9de97c700 1 -- 172.21.15.102:0/2501535534 shutdown_connections 2024-04-17T22:31:00.258 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.270 7fe9de97c700 1 --2- 172.21.15.102:0/2501535534 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fe9d810ba20 0x7fe9d81a0280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:00.258 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.270 7fe9de97c700 1 --2- 172.21.15.102:0/2501535534 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe9d80f57d0 0x7fe9d810afa0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:00.258 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.270 7fe9de97c700 1 -- 172.21.15.102:0/2501535534 >> 172.21.15.102:0/2501535534 conn(0x7fe9d8113830 msgr2=0x7fe9d80f4ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:00.259 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.270 7fe9de97c700 1 -- 172.21.15.102:0/2501535534 shutdown_connections 2024-04-17T22:31:00.259 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.270 7fe9de97c700 1 -- 172.21.15.102:0/2501535534 wait complete. 2024-04-17T22:31:00.279 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.292 7fe9de97c700 1 Processor -- start 2024-04-17T22:31:00.279 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.293 7fe9de97c700 1 -- start start 2024-04-17T22:31:00.281 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.293 7fe9de97c700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe9d80d0860 0x7fe9d80e0180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:00.281 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.293 7fe9de97c700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe9d80e07b0 0x7fe9d80ce1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:00.281 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.293 7fe9de97c700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fe9d80e0c00 0x7fe9d80ce810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:00.281 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.293 7fe9de97c700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9d80d1330 con 0x7fe9d80d0860 2024-04-17T22:31:00.281 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.293 7fe9de97c700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9d80d0cb0 con 0x7fe9d80e07b0 2024-04-17T22:31:00.281 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.293 7fe9de97c700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe9d80cee40 con 0x7fe9d80e0c00 2024-04-17T22:31:00.281 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.293 7fe9d75eb700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe9d80d0860 0x7fe9d80e0180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:00.282 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.293 7fe9d7dec700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fe9d80e0c00 0x7fe9d80ce810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:00.282 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.293 7fe9d7dec700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fe9d80e0c00 0x7fe9d80ce810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:59344/0 (socket says 172.21.15.102:59344) 2024-04-17T22:31:00.282 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.293 7fe9d7dec700 1 -- 172.21.15.102:0/1602123634 learned_addr learned my addr 172.21.15.102:0/1602123634 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:31:00.282 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.293 7fe9d75eb700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe9d80d0860 0x7fe9d80e0180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:40594/0 (socket says 172.21.15.102:40594) 2024-04-17T22:31:00.282 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.293 7fe9d6dea700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe9d80e07b0 0x7fe9d80ce1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:00.282 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.293 7fe9d6dea700 1 -- 172.21.15.102:0/1602123634 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fe9d80e0c00 msgr2=0x7fe9d80ce810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:00.282 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.293 7fe9d6dea700 1 --2- 172.21.15.102:0/1602123634 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fe9d80e0c00 0x7fe9d80ce810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:00.282 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.293 7fe9d6dea700 1 -- 172.21.15.102:0/1602123634 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe9d80d0860 msgr2=0x7fe9d80e0180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:00.282 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.293 7fe9d6dea700 1 --2- 172.21.15.102:0/1602123634 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe9d80d0860 0x7fe9d80e0180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:00.282 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.293 7fe9d6dea700 1 -- 172.21.15.102:0/1602123634 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9d81bb440 con 0x7fe9d80e07b0 2024-04-17T22:31:00.282 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.293 7fe9d7dec700 1 --2- 172.21.15.102:0/1602123634 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fe9d80e0c00 0x7fe9d80ce810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:00.282 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.293 7fe9d75eb700 1 --2- 172.21.15.102:0/1602123634 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe9d80d0860 0x7fe9d80e0180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:00.282 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.293 7fe9d6dea700 1 --2- 172.21.15.102:0/1602123634 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe9d80e07b0 0x7fe9d80ce1e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0x7fe9c8000bc0 tx=0x7fe9c800c800).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:00.283 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.294 7fe9d4de6700 1 -- 172.21.15.102:0/1602123634 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe9d81bb440 con 0x7fe9d80e07b0 2024-04-17T22:31:00.283 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.294 7fe9d4de6700 1 -- 172.21.15.102:0/1602123634 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9c80070d0 con 0x7fe9d80e07b0 2024-04-17T22:31:00.283 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.294 7fe9d4de6700 1 -- 172.21.15.102:0/1602123634 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe9c8005620 con 0x7fe9d80e07b0 2024-04-17T22:31:00.283 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.294 7fe9de97c700 1 -- 172.21.15.102:0/1602123634 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9d81bb640 con 0x7fe9d80e07b0 2024-04-17T22:31:00.283 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.294 7fe9de97c700 1 -- 172.21.15.102:0/1602123634 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9d81bba20 con 0x7fe9d80e07b0 2024-04-17T22:31:00.283 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.294 7fe9d4de6700 1 -- 172.21.15.102:0/1602123634 <== mon.1 v2:172.21.15.102:3301/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7fe9c8005820 con 0x7fe9d80e07b0 2024-04-17T22:31:00.283 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.294 7fe9d4de6700 1 -- 172.21.15.102:0/1602123634 <== mon.1 v2:172.21.15.102:3301/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3194+0+0 (secure 0 0 0) 0x7fe9c8014730 con 0x7fe9d80e07b0 2024-04-17T22:31:00.283 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.295 7fe9de97c700 1 -- 172.21.15.102:0/1602123634 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9d80f0790 con 0x7fe9d80e07b0 2024-04-17T22:31:00.284 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.298 7fe9d4de6700 1 -- 172.21.15.102:0/1602123634 <== mon.1 v2:172.21.15.102:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+110455 (secure 0 0 0) 0x7fe9c800cdf0 con 0x7fe9d80e07b0 2024-04-17T22:31:00.420 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.434 7fe9de97c700 1 -- 172.21.15.102:0/1602123634 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe9d81ae470 con 0x7fe9d80e07b0 2024-04-17T22:31:00.420 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.434 7fe9d4de6700 1 -- 172.21.15.102:0/1602123634 <== mon.1 v2:172.21.15.102:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) v1 ==== 93+0+1704 (secure 0 0 0) 0x7fe9c8000c30 con 0x7fe9d80e07b0 2024-04-17T22:31:00.421 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-17T22:31:00.421 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":5,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[],"filesystems":[{"mdsmap":{"epoch":5,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17 22:29:07.787730","modified":"2024-04-17 22:29:09.795249","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"-1 (unspecified)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4177},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4177":{"gid":4177,"name":"a","rank":0,"incarnation":3,"state":"up:active","state_seq":4,"addr":"172.21.15.102:6817/1944542598","addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6817","nonce":1944542598}]},"export_targets":[],"features":4611087854035861503,"flags":0}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-04-17T22:31:00.422 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.435 7fe9de97c700 1 -- 172.21.15.102:0/1602123634 >> v1:172.21.15.102:6816/42360 conn(0x7fe9b80145a0 legacy=0x7fe9b8016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:00.422 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.435 7fe9de97c700 1 -- 172.21.15.102:0/1602123634 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe9d80e07b0 msgr2=0x7fe9d80ce1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:00.422 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.435 7fe9de97c700 1 --2- 172.21.15.102:0/1602123634 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe9d80e07b0 0x7fe9d80ce1e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0x7fe9c8000bc0 tx=0x7fe9c800c800).stop 2024-04-17T22:31:00.422 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.435 7fe9de97c700 1 -- 172.21.15.102:0/1602123634 shutdown_connections 2024-04-17T22:31:00.422 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.435 7fe9de97c700 1 --2- 172.21.15.102:0/1602123634 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fe9d80e0c00 0x7fe9d80ce810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:00.422 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.435 7fe9de97c700 1 --2- 172.21.15.102:0/1602123634 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe9d80d0860 0x7fe9d80e0180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:00.422 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.435 7fe9de97c700 1 --2- 172.21.15.102:0/1602123634 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe9d80e07b0 0x7fe9d80ce1e0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:00.422 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.435 7fe9de97c700 1 -- 172.21.15.102:0/1602123634 >> 172.21.15.102:0/1602123634 conn(0x7fe9d80df660 msgr2=0x7fe9d80d1bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:00.422 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.436 7fe9de97c700 1 -- 172.21.15.102:0/1602123634 shutdown_connections 2024-04-17T22:31:00.423 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.436 7fe9de97c700 1 -- 172.21.15.102:0/1602123634 wait complete. 2024-04-17T22:31:00.423 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 5 2024-04-17T22:31:00.432 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T22:31:00.559 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.571 7fd3c9d8e700 1 Processor -- start 2024-04-17T22:31:00.560 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.571 7fd3c9d8e700 1 -- start start 2024-04-17T22:31:00.560 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.571 7fd3c9d8e700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fd3c410c910 0x7fd3c41a6300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:00.560 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.572 7fd3c9d8e700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd3c41a6930 0x7fd3c41a8d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:00.560 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.572 7fd3c9d8e700 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3c41a1c10 con 0x7fd3c40f5770 2024-04-17T22:31:00.560 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.572 7fd3c9d8e700 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3c40f93e0 con 0x7fd3c410c910 2024-04-17T22:31:00.560 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.572 7fd3c9d8e700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3c410cd60 con 0x7fd3c41a6930 2024-04-17T22:31:00.561 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.572 7fd3c22b1700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fd3c410c910 0x7fd3c41a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:00.561 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.572 7fd3c2ab2700 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7fd3c40f5770 0x7fd3c410c110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:41036/0 (socket says 172.21.15.102:41036) 2024-04-17T22:31:00.561 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.572 7fd3c22b1700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fd3c410c910 0x7fd3c41a6300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:40600/0 (socket says 172.21.15.102:40600) 2024-04-17T22:31:00.561 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.572 7fd3c2ab2700 1 -- 172.21.15.102:0/158911791 learned_addr learned my addr 172.21.15.102:0/158911791 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-17T22:31:00.561 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.572 7fd3c32b3700 1 --2- 172.21.15.102:0/158911791 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd3c41a6930 0x7fd3c41a8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:00.561 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.572 7fd3c22b1700 1 -- 172.21.15.102:0/158911791 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd3c41a6930 msgr2=0x7fd3c41a8d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:31:00.561 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.572 7fd3c22b1700 1 --2- 172.21.15.102:0/158911791 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd3c41a6930 0x7fd3c41a8d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:00.561 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.572 7fd3c22b1700 1 -- 172.21.15.102:0/158911791 >> v1:172.21.15.102:6789/0 conn(0x7fd3c40f5770 legacy=0x7fd3c410c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:00.561 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.572 7fd3c22b1700 1 -- 172.21.15.102:0/158911791 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3c41aa3d0 con 0x7fd3c410c910 2024-04-17T22:31:00.561 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.572 7fd3c1ab0700 1 -- 172.21.15.102:0/158911791 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1821815945 0 0) 0x7fd3c41a1c10 con 0x7fd3c40f5770 2024-04-17T22:31:00.561 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.572 7fd3c32b3700 1 --2- 172.21.15.102:0/158911791 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd3c41a6930 0x7fd3c41a8d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:00.561 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.572 7fd3c22b1700 1 --2- 172.21.15.102:0/158911791 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd3c410c910 0x7fd3c41a6300 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 rx=0x7fd3ac002f70 tx=0x7fd3ac00b9f0).ready entity=mon.0 client_cookie=45c1f1f6e8e262e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:00.561 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.572 7fd3c1ab0700 1 -- 172.21.15.102:0/158911791 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd3c41aa3d0 con 0x7fd3c410c910 2024-04-17T22:31:00.562 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.572 7fd3c1ab0700 1 -- 172.21.15.102:0/158911791 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3ac00d5c0 con 0x7fd3c410c910 2024-04-17T22:31:00.562 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.572 7fd3c1ab0700 1 -- 172.21.15.102:0/158911791 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd3ac00d990 con 0x7fd3c410c910 2024-04-17T22:31:00.562 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.572 7fd3c9d8e700 1 -- 172.21.15.102:0/158911791 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd3c410c910 msgr2=0x7fd3c41a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:00.562 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.572 7fd3c9d8e700 1 --2- 172.21.15.102:0/158911791 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd3c410c910 0x7fd3c41a6300 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 rx=0x7fd3ac002f70 tx=0x7fd3ac00b9f0).stop 2024-04-17T22:31:00.562 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.573 7fd3c9d8e700 1 -- 172.21.15.102:0/158911791 shutdown_connections 2024-04-17T22:31:00.562 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.573 7fd3c9d8e700 1 --2- 172.21.15.102:0/158911791 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd3c41a6930 0x7fd3c41a8d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:00.562 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.573 7fd3c9d8e700 1 --2- 172.21.15.102:0/158911791 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd3c410c910 0x7fd3c41a6300 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:00.562 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.573 7fd3c9d8e700 1 -- 172.21.15.102:0/158911791 >> 172.21.15.102:0/158911791 conn(0x7fd3c41138c0 msgr2=0x7fd3c40f4f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:00.562 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.573 7fd3c9d8e700 1 -- 172.21.15.102:0/158911791 shutdown_connections 2024-04-17T22:31:00.563 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.573 7fd3c9d8e700 1 -- 172.21.15.102:0/158911791 wait complete. 2024-04-17T22:31:00.583 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.595 7fd3c9d8e700 1 Processor -- start 2024-04-17T22:31:00.583 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.595 7fd3c9d8e700 1 -- start start 2024-04-17T22:31:00.583 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.596 7fd3c9d8e700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd3c40dd390 0x7fd3c40d51d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:00.584 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.596 7fd3c9d8e700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd3c40d5800 0x7fd3c41b20f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:00.584 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.596 7fd3c9d8e700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd3c40d5c50 0x7fd3c41b2720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:00.584 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.596 7fd3c9d8e700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3c4089da0 con 0x7fd3c40d5800 2024-04-17T22:31:00.584 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.596 7fd3c9d8e700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3c40dcdc0 con 0x7fd3c40d5c50 2024-04-17T22:31:00.584 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.596 7fd3c9d8e700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd3c40dc770 con 0x7fd3c40dd390 2024-04-17T22:31:00.584 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.596 7fd3c32b3700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd3c40d5c50 0x7fd3c41b2720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:00.584 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.596 7fd3c2ab2700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd3c40dd390 0x7fd3c40d51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:00.584 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.596 7fd3c22b1700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd3c40d5800 0x7fd3c41b20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:00.584 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.596 7fd3c32b3700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd3c40d5c50 0x7fd3c41b2720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:49012/0 (socket says 172.21.15.102:49012) 2024-04-17T22:31:00.584 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.596 7fd3c32b3700 1 -- 172.21.15.102:0/3281213581 learned_addr learned my addr 172.21.15.102:0/3281213581 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:31:00.584 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.596 7fd3c22b1700 1 -- 172.21.15.102:0/3281213581 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd3c40dd390 msgr2=0x7fd3c40d51d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:00.584 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.596 7fd3c22b1700 1 --2- 172.21.15.102:0/3281213581 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd3c40dd390 0x7fd3c40d51d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:00.584 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.596 7fd3c22b1700 1 -- 172.21.15.102:0/3281213581 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd3c40d5c50 msgr2=0x7fd3c41b2720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:00.585 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.596 7fd3c22b1700 1 --2- 172.21.15.102:0/3281213581 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd3c40d5c50 0x7fd3c41b2720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:00.585 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.596 7fd3c22b1700 1 -- 172.21.15.102:0/3281213581 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3c41b2d50 con 0x7fd3c40d5800 2024-04-17T22:31:00.585 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.596 7fd3c22b1700 1 --2- 172.21.15.102:0/3281213581 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd3c40d5800 0x7fd3c41b20f0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 rx=0x7fd3c40d7470 tx=0x7fd3ac0068c0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:00.585 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.596 7fd3bbfff700 1 -- 172.21.15.102:0/3281213581 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd3c41b2d50 con 0x7fd3c40d5800 2024-04-17T22:31:00.585 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.596 7fd3bbfff700 1 -- 172.21.15.102:0/3281213581 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3ac011940 con 0x7fd3c40d5800 2024-04-17T22:31:00.585 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.596 7fd3c9d8e700 1 -- 172.21.15.102:0/3281213581 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3c41b3010 con 0x7fd3c40d5800 2024-04-17T22:31:00.585 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.596 7fd3bbfff700 1 -- 172.21.15.102:0/3281213581 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd3ac017040 con 0x7fd3c40d5800 2024-04-17T22:31:00.585 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.596 7fd3c9d8e700 1 -- 172.21.15.102:0/3281213581 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3c41b9a40 con 0x7fd3c40d5800 2024-04-17T22:31:00.585 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.597 7fd3bbfff700 1 -- 172.21.15.102:0/3281213581 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7fd3ac011d10 con 0x7fd3c40d5800 2024-04-17T22:31:00.585 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.597 7fd3bbfff700 1 -- 172.21.15.102:0/3281213581 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3194+0+0 (secure 0 0 0) 0x7fd3ac018070 con 0x7fd3c40d5800 2024-04-17T22:31:00.585 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.597 7fd3c9d8e700 1 -- 172.21.15.102:0/3281213581 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3c40f0820 con 0x7fd3c40d5800 2024-04-17T22:31:00.586 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.600 7fd3bbfff700 1 -- 172.21.15.102:0/3281213581 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+110455 (secure 0 0 0) 0x7fd3ac00d3b0 con 0x7fd3c40d5800 2024-04-17T22:31:00.691 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.705 7fd3c9d8e700 1 -- 172.21.15.102:0/3281213581 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd3c41ac830 con 0x7fd3c40d5800 2024-04-17T22:31:00.691 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.705 7fd3bbfff700 1 -- 172.21.15.102:0/3281213581 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) v1 ==== 74+0+8764 (secure 0 0 0) 0x7fd3ac011730 con 0x7fd3c40d5800 2024-04-17T22:31:00.692 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-17T22:31:00.692 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":17,"fsid":"cb63e789-091d-49b1-b120-55f9db526871","created":"2024-04-17 22:28:48.209635","modified":"2024-04-17 22:29:23.257387","last_up_change":"2024-04-17 22:28:54.236215","last_in_change":"2024-04-17 22:28:54.236215","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":4,"require_min_compat_client":"nautilus","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-04-17 22:28:57.478545","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-17 22:29:06.144708","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-17 22:29:06.748924","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3351ef85-ade4-4b90-a8f4-213f4e00ba01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6812","nonce":42955}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6813","nonce":42955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6815","nonce":42955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6814","nonce":42955}]},"public_addr":"172.21.15.102:6812/42955","cluster_addr":"172.21.15.102:6813/42955","heartbeat_back_addr":"172.21.15.102:6815/42955","heartbeat_front_addr":"172.21.15.102:6814/42955","state":["exists","up"]},{"osd":1,"uuid":"04f67eac-f36b-48f5-982d-4cfc776596d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6808","nonce":42958}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6809","nonce":42958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6811","nonce":42958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6810","nonce":42958}]},"public_addr":"172.21.15.102:6808/42958","cluster_addr":"172.21.15.102:6809/42958","heartbeat_back_addr":"172.21.15.102:6811/42958","heartbeat_front_addr":"172.21.15.102:6810/42958","state":["exists","up"]},{"osd":2,"uuid":"d524204b-70b2-40a8-8f73-f9e7d44318af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6804","nonce":42954}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6805","nonce":42954}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6807","nonce":42954}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6806","nonce":42954}]},"public_addr":"172.21.15.102:6804/42954","cluster_addr":"172.21.15.102:6805/42954","heartbeat_back_addr":"172.21.15.102:6807/42954","heartbeat_front_addr":"172.21.15.102:6806/42954","state":["exists","up"]},{"osd":3,"uuid":"4fc9d20c-1ad0-484f-aafa-2c9a7574245b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6800","nonce":42953}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6801","nonce":42953}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6803","nonce":42953}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6802","nonce":42953}]},"public_addr":"172.21.15.102:6800/42953","cluster_addr":"172.21.15.102:6801/42953","heartbeat_back_addr":"172.21.15.102:6803/42953","heartbeat_front_addr":"172.21.15.102:6802/42953","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2024-04-17T22:31:00.693 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.707 7fd3c9d8e700 1 -- 172.21.15.102:0/3281213581 >> v1:172.21.15.102:6816/42360 conn(0x7fd3a80145a0 legacy=0x7fd3a8016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:00.693 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.707 7fd3c9d8e700 1 -- 172.21.15.102:0/3281213581 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd3c40d5800 msgr2=0x7fd3c41b20f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:00.693 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.707 7fd3c9d8e700 1 --2- 172.21.15.102:0/3281213581 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd3c40d5800 0x7fd3c41b20f0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 rx=0x7fd3c40d7470 tx=0x7fd3ac0068c0).stop 2024-04-17T22:31:00.693 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.707 7fd3c9d8e700 1 -- 172.21.15.102:0/3281213581 shutdown_connections 2024-04-17T22:31:00.693 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.707 7fd3c9d8e700 1 --2- 172.21.15.102:0/3281213581 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd3c40d5c50 0x7fd3c41b2720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:00.693 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.707 7fd3c9d8e700 1 --2- 172.21.15.102:0/3281213581 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd3c40dd390 0x7fd3c40d51d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:00.693 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.707 7fd3c9d8e700 1 --2- 172.21.15.102:0/3281213581 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd3c40d5800 0x7fd3c41b20f0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:00.694 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.707 7fd3c9d8e700 1 -- 172.21.15.102:0/3281213581 >> 172.21.15.102:0/3281213581 conn(0x7fd3c40c6af0 msgr2=0x7fd3c40d63f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:00.694 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.707 7fd3c9d8e700 1 -- 172.21.15.102:0/3281213581 shutdown_connections 2024-04-17T22:31:00.694 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.707 7fd3c9d8e700 1 -- 172.21.15.102:0/3281213581 wait complete. 2024-04-17T22:31:00.704 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T22:31:00.831 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.842 7f709dc8c700 1 Processor -- start 2024-04-17T22:31:00.831 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.842 7f709dc8c700 1 -- start start 2024-04-17T22:31:00.831 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.843 7f709dc8c700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f70980f57d0 0x7f709810afa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:00.831 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.843 7f709dc8c700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f709810ba20 0x7f70981a0280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:00.831 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.843 7f709dc8c700 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7098103ad0 con 0x7f709810b5d0 2024-04-17T22:31:00.831 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.843 7f709dc8c700 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f709810be70 con 0x7f70980f57d0 2024-04-17T22:31:00.831 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.843 7f709dc8c700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70980fa300 con 0x7f709810ba20 2024-04-17T22:31:00.831 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.843 7f7096a70700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f70980f57d0 0x7f709810afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:00.832 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.843 7f709626f700 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f709810b5d0 0x7f709819fa80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:41048/0 (socket says 172.21.15.102:41048) 2024-04-17T22:31:00.832 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.843 7f7096a70700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f70980f57d0 0x7f709810afa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:40606/0 (socket says 172.21.15.102:40606) 2024-04-17T22:31:00.832 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.843 7f7096a70700 1 -- 172.21.15.102:0/1110617950 learned_addr learned my addr 172.21.15.102:0/1110617950 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:31:00.832 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.843 7f7097271700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f709810ba20 0x7f70981a0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:00.832 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.843 7f7096a70700 1 -- 172.21.15.102:0/1110617950 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f709810ba20 msgr2=0x7f70981a0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:31:00.832 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.843 7f7096a70700 1 --2- 172.21.15.102:0/1110617950 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f709810ba20 0x7f70981a0280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:00.832 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.843 7f7096a70700 1 -- 172.21.15.102:0/1110617950 >> v1:172.21.15.102:6789/0 conn(0x7f709810b5d0 legacy=0x7f709819fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:00.832 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.843 7f7096a70700 1 -- 172.21.15.102:0/1110617950 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70981a08b0 con 0x7f70980f57d0 2024-04-17T22:31:00.832 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.843 7f7095a6e700 1 -- 172.21.15.102:0/1110617950 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2333241836 0 0) 0x7f7080001110 con 0x7f709810b5d0 2024-04-17T22:31:00.832 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.843 7f7097271700 1 --2- 172.21.15.102:0/1110617950 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f709810ba20 0x7f70981a0280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:00.832 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.843 7f7096a70700 1 --2- 172.21.15.102:0/1110617950 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f70980f57d0 0x7f709810afa0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 rx=0x7f7098103cf0 tx=0x7f7090009a80).ready entity=mon.0 client_cookie=ecae15a6ff6a7c2f server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:00.832 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.843 7f7095a6e700 1 -- 172.21.15.102:0/1110617950 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f70981a08b0 con 0x7f70980f57d0 2024-04-17T22:31:00.832 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.843 7f7095a6e700 1 -- 172.21.15.102:0/1110617950 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f709000f510 con 0x7f70980f57d0 2024-04-17T22:31:00.832 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.843 7f7095a6e700 1 -- 172.21.15.102:0/1110617950 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f709000f6f0 con 0x7f70980f57d0 2024-04-17T22:31:00.833 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.843 7f709dc8c700 1 -- 172.21.15.102:0/1110617950 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f70980f57d0 msgr2=0x7f709810afa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:00.833 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.843 7f709dc8c700 1 --2- 172.21.15.102:0/1110617950 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f70980f57d0 0x7f709810afa0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 rx=0x7f7098103cf0 tx=0x7f7090009a80).stop 2024-04-17T22:31:00.833 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.844 7f709dc8c700 1 -- 172.21.15.102:0/1110617950 shutdown_connections 2024-04-17T22:31:00.833 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.844 7f709dc8c700 1 --2- 172.21.15.102:0/1110617950 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f709810ba20 0x7f70981a0280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:00.833 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.844 7f709dc8c700 1 --2- 172.21.15.102:0/1110617950 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f70980f57d0 0x7f709810afa0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:00.833 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.844 7f709dc8c700 1 -- 172.21.15.102:0/1110617950 >> 172.21.15.102:0/1110617950 conn(0x7f7098113830 msgr2=0x7f70980f4ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:00.833 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.844 7f709dc8c700 1 -- 172.21.15.102:0/1110617950 shutdown_connections 2024-04-17T22:31:00.833 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.844 7f709dc8c700 1 -- 172.21.15.102:0/1110617950 wait complete. 2024-04-17T22:31:00.855 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.867 7f709dc8c700 1 Processor -- start 2024-04-17T22:31:00.855 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.867 7f709dc8c700 1 -- start start 2024-04-17T22:31:00.855 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.867 7f709dc8c700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f70980de710 0x7f70980d7140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:00.855 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.867 7f709dc8c700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f70980d7770 0x7f70981a5320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:00.855 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.867 7f709dc8c700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f70980d7bc0 0x7f70981a5950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:00.855 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.867 7f709dc8c700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70980c57b0 con 0x7f70980d7770 2024-04-17T22:31:00.855 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.867 7f709dc8c700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70980df160 con 0x7f70980d7bc0 2024-04-17T22:31:00.855 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.867 7f709dc8c700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f70981a5f80 con 0x7f70980de710 2024-04-17T22:31:00.855 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.867 7f709626f700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f70980d7770 0x7f70981a5320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:00.855 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.867 7f7097271700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f70980d7bc0 0x7f70981a5950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:00.856 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.867 7f709626f700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f70980d7770 0x7f70981a5320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:40612/0 (socket says 172.21.15.102:40612) 2024-04-17T22:31:00.856 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.867 7f7096a70700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f70980de710 0x7f70980d7140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:00.856 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.867 7f709626f700 1 -- 172.21.15.102:0/262998283 learned_addr learned my addr 172.21.15.102:0/262998283 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:31:00.856 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.867 7f709626f700 1 -- 172.21.15.102:0/262998283 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f70980de710 msgr2=0x7f70980d7140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:00.856 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.867 7f709626f700 1 --2- 172.21.15.102:0/262998283 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f70980de710 0x7f70980d7140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:00.856 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.868 7f709626f700 1 -- 172.21.15.102:0/262998283 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f70980d7bc0 msgr2=0x7f70981a5950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:00.856 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.868 7f709626f700 1 --2- 172.21.15.102:0/262998283 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f70980d7bc0 0x7f70981a5950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:00.856 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.868 7f709626f700 1 -- 172.21.15.102:0/262998283 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70981a61b0 con 0x7f70980d7770 2024-04-17T22:31:00.856 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.868 7f7097271700 1 --2- 172.21.15.102:0/262998283 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f70980d7bc0 0x7f70981a5950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:00.856 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.868 7f7096a70700 1 --2- 172.21.15.102:0/262998283 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f70980de710 0x7f70980d7140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:00.856 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.868 7f709626f700 1 --2- 172.21.15.102:0/262998283 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f70980d7770 0x7f70981a5320 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 rx=0x7f7080000bc0 tx=0x7f708000c830).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:00.856 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.868 7f708ffff700 1 -- 172.21.15.102:0/262998283 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f70800072e0 con 0x7f70980d7770 2024-04-17T22:31:00.856 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.868 7f708ffff700 1 -- 172.21.15.102:0/262998283 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7080005560 con 0x7f70980d7770 2024-04-17T22:31:00.856 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.868 7f708ffff700 1 -- 172.21.15.102:0/262998283 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f7080005930 con 0x7f70980d7770 2024-04-17T22:31:00.856 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.868 7f709dc8c700 1 -- 172.21.15.102:0/262998283 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70981bb430 con 0x7f70980d7770 2024-04-17T22:31:00.857 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.868 7f709dc8c700 1 -- 172.21.15.102:0/262998283 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70981bb810 con 0x7f70980d7770 2024-04-17T22:31:00.857 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.868 7f708ffff700 1 -- 172.21.15.102:0/262998283 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7f708001b070 con 0x7f70980d7770 2024-04-17T22:31:00.857 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.868 7f708ffff700 1 -- 172.21.15.102:0/262998283 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3194+0+0 (secure 0 0 0) 0x7f7080004bf0 con 0x7f70980d7770 2024-04-17T22:31:00.857 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.869 7f709dc8c700 1 -- 172.21.15.102:0/262998283 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70980f0790 con 0x7f70980d7770 2024-04-17T22:31:00.858 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:00.872 7f708ffff700 1 -- 172.21.15.102:0/262998283 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+110455 (secure 0 0 0) 0x7f70800070d0 con 0x7f70980d7770 2024-04-17T22:31:00.993 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.007 7f709dc8c700 1 -- 172.21.15.102:0/262998283 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f70981ae460 con 0x7f70980d7770 2024-04-17T22:31:00.994 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.007 7f708ffff700 1 -- 172.21.15.102:0/262998283 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) v1 ==== 93+0+1704 (secure 0 0 0) 0x7f7080000c30 con 0x7f70980d7770 2024-04-17T22:31:00.994 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-17T22:31:00.994 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":5,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[],"filesystems":[{"mdsmap":{"epoch":5,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17 22:29:07.787730","modified":"2024-04-17 22:29:09.795249","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"-1 (unspecified)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4177},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4177":{"gid":4177,"name":"a","rank":0,"incarnation":3,"state":"up:active","state_seq":4,"addr":"172.21.15.102:6817/1944542598","addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6817","nonce":1944542598}]},"export_targets":[],"features":4611087854035861503,"flags":0}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-04-17T22:31:00.995 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.008 7f709dc8c700 1 -- 172.21.15.102:0/262998283 >> v1:172.21.15.102:6816/42360 conn(0x7f70700145a0 legacy=0x7f7070016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:00.995 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.008 7f709dc8c700 1 -- 172.21.15.102:0/262998283 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f70980d7770 msgr2=0x7f70981a5320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:00.995 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.008 7f709dc8c700 1 --2- 172.21.15.102:0/262998283 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f70980d7770 0x7f70981a5320 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 rx=0x7f7080000bc0 tx=0x7f708000c830).stop 2024-04-17T22:31:00.995 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.009 7f709dc8c700 1 -- 172.21.15.102:0/262998283 shutdown_connections 2024-04-17T22:31:00.995 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.009 7f709dc8c700 1 --2- 172.21.15.102:0/262998283 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f70980d7bc0 0x7f70981a5950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:00.995 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.009 7f709dc8c700 1 --2- 172.21.15.102:0/262998283 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f70980de710 0x7f70980d7140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:00.995 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.009 7f709dc8c700 1 --2- 172.21.15.102:0/262998283 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f70980d7770 0x7f70981a5320 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:00.995 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.009 7f709dc8c700 1 -- 172.21.15.102:0/262998283 >> 172.21.15.102:0/262998283 conn(0x7f70980c5290 msgr2=0x7f70980f1740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:00.995 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.009 7f709dc8c700 1 -- 172.21.15.102:0/262998283 shutdown_connections 2024-04-17T22:31:00.996 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.009 7f709dc8c700 1 -- 172.21.15.102:0/262998283 wait complete. 2024-04-17T22:31:00.996 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 5 2024-04-17T22:31:01.106 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 5, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-17 22:29:07.787730', 'modified': '2024-04-17 22:29:09.795249', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '-1 (unspecified)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4177}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4177': {'gid': 4177, 'name': 'a', 'rank': 0, 'incarnation': 3, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.102:6817/1944542598', 'addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.102:6817', 'nonce': 1944542598}]}, 'export_targets': [], 'features': 4611087854035861503, 'flags': 0}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2024-04-17T22:31:01.106 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-17T22:31:01.107 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-17T22:31:01.235 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.246 7fb0e64ce700 1 Processor -- start 2024-04-17T22:31:01.235 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.246 7fb0e64ce700 1 -- start start 2024-04-17T22:31:01.235 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.247 7fb0e64ce700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb0e00f57d0 0x7fb0e010afa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:01.235 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.247 7fb0e64ce700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb0e010ba20 0x7fb0e01a0280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:01.235 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.247 7fb0e64ce700 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0e0103ad0 con 0x7fb0e010b5d0 2024-04-17T22:31:01.235 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.247 7fb0e64ce700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0e010be70 con 0x7fb0e00f57d0 2024-04-17T22:31:01.235 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.247 7fb0e64ce700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb0e00fa300 con 0x7fb0e010ba20 2024-04-17T22:31:01.235 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.247 7fb0dfac2700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb0e010ba20 0x7fb0e01a0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:01.235 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.247 7fb0deac0700 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7fb0e010b5d0 0x7fb0e019fa80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:41062/0 (socket says 172.21.15.102:41062) 2024-04-17T22:31:01.235 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.247 7fb0df2c1700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb0e00f57d0 0x7fb0e010afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:01.236 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.247 7fb0deac0700 1 -- 172.21.15.102:0/3069938772 learned_addr learned my addr 172.21.15.102:0/3069938772 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-17T22:31:01.236 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.247 7fb0dfac2700 1 -- 172.21.15.102:0/3069938772 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb0e00f57d0 msgr2=0x7fb0e010afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:31:01.236 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.247 7fb0dfac2700 1 --2- 172.21.15.102:0/3069938772 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb0e00f57d0 0x7fb0e010afa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:01.236 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.247 7fb0dfac2700 1 -- 172.21.15.102:0/3069938772 >> v1:172.21.15.102:6789/0 conn(0x7fb0e010b5d0 legacy=0x7fb0e019fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:01.236 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.247 7fb0dfac2700 1 -- 172.21.15.102:0/3069938772 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0e01a08b0 con 0x7fb0e010ba20 2024-04-17T22:31:01.236 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.247 7fb0de2bf700 1 -- 172.21.15.102:0/3069938772 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3534157386 0 0) 0x7fb0d8001110 con 0x7fb0e010b5d0 2024-04-17T22:31:01.236 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.247 7fb0df2c1700 1 --2- 172.21.15.102:0/3069938772 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb0e00f57d0 0x7fb0e010afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:01.236 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.247 7fb0dfac2700 1 --2- 172.21.15.102:0/3069938772 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb0e010ba20 0x7fb0e01a0280 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 rx=0x7fb0e00f4630 tx=0x7fb0d000b9d0).ready entity=mon.2 client_cookie=10f65207f32e215d server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:01.236 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.247 7fb0de2bf700 1 -- 172.21.15.102:0/3069938772 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb0e01a08b0 con 0x7fb0e010ba20 2024-04-17T22:31:01.236 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.247 7fb0de2bf700 1 -- 172.21.15.102:0/3069938772 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0d000d570 con 0x7fb0e010ba20 2024-04-17T22:31:01.236 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.247 7fb0de2bf700 1 -- 172.21.15.102:0/3069938772 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb0d000d940 con 0x7fb0e010ba20 2024-04-17T22:31:01.236 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.247 7fb0e64ce700 1 -- 172.21.15.102:0/3069938772 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb0e010ba20 msgr2=0x7fb0e01a0280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:01.236 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.247 7fb0e64ce700 1 --2- 172.21.15.102:0/3069938772 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb0e010ba20 0x7fb0e01a0280 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 rx=0x7fb0e00f4630 tx=0x7fb0d000b9d0).stop 2024-04-17T22:31:01.236 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.248 7fb0e64ce700 1 -- 172.21.15.102:0/3069938772 shutdown_connections 2024-04-17T22:31:01.237 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.248 7fb0e64ce700 1 --2- 172.21.15.102:0/3069938772 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb0e010ba20 0x7fb0e01a0280 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:01.237 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.248 7fb0e64ce700 1 --2- 172.21.15.102:0/3069938772 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb0e00f57d0 0x7fb0e010afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:01.237 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.248 7fb0e64ce700 1 -- 172.21.15.102:0/3069938772 >> 172.21.15.102:0/3069938772 conn(0x7fb0e0113830 msgr2=0x7fb0e00f4ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:01.237 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.248 7fb0e64ce700 1 -- 172.21.15.102:0/3069938772 shutdown_connections 2024-04-17T22:31:01.237 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.248 7fb0e64ce700 1 -- 172.21.15.102:0/3069938772 wait complete. 2024-04-17T22:31:01.256 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.270 7fb0e64ce700 1 Processor -- start 2024-04-17T22:31:01.256 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.271 7fb0e64ce700 1 -- start start 2024-04-17T22:31:01.259 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.271 7fb0e64ce700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb0e00f1850 0x7fb0e00d5f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:01.259 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.271 7fb0e64ce700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb0e00d6570 0x7fb0e00cdab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:01.259 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.271 7fb0e64ce700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb0e00d69c0 0x7fb0e00ce0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:01.259 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.271 7fb0e64ce700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0e01a5dd0 con 0x7fb0e00f1850 2024-04-17T22:31:01.259 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.271 7fb0e64ce700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0e00ce710 con 0x7fb0e00d69c0 2024-04-17T22:31:01.259 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.271 7fb0e64ce700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb0e00ce940 con 0x7fb0e00d6570 2024-04-17T22:31:01.259 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.271 7fb0df2c1700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb0e00f1850 0x7fb0e00d5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:01.259 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.271 7fb0deac0700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb0e00d6570 0x7fb0e00cdab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:01.259 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.271 7fb0dfac2700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb0e00d69c0 0x7fb0e00ce0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:01.259 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.271 7fb0df2c1700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb0e00f1850 0x7fb0e00d5f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:40618/0 (socket says 172.21.15.102:40618) 2024-04-17T22:31:01.259 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.271 7fb0df2c1700 1 -- 172.21.15.102:0/1389094204 learned_addr learned my addr 172.21.15.102:0/1389094204 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:31:01.260 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.271 7fb0deac0700 1 --2- 172.21.15.102:0/1389094204 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb0e00d6570 0x7fb0e00cdab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:59378/0 (socket says 172.21.15.102:59378) 2024-04-17T22:31:01.260 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.271 7fb0dfac2700 1 -- 172.21.15.102:0/1389094204 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb0e00d6570 msgr2=0x7fb0e00cdab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:01.260 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.271 7fb0dfac2700 1 --2- 172.21.15.102:0/1389094204 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb0e00d6570 0x7fb0e00cdab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:01.260 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.271 7fb0dfac2700 1 -- 172.21.15.102:0/1389094204 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb0e00f1850 msgr2=0x7fb0e00d5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:01.260 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.271 7fb0dfac2700 1 --2- 172.21.15.102:0/1389094204 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb0e00f1850 0x7fb0e00d5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:01.260 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.271 7fb0dfac2700 1 -- 172.21.15.102:0/1389094204 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0e01b9500 con 0x7fb0e00d69c0 2024-04-17T22:31:01.260 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.271 7fb0df2c1700 1 --2- 172.21.15.102:0/1389094204 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb0e00f1850 0x7fb0e00d5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:01.260 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.271 7fb0dfac2700 1 --2- 172.21.15.102:0/1389094204 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb0e00d69c0 0x7fb0e00ce0e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 rx=0x7fb0e00f4630 tx=0x7fb0d00068c0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:01.260 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.272 7fb0dcabc700 1 -- 172.21.15.102:0/1389094204 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb0e01b9500 con 0x7fb0e00d69c0 2024-04-17T22:31:01.260 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.272 7fb0dcabc700 1 -- 172.21.15.102:0/1389094204 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0d00117e0 con 0x7fb0e00d69c0 2024-04-17T22:31:01.260 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.272 7fb0e64ce700 1 -- 172.21.15.102:0/1389094204 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0e00cb8a0 con 0x7fb0e00d69c0 2024-04-17T22:31:01.260 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.272 7fb0dcabc700 1 -- 172.21.15.102:0/1389094204 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb0d0017040 con 0x7fb0e00d69c0 2024-04-17T22:31:01.260 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.272 7fb0e64ce700 1 -- 172.21.15.102:0/1389094204 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0e00cbd40 con 0x7fb0e00d69c0 2024-04-17T22:31:01.260 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.272 7fb0dcabc700 1 -- 172.21.15.102:0/1389094204 <== mon.1 v2:172.21.15.102:3301/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7fb0d00263f0 con 0x7fb0e00d69c0 2024-04-17T22:31:01.260 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.272 7fb0dcabc700 1 -- 172.21.15.102:0/1389094204 <== mon.1 v2:172.21.15.102:3301/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3194+0+0 (secure 0 0 0) 0x7fb0d0018070 con 0x7fb0e00d69c0 2024-04-17T22:31:01.267 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.275 7fb0f8d87b80 1 Processor -- start 2024-04-17T22:31:01.268 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.275 7fb0f8d87b80 1 -- start start 2024-04-17T22:31:01.268 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.275 7fb0f8d87b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c42248b220 0x55c42248b670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:01.268 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.275 7fb0f8d87b80 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x55c42248bca0 0x55c42247e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:01.268 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.275 7fb0f8d87b80 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x55c42247ef10 0x55c42247f360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:01.268 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.275 7fb0f8d87b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4223b6c80 con 0x55c42248b220 2024-04-17T22:31:01.268 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.275 7fb0f8d87b80 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c4223d4a40 con 0x55c42247ef10 2024-04-17T22:31:01.268 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.275 7fb0f8d87b80 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x55c4223a1e00 con 0x55c42248bca0 2024-04-17T22:31:01.268 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.275 7fb0df2c1700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c42248b220 0x55c42248b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:01.268 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.275 7fb0deac0700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x55c42248bca0 0x55c42247e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:01.268 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.275 7fb0df2c1700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c42248b220 0x55c42248b670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:40624/0 (socket says 172.21.15.102:40624) 2024-04-17T22:31:01.268 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.275 7fb0df2c1700 1 -- 172.21.15.102:0/3086592795 learned_addr learned my addr 172.21.15.102:0/3086592795 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:31:01.268 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.275 7fb0dfac2700 1 --2- 172.21.15.102:0/3086592795 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x55c42247ef10 0x55c42247f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:01.268 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.275 7fb0deac0700 1 -- 172.21.15.102:0/3086592795 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x55c42247ef10 msgr2=0x55c42247f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:31:01.269 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.275 7fb0deac0700 1 --2- 172.21.15.102:0/3086592795 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x55c42247ef10 0x55c42247f360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:01.269 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.275 7fb0deac0700 1 -- 172.21.15.102:0/3086592795 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c42248b220 msgr2=0x55c42248b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:31:01.269 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.275 7fb0deac0700 1 --2- 172.21.15.102:0/3086592795 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c42248b220 0x55c42248b670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:01.269 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.275 7fb0deac0700 1 -- 172.21.15.102:0/3086592795 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c4224543a0 con 0x55c42248bca0 2024-04-17T22:31:01.269 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.275 7fb0df2c1700 1 --2- 172.21.15.102:0/3086592795 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c42248b220 0x55c42248b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:01.269 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.276 7fb0deac0700 1 --2- 172.21.15.102:0/3086592795 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x55c42248bca0 0x55c42247e8e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 rx=0x55c4223c2610 tx=0x7fb0d8012c30).ready entity=mon.2 client_cookie=453179a8518f7022 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:01.269 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.276 7fb0e64ce700 1 -- 172.21.15.102:0/3086592795 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x55c4224543a0 con 0x55c42248bca0 2024-04-17T22:31:01.269 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.276 7fb0e64ce700 1 -- 172.21.15.102:0/3086592795 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0d8019040 con 0x55c42248bca0 2024-04-17T22:31:01.269 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.276 7fb0e64ce700 1 -- 172.21.15.102:0/3086592795 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb0d80120d0 con 0x55c42248bca0 2024-04-17T22:31:01.269 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.276 7fb0f8d87b80 1 -- 172.21.15.102:0/3086592795 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x55c42248bca0 msgr2=0x55c42247e8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:01.269 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.276 7fb0f8d87b80 1 --2- 172.21.15.102:0/3086592795 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x55c42248bca0 0x55c42247e8e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 rx=0x55c4223c2610 tx=0x7fb0d8012c30).stop 2024-04-17T22:31:01.269 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.276 7fb0f8d87b80 1 -- 172.21.15.102:0/3086592795 shutdown_connections 2024-04-17T22:31:01.269 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.276 7fb0f8d87b80 1 --2- 172.21.15.102:0/3086592795 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x55c42247ef10 0x55c42247f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:01.269 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.276 7fb0f8d87b80 1 --2- 172.21.15.102:0/3086592795 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c42248b220 0x55c42248b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:01.270 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.276 7fb0f8d87b80 1 --2- 172.21.15.102:0/3086592795 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x55c42248bca0 0x55c42247e8e0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:01.270 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.276 7fb0f8d87b80 1 -- 172.21.15.102:0/3086592795 >> 172.21.15.102:0/3086592795 conn(0x55c422477cc0 msgr2=0x55c422499200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:01.270 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.276 7fb0f8d87b80 1 -- 172.21.15.102:0/3086592795 shutdown_connections 2024-04-17T22:31:01.270 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.276 7fb0f8d87b80 1 -- 172.21.15.102:0/3086592795 wait complete. 2024-04-17T22:31:01.270 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.277 7fb0f8d87b80 1 -- start start 2024-04-17T22:31:01.270 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.277 7fb0f8d87b80 1 Processor -- start 2024-04-17T22:31:01.270 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.277 7fb0f8d87b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c42247d610 0x55c42247da60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:01.270 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.277 7fb0f8d87b80 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x55c42247e090 0x55c4224a68a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:01.270 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.277 7fb0f8d87b80 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x55c4224a6ed0 0x55c4224a7320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:01.270 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.277 7fb0f8d87b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c422477500 con 0x55c42247d610 2024-04-17T22:31:01.270 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.277 7fb0f8d87b80 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c422495080 con 0x55c42247e090 2024-04-17T22:31:01.270 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.277 7fb0f8d87b80 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x55c422494cf0 con 0x55c4224a6ed0 2024-04-17T22:31:01.270 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.277 7fb0f8d87b80 10 client.0 ll_register_callbacks cb 0x55c4223acd30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T22:31:01.271 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.277 7fb0df2c1700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c42247d610 0x55c42247da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:01.271 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.277 7fb0df2c1700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c42247d610 0x55c42247da60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:40626/0 (socket says 172.21.15.102:40626) 2024-04-17T22:31:01.271 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.277 7fb0df2c1700 1 -- 172.21.15.102:0/3887276652 learned_addr learned my addr 172.21.15.102:0/3887276652 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:31:01.271 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.277 7fb0dfac2700 1 --2- 172.21.15.102:0/3887276652 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x55c4224a6ed0 0x55c4224a7320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:01.271 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.277 7fb0deac0700 1 --2- 172.21.15.102:0/3887276652 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x55c42247e090 0x55c4224a68a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:01.271 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.278 7fb0df2c1700 1 -- 172.21.15.102:0/3887276652 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x55c4224a6ed0 msgr2=0x55c4224a7320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:31:01.271 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.278 7fb0df2c1700 1 --2- 172.21.15.102:0/3887276652 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x55c4224a6ed0 0x55c4224a7320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:01.271 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.278 7fb0df2c1700 1 -- 172.21.15.102:0/3887276652 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x55c42247e090 msgr2=0x55c4224a68a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:31:01.271 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.278 7fb0df2c1700 1 --2- 172.21.15.102:0/3887276652 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x55c42247e090 0x55c4224a68a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:01.271 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.278 7fb0df2c1700 1 -- 172.21.15.102:0/3887276652 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c4222c4b60 con 0x55c42247d610 2024-04-17T22:31:01.271 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.278 7fb0df2c1700 1 --2- 172.21.15.102:0/3887276652 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c42247d610 0x55c42247da60 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 rx=0x7fb0c8006fc0 tx=0x7fb0c80168a0).ready entity=mon.0 client_cookie=5844b8212e17bf53 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:01.271 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.278 7fb0ce7fc700 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-04-17T22:31:01.271 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.278 7fb0ce7fc700 1 -- 172.21.15.102:0/3887276652 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x55c4222c4b60 con 0x55c42247d610 2024-04-17T22:31:01.272 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.278 7fb0ce7fc700 1 -- 172.21.15.102:0/3887276652 <== mon.0 v2:172.21.15.102:3300/0 2 ==== osd_map(17..17 src has 1..17) v4 ==== 3194+0+0 (secure 0 0 0) 0x7fb0c8019540 con 0x55c42247d610 2024-04-17T22:31:01.272 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.278 7fb0f8d87b80 1 -- 172.21.15.102:0/3887276652 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c4223bfcd0 con 0x55c42247d610 2024-04-17T22:31:01.272 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.278 7fb0ce7fc700 1 -- 172.21.15.102:0/3887276652 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=5) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb0c8010e10 con 0x55c42247d610 2024-04-17T22:31:01.272 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.278 7fb0f8d87b80 10 client.4947 fetch_fsmap learned FSMap version 5 2024-04-17T22:31:01.272 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.278 7fb0f8d87b80 1 -- 172.21.15.102:0/3887276652 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=5,monmap=2+}) v3 -- 0x55c4224ae2e0 con 0x55c42247d610 2024-04-17T22:31:01.272 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.278 7fb0ce7fc700 1 -- 172.21.15.102:0/3887276652 <== mon.0 v2:172.21.15.102:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c4223bfcd0 con 0x55c42247d610 2024-04-17T22:31:01.272 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.278 7fb0ce7fc700 1 -- 172.21.15.102:0/3887276652 <== mon.0 v2:172.21.15.102:3300/0 5 ==== fsmap(e 5) v1 ==== 1025+0+0 (secure 0 0 0) 0x7fb0c8019c30 con 0x55c42247d610 2024-04-17T22:31:01.272 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.278 7fb0f8d87b80 10 client.4947 fetch_fsmap finished waiting for FSMap version 5 2024-04-17T22:31:01.272 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.278 7fb0f8d87b80 10 client.4947 resolve_mds: resolved ID 'a' to GID 4177 2024-04-17T22:31:01.272 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.278 7fb0f8d87b80 20 client.4947 populate_metadata read hostname 'smithi102' 2024-04-17T22:31:01.272 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.278 7fb0f8d87b80 4 client.4947 mds_command: new command op to 4177 tid=0[{"prefix": "get_command_descriptions"}] 2024-04-17T22:31:01.272 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.278 7fb0f8d87b80 1 -- 172.21.15.102:0/3887276652 --> v1:172.21.15.102:6817/1944542598 -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c4222b8ba0 con 0x55c4224ae8e0 2024-04-17T22:31:01.272 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.278 7fb0ce7fc700 10 client.4947 ms_handle_connect on v1:172.21.15.102:6817/1944542598 2024-04-17T22:31:01.272 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.279 7fb0ce7fc700 1 -- 172.21.15.102:0/3887276652 <== mds.0 v1:172.21.15.102:6817/1944542598 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3729 (unknown 0 0 338808096) 0x55c4222b8ba0 con 0x55c4224ae8e0 2024-04-17T22:31:01.273 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.279 7fb0ce7fc700 10 client.4947 handle_command_reply: tid=0 2024-04-17T22:31:01.273 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.279 7fb0f8d87b80 1 client.4947 shutdown 2024-04-17T22:31:01.273 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.279 7fb0f8d87b80 1 -- 172.21.15.102:0/3887276652 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c42247d610 msgr2=0x55c42247da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:01.273 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.279 7fb0f8d87b80 1 --2- 172.21.15.102:0/3887276652 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c42247d610 0x55c42247da60 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 rx=0x7fb0c8006fc0 tx=0x7fb0c80168a0).stop 2024-04-17T22:31:01.273 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.279 7fb0f8d87b80 1 -- 172.21.15.102:0/3887276652 shutdown_connections 2024-04-17T22:31:01.273 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.279 7fb0f8d87b80 1 --2- 172.21.15.102:0/3887276652 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x55c4224a6ed0 0x55c4224a7320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:01.273 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.279 7fb0f8d87b80 1 --2- 172.21.15.102:0/3887276652 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c42247d610 0x55c42247da60 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:01.273 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.279 7fb0f8d87b80 1 --2- 172.21.15.102:0/3887276652 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x55c42247e090 0x55c4224a68a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:01.273 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.279 7fb0f8d87b80 1 -- 172.21.15.102:0/3887276652 >> 172.21.15.102:0/3887276652 conn(0x55c422499330 msgr2=0x55c422495bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:01.273 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.279 7fb0ce7fc700 0 client.4947 ms_handle_reset on v1:172.21.15.102:6817/1944542598 2024-04-17T22:31:01.273 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.279 7fb0f8d87b80 1 -- 172.21.15.102:0/3887276652 shutdown_connections 2024-04-17T22:31:01.273 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.279 7fb0f8d87b80 1 -- 172.21.15.102:0/3887276652 wait complete. 2024-04-17T22:31:01.273 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.279 7fb0f8d87b80 20 client.4947 trim_cache size 0 max 16384 2024-04-17T22:31:01.275 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.289 7fb0d4ca4700 1 -- 172.21.15.102:0/1389094204 >> v1:172.21.15.102:6816/42360 conn(0x7fb0b80145a0 legacy=0x7fb0b8016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:01.275 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.289 7fb0d4ca4700 1 -- 172.21.15.102:0/1389094204 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb0e00d69c0 msgr2=0x7fb0e00ce0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:01.275 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.289 7fb0d4ca4700 1 --2- 172.21.15.102:0/1389094204 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb0e00d69c0 0x7fb0e00ce0e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 rx=0x7fb0e00f4630 tx=0x7fb0d00068c0).stop 2024-04-17T22:31:01.275 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.289 7fb0d4ca4700 1 -- 172.21.15.102:0/1389094204 shutdown_connections 2024-04-17T22:31:01.275 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.289 7fb0d4ca4700 1 --2- 172.21.15.102:0/1389094204 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb0e00d69c0 0x7fb0e00ce0e0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:01.275 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.289 7fb0d4ca4700 1 --2- 172.21.15.102:0/1389094204 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb0e00f1850 0x7fb0e00d5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:01.276 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.289 7fb0d4ca4700 1 --2- 172.21.15.102:0/1389094204 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb0e00d6570 0x7fb0e00cdab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:01.276 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.289 7fb0d4ca4700 1 -- 172.21.15.102:0/1389094204 >> 172.21.15.102:0/1389094204 conn(0x7fb0e00c3780 msgr2=0x7fb0e00dadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:01.276 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.289 7fb0d4ca4700 1 -- 172.21.15.102:0/1389094204 shutdown_connections 2024-04-17T22:31:01.276 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.289 7fb0d4ca4700 1 -- 172.21.15.102:0/1389094204 wait complete. 2024-04-17T22:31:01.276 INFO:teuthology.orchestra.run.smithi102.stderr:no valid command found; 10 closest matches: 2024-04-17T22:31:01.276 INFO:teuthology.orchestra.run.smithi102.stderr:injectargs [...] 2024-04-17T22:31:01.276 INFO:teuthology.orchestra.run.smithi102.stderr:config set 2024-04-17T22:31:01.276 INFO:teuthology.orchestra.run.smithi102.stderr:config unset 2024-04-17T22:31:01.276 INFO:teuthology.orchestra.run.smithi102.stderr:exit 2024-04-17T22:31:01.276 INFO:teuthology.orchestra.run.smithi102.stderr:respawn 2024-04-17T22:31:01.277 INFO:teuthology.orchestra.run.smithi102.stderr:session kill 2024-04-17T22:31:01.277 INFO:teuthology.orchestra.run.smithi102.stderr:cpu_profiler status|flush 2024-04-17T22:31:01.277 INFO:teuthology.orchestra.run.smithi102.stderr:session ls { [...]} 2024-04-17T22:31:01.277 INFO:teuthology.orchestra.run.smithi102.stderr:client ls { [...]} 2024-04-17T22:31:01.277 INFO:teuthology.orchestra.run.smithi102.stderr:session evict { [...]} 2024-04-17T22:31:01.277 INFO:teuthology.orchestra.run.smithi102.stderr:Error EINVAL: invalid command 2024-04-17T22:31:01.285 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-17T22:31:01.285 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-04-17T22:31:01.295 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-04-17T22:31:01.295 INFO:tasks.ceph_fuse:config is {'client.0': None} 2024-04-17T22:31:01.296 INFO:tasks.ceph_fuse:client.0 config is {} 2024-04-17T22:31:01.296 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T22:31:01.296 DEBUG:teuthology.orchestra.run.smithi190:> ip netns list 2024-04-17T22:31:01.319 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T22:31:01.319 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip link delete ceph-brx 2024-04-17T22:31:01.397 INFO:teuthology.orchestra.run.smithi190.stderr:Cannot find device "ceph-brx" 2024-04-17T22:31:01.398 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-17T22:31:01.398 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-04-17T22:31:01.399 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-17T22:31:01.526 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.537 7f0001521700 1 Processor -- start 2024-04-17T22:31:01.526 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.537 7f0001521700 1 -- start start 2024-04-17T22:31:01.526 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.538 7f0001521700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7efffc0f5860 0x7efffc10b030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:01.526 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.538 7f0001521700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7efffc10b660 0x7efffc19fb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:01.526 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.538 7f0001521700 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efffc103b60 con 0x7efffc10bab0 2024-04-17T22:31:01.526 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.538 7f0001521700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efffc10bf00 con 0x7efffc0f5860 2024-04-17T22:31:01.526 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.538 7f0001521700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7efffc0fa390 con 0x7efffc10b660 2024-04-17T22:31:01.526 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.538 7efffa2b1700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7efffc0f5860 0x7efffc10b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:01.526 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.538 7efffaab2700 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7efffc10bab0 0x7efffc1a0140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:41074/0 (socket says 172.21.15.102:41074) 2024-04-17T22:31:01.526 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.538 7efffa2b1700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7efffc0f5860 0x7efffc10b030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:49082/0 (socket says 172.21.15.102:49082) 2024-04-17T22:31:01.526 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.538 7efffaab2700 1 -- 172.21.15.102:0/1706002780 learned_addr learned my addr 172.21.15.102:0/1706002780 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-17T22:31:01.527 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.538 7efff3fff700 1 --2- 172.21.15.102:0/1706002780 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7efffc10b660 0x7efffc19fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:01.527 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.538 7efff9ab0700 1 -- 172.21.15.102:0/1706002780 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2545621510 0 0) 0x7efff4001490 con 0x7efffc10bab0 2024-04-17T22:31:01.527 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.538 7efffa2b1700 1 -- 172.21.15.102:0/1706002780 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7efffc10b660 msgr2=0x7efffc19fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:31:01.527 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.538 7efffa2b1700 1 --2- 172.21.15.102:0/1706002780 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7efffc10b660 0x7efffc19fb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:01.527 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.538 7efffa2b1700 1 -- 172.21.15.102:0/1706002780 >> v1:172.21.15.102:6789/0 conn(0x7efffc10bab0 legacy=0x7efffc1a0140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:01.527 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.538 7efffa2b1700 1 -- 172.21.15.102:0/1706002780 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efffc1a0940 con 0x7efffc0f5860 2024-04-17T22:31:01.527 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.538 7efffa2b1700 1 --2- 172.21.15.102:0/1706002780 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7efffc0f5860 0x7efffc10b030 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 rx=0x7efffc103d80 tx=0x7effec00ba50).ready entity=mon.1 client_cookie=8ccab3b86c91b8cb server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:01.527 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.538 7efff9ab0700 1 -- 172.21.15.102:0/1706002780 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7efffc1a0940 con 0x7efffc0f5860 2024-04-17T22:31:01.527 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.539 7efff9ab0700 1 -- 172.21.15.102:0/1706002780 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effec00d570 con 0x7efffc0f5860 2024-04-17T22:31:01.527 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.539 7efff9ab0700 1 -- 172.21.15.102:0/1706002780 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7effec00d940 con 0x7efffc0f5860 2024-04-17T22:31:01.527 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.539 7f0001521700 1 -- 172.21.15.102:0/1706002780 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7efffc0f5860 msgr2=0x7efffc10b030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:01.527 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.539 7f0001521700 1 --2- 172.21.15.102:0/1706002780 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7efffc0f5860 0x7efffc10b030 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 rx=0x7efffc103d80 tx=0x7effec00ba50).stop 2024-04-17T22:31:01.527 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.539 7f0001521700 1 -- 172.21.15.102:0/1706002780 shutdown_connections 2024-04-17T22:31:01.527 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.539 7f0001521700 1 --2- 172.21.15.102:0/1706002780 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7efffc0f5860 0x7efffc10b030 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:01.528 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.539 7f0001521700 1 --2- 172.21.15.102:0/1706002780 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7efffc10b660 0x7efffc19fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:01.528 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.539 7f0001521700 1 -- 172.21.15.102:0/1706002780 >> 172.21.15.102:0/1706002780 conn(0x7efffc1138c0 msgr2=0x7efffc0f4f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:01.528 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.539 7f0001521700 1 -- 172.21.15.102:0/1706002780 shutdown_connections 2024-04-17T22:31:01.528 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.539 7f0001521700 1 -- 172.21.15.102:0/1706002780 wait complete. 2024-04-17T22:31:01.550 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.562 7f0001521700 1 Processor -- start 2024-04-17T22:31:01.550 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.562 7f0001521700 1 -- start start 2024-04-17T22:31:01.550 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.562 7f0001521700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7efffc0d4df0 0x7efffc0d5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:01.550 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.562 7f0001521700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7efffc0d5870 0x7efffc1afcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:01.550 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.562 7f0001521700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efffc1b02f0 0x7efffc1b0740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:01.550 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.562 7f0001521700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efffc0d9660 con 0x7efffc1b02f0 2024-04-17T22:31:01.550 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.562 7f0001521700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efffc0c3380 con 0x7efffc0d5870 2024-04-17T22:31:01.550 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.562 7f0001521700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7efffc0d98a0 con 0x7efffc0d4df0 2024-04-17T22:31:01.551 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.562 7efff3fff700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7efffc0d5870 0x7efffc1afcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:01.551 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.562 7efffaab2700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efffc1b02f0 0x7efffc1b0740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:01.551 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.562 7efffa2b1700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7efffc0d4df0 0x7efffc0d5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:01.551 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.562 7efff3fff700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7efffc0d5870 0x7efffc1afcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:49090/0 (socket says 172.21.15.102:49090) 2024-04-17T22:31:01.551 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.562 7efffaab2700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efffc1b02f0 0x7efffc1b0740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:40642/0 (socket says 172.21.15.102:40642) 2024-04-17T22:31:01.551 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.562 7efff3fff700 1 -- 172.21.15.102:0/3728701090 learned_addr learned my addr 172.21.15.102:0/3728701090 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:31:01.551 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.562 7efffaab2700 1 -- 172.21.15.102:0/3728701090 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7efffc0d4df0 msgr2=0x7efffc0d5240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:01.551 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.562 7efffaab2700 1 --2- 172.21.15.102:0/3728701090 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7efffc0d4df0 0x7efffc0d5240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:01.551 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.562 7efffaab2700 1 -- 172.21.15.102:0/3728701090 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7efffc0d5870 msgr2=0x7efffc1afcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:01.551 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.562 7efffaab2700 1 --2- 172.21.15.102:0/3728701090 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7efffc0d5870 0x7efffc1afcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:01.551 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.563 7efffaab2700 1 -- 172.21.15.102:0/3728701090 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efffc1a1290 con 0x7efffc1b02f0 2024-04-17T22:31:01.551 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.563 7efff3fff700 1 --2- 172.21.15.102:0/3728701090 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7efffc0d5870 0x7efffc1afcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:01.551 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.563 7efffa2b1700 1 --2- 172.21.15.102:0/3728701090 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7efffc0d4df0 0x7efffc0d5240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:01.552 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.563 7efffaab2700 1 --2- 172.21.15.102:0/3728701090 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efffc1b02f0 0x7efffc1b0740 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 rx=0x7efffc0c3940 tx=0x7efff400c9b0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:01.552 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.563 7efff37fe700 1 -- 172.21.15.102:0/3728701090 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7efffc1a1290 con 0x7efffc1b02f0 2024-04-17T22:31:01.552 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.563 7efff37fe700 1 -- 172.21.15.102:0/3728701090 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efff4005bb0 con 0x7efffc1b02f0 2024-04-17T22:31:01.552 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.563 7f0001521700 1 -- 172.21.15.102:0/3728701090 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efffc1b73b0 con 0x7efffc1b02f0 2024-04-17T22:31:01.552 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.563 7efff37fe700 1 -- 172.21.15.102:0/3728701090 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7efff40048b0 con 0x7efffc1b02f0 2024-04-17T22:31:01.552 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.563 7f0001521700 1 -- 172.21.15.102:0/3728701090 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efffc1b7790 con 0x7efffc1b02f0 2024-04-17T22:31:01.552 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.563 7efff37fe700 1 -- 172.21.15.102:0/3728701090 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7efff4004ab0 con 0x7efffc1b02f0 2024-04-17T22:31:01.552 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.563 7efff37fe700 1 -- 172.21.15.102:0/3728701090 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3194+0+0 (secure 0 0 0) 0x7efff400b960 con 0x7efffc1b02f0 2024-04-17T22:31:01.552 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.564 7f0001521700 1 -- 172.21.15.102:0/3728701090 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efffc0f0820 con 0x7efffc1b02f0 2024-04-17T22:31:01.553 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.567 7efff37fe700 1 -- 172.21.15.102:0/3728701090 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+110455 (secure 0 0 0) 0x7efff40059a0 con 0x7efffc1b02f0 2024-04-17T22:31:01.690 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.703 7f0001521700 1 -- 172.21.15.102:0/3728701090 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7efffc1aa3e0 con 0x7efffc1b02f0 2024-04-17T22:31:01.690 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.704 7efff37fe700 1 -- 172.21.15.102:0/3728701090 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v5) v1 ==== 53+0+73 (secure 0 0 0) 0x7efffc114440 con 0x7efffc1b02f0 2024-04-17T22:31:01.690 INFO:teuthology.orchestra.run.smithi102.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-17T22:31:01.691 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.705 7f0001521700 1 -- 172.21.15.102:0/3728701090 >> v1:172.21.15.102:6816/42360 conn(0x7effe00145a0 legacy=0x7effe0016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:01.692 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.705 7f0001521700 1 -- 172.21.15.102:0/3728701090 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efffc1b02f0 msgr2=0x7efffc1b0740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:01.692 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.705 7f0001521700 1 --2- 172.21.15.102:0/3728701090 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efffc1b02f0 0x7efffc1b0740 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 rx=0x7efffc0c3940 tx=0x7efff400c9b0).stop 2024-04-17T22:31:01.692 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.705 7f0001521700 1 -- 172.21.15.102:0/3728701090 shutdown_connections 2024-04-17T22:31:01.692 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.705 7f0001521700 1 --2- 172.21.15.102:0/3728701090 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efffc1b02f0 0x7efffc1b0740 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:01.692 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.705 7f0001521700 1 --2- 172.21.15.102:0/3728701090 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7efffc0d4df0 0x7efffc0d5240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:01.692 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.705 7f0001521700 1 --2- 172.21.15.102:0/3728701090 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7efffc0d5870 0x7efffc1afcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:01.692 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.705 7f0001521700 1 -- 172.21.15.102:0/3728701090 >> 172.21.15.102:0/3728701090 conn(0x7efffc0e0f60 msgr2=0x7efffc0c1e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:01.692 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.705 7f0001521700 1 -- 172.21.15.102:0/3728701090 shutdown_connections 2024-04-17T22:31:01.692 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:01.705 7f0001521700 1 -- 172.21.15.102:0/3728701090 wait complete. 2024-04-17T22:31:01.702 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-17T22:31:01.702 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-17T22:31:01.702 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi190.front.sepia.ceph.com 2024-04-17T22:31:01.702 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-17T22:31:01.702 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-17T22:31:01.702 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-17T22:31:01.703 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-17T22:31:01.703 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-17T22:31:01.703 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-17T22:31:01.703 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T22:31:01.703 DEBUG:teuthology.orchestra.run.smithi190:> ip addr 2024-04-17T22:31:01.725 INFO:teuthology.orchestra.run.smithi190.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-17T22:31:01.725 INFO:teuthology.orchestra.run.smithi190.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-17T22:31:01.725 INFO:teuthology.orchestra.run.smithi190.stdout: inet 127.0.0.1/8 scope host lo 2024-04-17T22:31:01.725 INFO:teuthology.orchestra.run.smithi190.stdout: valid_lft forever preferred_lft forever 2024-04-17T22:31:01.725 INFO:teuthology.orchestra.run.smithi190.stdout: inet6 ::1/128 scope host 2024-04-17T22:31:01.725 INFO:teuthology.orchestra.run.smithi190.stdout: valid_lft forever preferred_lft forever 2024-04-17T22:31:01.725 INFO:teuthology.orchestra.run.smithi190.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-17T22:31:01.725 INFO:teuthology.orchestra.run.smithi190.stdout: link/ether 0c:c4:7a:d9:36:d0 brd ff:ff:ff:ff:ff:ff 2024-04-17T22:31:01.725 INFO:teuthology.orchestra.run.smithi190.stdout: altname enp7s0f0 2024-04-17T22:31:01.725 INFO:teuthology.orchestra.run.smithi190.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-17T22:31:01.725 INFO:teuthology.orchestra.run.smithi190.stdout: link/ether 0c:c4:7a:d9:36:d1 brd ff:ff:ff:ff:ff:ff 2024-04-17T22:31:01.725 INFO:teuthology.orchestra.run.smithi190.stdout: altname enp7s0f1 2024-04-17T22:31:01.725 INFO:teuthology.orchestra.run.smithi190.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-17T22:31:01.726 INFO:teuthology.orchestra.run.smithi190.stdout: link/ether 0c:c4:7a:88:77:62 brd ff:ff:ff:ff:ff:ff 2024-04-17T22:31:01.726 INFO:teuthology.orchestra.run.smithi190.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-17T22:31:01.726 INFO:teuthology.orchestra.run.smithi190.stdout: link/ether 0c:c4:7a:88:77:63 brd ff:ff:ff:ff:ff:ff 2024-04-17T22:31:01.726 INFO:teuthology.orchestra.run.smithi190.stdout: inet 172.21.15.190/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-17T22:31:01.726 INFO:teuthology.orchestra.run.smithi190.stdout: valid_lft 42563sec preferred_lft 42563sec 2024-04-17T22:31:01.726 INFO:teuthology.orchestra.run.smithi190.stdout: inet6 fe80::ec4:7aff:fe88:7763/64 scope link 2024-04-17T22:31:01.726 INFO:teuthology.orchestra.run.smithi190.stdout: valid_lft forever preferred_lft forever 2024-04-17T22:31:01.726 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-17T22:31:01.726 DEBUG:teuthology.orchestra.run.smithi190:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-17T22:31:01.726 DEBUG:teuthology.orchestra.run.smithi190:> set -e 2024-04-17T22:31:01.726 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip link add name ceph-brx type bridge 2024-04-17T22:31:01.727 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip addr flush dev ceph-brx 2024-04-17T22:31:01.727 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip link set ceph-brx up 2024-04-17T22:31:01.727 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-17T22:31:01.727 DEBUG:teuthology.orchestra.run.smithi190:> ') 2024-04-17T22:31:01.824 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T22:31:01 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-17T22:31:01.887 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T22:31:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T22:31:01.893 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T22:31:01.893 DEBUG:teuthology.orchestra.run.smithi190:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-17T22:31:01.922 INFO:teuthology.orchestra.run.smithi190.stdout:1 2024-04-17T22:31:01.923 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T22:31:01.923 DEBUG:teuthology.orchestra.run.smithi190:> route 2024-04-17T22:31:01.983 INFO:teuthology.orchestra.run.smithi190.stdout:Kernel IP routing table 2024-04-17T22:31:01.983 INFO:teuthology.orchestra.run.smithi190.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-17T22:31:01.983 INFO:teuthology.orchestra.run.smithi190.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-17T22:31:01.983 INFO:teuthology.orchestra.run.smithi190.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-17T22:31:01.983 INFO:teuthology.orchestra.run.smithi190.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-17T22:31:01.984 DEBUG:teuthology.orchestra.run.smithi190:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-17T22:31:01.984 DEBUG:teuthology.orchestra.run.smithi190:> set -e 2024-04-17T22:31:01.984 DEBUG:teuthology.orchestra.run.smithi190:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-17T22:31:01.984 DEBUG:teuthology.orchestra.run.smithi190:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-17T22:31:01.984 DEBUG:teuthology.orchestra.run.smithi190:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-17T22:31:01.984 DEBUG:teuthology.orchestra.run.smithi190:> ') 2024-04-17T22:31:02.073 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T22:31:02 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-17T22:31:02.121 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T22:31:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T22:31:02.126 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T22:31:02.126 DEBUG:teuthology.orchestra.run.smithi190:> ip netns list 2024-04-17T22:31:02.144 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T22:31:02.144 DEBUG:teuthology.orchestra.run.smithi190:> ip netns list-id 2024-04-17T22:31:02.203 DEBUG:teuthology.orchestra.run.smithi190:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-17T22:31:02.203 DEBUG:teuthology.orchestra.run.smithi190:> set -e 2024-04-17T22:31:02.203 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-17T22:31:02.203 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-04-17T22:31:02.203 DEBUG:teuthology.orchestra.run.smithi190:> ') 2024-04-17T22:31:02.294 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T22:31:02 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-17T22:31:02.330 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T22:31:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T22:31:02.335 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-04-17T22:31:02.336 DEBUG:teuthology.orchestra.run.smithi190:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-17T22:31:02.336 DEBUG:teuthology.orchestra.run.smithi190:> set -e 2024-04-17T22:31:02.336 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-04-17T22:31:02.336 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-17T22:31:02.336 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-04-17T22:31:02.336 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-04-17T22:31:02.336 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-04-17T22:31:02.336 DEBUG:teuthology.orchestra.run.smithi190:> ') 2024-04-17T22:31:02.387 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T22:31:02 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-17T22:31:02.569 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T22:31:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T22:31:02.575 DEBUG:teuthology.orchestra.run.smithi190:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-17T22:31:02.575 DEBUG:teuthology.orchestra.run.smithi190:> set -e 2024-04-17T22:31:02.575 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip link set brx.0 up 2024-04-17T22:31:02.575 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip link set dev brx.0 master ceph-brx 2024-04-17T22:31:02.575 DEBUG:teuthology.orchestra.run.smithi190:> ') 2024-04-17T22:31:02.626 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T22:31:02 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-17T22:31:02.705 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T22:31:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T22:31:02.710 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-17T22:31:02.710 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-17T22:31:02.711 DEBUG:teuthology.orchestra.run.smithi190:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-17T22:31:02.728 INFO:teuthology.orchestra.run.smithi190.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-17T22:31:02.728 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-17T22:31:02.728 DEBUG:teuthology.orchestra.run.smithi190:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-17T22:31:02.785 DEBUG:teuthology.orchestra.run.smithi190:> sudo modprobe fuse 2024-04-17T22:31:02.851 DEBUG:teuthology.orchestra.run.smithi190:> cat /proc/self/mounts | awk '{print $2}' 2024-04-17T22:31:02.910 INFO:teuthology.orchestra.run.smithi190.stdout:/sys 2024-04-17T22:31:02.910 INFO:teuthology.orchestra.run.smithi190.stdout:/proc 2024-04-17T22:31:02.910 INFO:teuthology.orchestra.run.smithi190.stdout:/dev 2024-04-17T22:31:02.910 INFO:teuthology.orchestra.run.smithi190.stdout:/sys/kernel/security 2024-04-17T22:31:02.910 INFO:teuthology.orchestra.run.smithi190.stdout:/dev/shm 2024-04-17T22:31:02.910 INFO:teuthology.orchestra.run.smithi190.stdout:/dev/pts 2024-04-17T22:31:02.911 INFO:teuthology.orchestra.run.smithi190.stdout:/run 2024-04-17T22:31:02.911 INFO:teuthology.orchestra.run.smithi190.stdout:/sys/fs/cgroup 2024-04-17T22:31:02.911 INFO:teuthology.orchestra.run.smithi190.stdout:/sys/fs/cgroup/systemd 2024-04-17T22:31:02.911 INFO:teuthology.orchestra.run.smithi190.stdout:/sys/fs/pstore 2024-04-17T22:31:02.911 INFO:teuthology.orchestra.run.smithi190.stdout:/sys/fs/bpf 2024-04-17T22:31:02.911 INFO:teuthology.orchestra.run.smithi190.stdout:/sys/fs/cgroup/pids 2024-04-17T22:31:02.911 INFO:teuthology.orchestra.run.smithi190.stdout:/sys/fs/cgroup/cpu,cpuacct 2024-04-17T22:31:02.911 INFO:teuthology.orchestra.run.smithi190.stdout:/sys/fs/cgroup/net_cls,net_prio 2024-04-17T22:31:02.911 INFO:teuthology.orchestra.run.smithi190.stdout:/sys/fs/cgroup/rdma 2024-04-17T22:31:02.911 INFO:teuthology.orchestra.run.smithi190.stdout:/sys/fs/cgroup/hugetlb 2024-04-17T22:31:02.911 INFO:teuthology.orchestra.run.smithi190.stdout:/sys/fs/cgroup/blkio 2024-04-17T22:31:02.911 INFO:teuthology.orchestra.run.smithi190.stdout:/sys/fs/cgroup/freezer 2024-04-17T22:31:02.911 INFO:teuthology.orchestra.run.smithi190.stdout:/sys/fs/cgroup/cpuset 2024-04-17T22:31:02.911 INFO:teuthology.orchestra.run.smithi190.stdout:/sys/fs/cgroup/devices 2024-04-17T22:31:02.912 INFO:teuthology.orchestra.run.smithi190.stdout:/sys/fs/cgroup/memory 2024-04-17T22:31:02.912 INFO:teuthology.orchestra.run.smithi190.stdout:/sys/fs/cgroup/perf_event 2024-04-17T22:31:02.912 INFO:teuthology.orchestra.run.smithi190.stdout:/sys/kernel/tracing 2024-04-17T22:31:02.912 INFO:teuthology.orchestra.run.smithi190.stdout:/sys/kernel/config 2024-04-17T22:31:02.912 INFO:teuthology.orchestra.run.smithi190.stdout:/ 2024-04-17T22:31:02.912 INFO:teuthology.orchestra.run.smithi190.stdout:/sys/fs/selinux 2024-04-17T22:31:02.912 INFO:teuthology.orchestra.run.smithi190.stdout:/sys/kernel/debug 2024-04-17T22:31:02.912 INFO:teuthology.orchestra.run.smithi190.stdout:/dev/hugepages 2024-04-17T22:31:02.912 INFO:teuthology.orchestra.run.smithi190.stdout:/dev/mqueue 2024-04-17T22:31:02.912 INFO:teuthology.orchestra.run.smithi190.stdout:/proc/sys/fs/binfmt_misc 2024-04-17T22:31:02.912 INFO:teuthology.orchestra.run.smithi190.stdout:/sys/fs/fuse/connections 2024-04-17T22:31:02.912 INFO:teuthology.orchestra.run.smithi190.stdout:/var/lib/nfs/rpc_pipefs 2024-04-17T22:31:02.912 INFO:teuthology.orchestra.run.smithi190.stdout:/run/user/1000 2024-04-17T22:31:02.912 INFO:teuthology.orchestra.run.smithi190.stdout:/var/lib/ceph 2024-04-17T22:31:02.913 INFO:teuthology.orchestra.run.smithi190.stdout:/run/netns 2024-04-17T22:31:02.913 INFO:teuthology.orchestra.run.smithi190.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-17T22:31:02.913 INFO:teuthology.orchestra.run.smithi190.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-17T22:31:02.913 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T22:31:02.913 DEBUG:teuthology.orchestra.run.smithi190:> ls /sys/fs/fuse/connections 2024-04-17T22:31:02.967 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-17T22:31:02.967 DEBUG:teuthology.orchestra.run.smithi190:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-17T22:31:03.009 DEBUG:teuthology.orchestra.run.smithi190:> sudo modprobe fuse 2024-04-17T22:31:03.035 DEBUG:teuthology.orchestra.run.smithi190:> cat /proc/self/mounts | awk '{print $2}' 2024-04-17T22:31:03.091 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi190.stderr:2024-04-17T22:31:03.105+0000 7fbebd07d3c0 -1 init, newargv = 0x55778d305250 newargc=15 2024-04-17T22:31:03.091 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi190.stderr:ceph-fuse[42143]: starting ceph client 2024-04-17T22:31:03.094 INFO:teuthology.orchestra.run.smithi190.stdout:/sys 2024-04-17T22:31:03.094 INFO:teuthology.orchestra.run.smithi190.stdout:/proc 2024-04-17T22:31:03.094 INFO:teuthology.orchestra.run.smithi190.stdout:/dev 2024-04-17T22:31:03.094 INFO:teuthology.orchestra.run.smithi190.stdout:/sys/kernel/security 2024-04-17T22:31:03.094 INFO:teuthology.orchestra.run.smithi190.stdout:/dev/shm 2024-04-17T22:31:03.095 INFO:teuthology.orchestra.run.smithi190.stdout:/dev/pts 2024-04-17T22:31:03.095 INFO:teuthology.orchestra.run.smithi190.stdout:/run 2024-04-17T22:31:03.095 INFO:teuthology.orchestra.run.smithi190.stdout:/sys/fs/cgroup 2024-04-17T22:31:03.095 INFO:teuthology.orchestra.run.smithi190.stdout:/sys/fs/cgroup/systemd 2024-04-17T22:31:03.095 INFO:teuthology.orchestra.run.smithi190.stdout:/sys/fs/pstore 2024-04-17T22:31:03.095 INFO:teuthology.orchestra.run.smithi190.stdout:/sys/fs/bpf 2024-04-17T22:31:03.095 INFO:teuthology.orchestra.run.smithi190.stdout:/sys/fs/cgroup/pids 2024-04-17T22:31:03.095 INFO:teuthology.orchestra.run.smithi190.stdout:/sys/fs/cgroup/cpu,cpuacct 2024-04-17T22:31:03.095 INFO:teuthology.orchestra.run.smithi190.stdout:/sys/fs/cgroup/net_cls,net_prio 2024-04-17T22:31:03.095 INFO:teuthology.orchestra.run.smithi190.stdout:/sys/fs/cgroup/rdma 2024-04-17T22:31:03.095 INFO:teuthology.orchestra.run.smithi190.stdout:/sys/fs/cgroup/hugetlb 2024-04-17T22:31:03.095 INFO:teuthology.orchestra.run.smithi190.stdout:/sys/fs/cgroup/blkio 2024-04-17T22:31:03.095 INFO:teuthology.orchestra.run.smithi190.stdout:/sys/fs/cgroup/freezer 2024-04-17T22:31:03.095 INFO:teuthology.orchestra.run.smithi190.stdout:/sys/fs/cgroup/cpuset 2024-04-17T22:31:03.095 INFO:teuthology.orchestra.run.smithi190.stdout:/sys/fs/cgroup/devices 2024-04-17T22:31:03.095 INFO:teuthology.orchestra.run.smithi190.stdout:/sys/fs/cgroup/memory 2024-04-17T22:31:03.095 INFO:teuthology.orchestra.run.smithi190.stdout:/sys/fs/cgroup/perf_event 2024-04-17T22:31:03.096 INFO:teuthology.orchestra.run.smithi190.stdout:/sys/kernel/tracing 2024-04-17T22:31:03.096 INFO:teuthology.orchestra.run.smithi190.stdout:/sys/kernel/config 2024-04-17T22:31:03.096 INFO:teuthology.orchestra.run.smithi190.stdout:/ 2024-04-17T22:31:03.096 INFO:teuthology.orchestra.run.smithi190.stdout:/sys/fs/selinux 2024-04-17T22:31:03.096 INFO:teuthology.orchestra.run.smithi190.stdout:/sys/kernel/debug 2024-04-17T22:31:03.096 INFO:teuthology.orchestra.run.smithi190.stdout:/dev/hugepages 2024-04-17T22:31:03.096 INFO:teuthology.orchestra.run.smithi190.stdout:/dev/mqueue 2024-04-17T22:31:03.096 INFO:teuthology.orchestra.run.smithi190.stdout:/proc/sys/fs/binfmt_misc 2024-04-17T22:31:03.096 INFO:teuthology.orchestra.run.smithi190.stdout:/sys/fs/fuse/connections 2024-04-17T22:31:03.096 INFO:teuthology.orchestra.run.smithi190.stdout:/var/lib/nfs/rpc_pipefs 2024-04-17T22:31:03.096 INFO:teuthology.orchestra.run.smithi190.stdout:/run/user/1000 2024-04-17T22:31:03.096 INFO:teuthology.orchestra.run.smithi190.stdout:/var/lib/ceph 2024-04-17T22:31:03.096 INFO:teuthology.orchestra.run.smithi190.stdout:/run/netns 2024-04-17T22:31:03.096 INFO:teuthology.orchestra.run.smithi190.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-17T22:31:03.096 INFO:teuthology.orchestra.run.smithi190.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-17T22:31:03.097 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T22:31:03.097 DEBUG:teuthology.orchestra.run.smithi190:> ls /sys/fs/fuse/connections 2024-04-17T22:31:03.097 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi190.stderr:ceph-fuse[42143]: starting fuse 2024-04-17T22:31:03.114 INFO:teuthology.orchestra.run.smithi190.stdout:46 2024-04-17T22:31:03.114 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [46] 2024-04-17T22:31:03.114 DEBUG:teuthology.orchestra.run.smithi190:> sudo stdin-killer -- python3 -c ' 2024-04-17T22:31:03.114 DEBUG:teuthology.orchestra.run.smithi190:> import glob 2024-04-17T22:31:03.115 DEBUG:teuthology.orchestra.run.smithi190:> import re 2024-04-17T22:31:03.115 DEBUG:teuthology.orchestra.run.smithi190:> import os 2024-04-17T22:31:03.115 DEBUG:teuthology.orchestra.run.smithi190:> import subprocess 2024-04-17T22:31:03.115 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T22:31:03.115 DEBUG:teuthology.orchestra.run.smithi190:> def _find_admin_socket(client_name): 2024-04-17T22:31:03.115 DEBUG:teuthology.orchestra.run.smithi190:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-17T22:31:03.115 DEBUG:teuthology.orchestra.run.smithi190:> files = glob.glob(asok_path) 2024-04-17T22:31:03.115 DEBUG:teuthology.orchestra.run.smithi190:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-17T22:31:03.115 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T22:31:03.115 DEBUG:teuthology.orchestra.run.smithi190:> # Given a non-glob path, it better be there 2024-04-17T22:31:03.115 DEBUG:teuthology.orchestra.run.smithi190:> if "*" not in asok_path: 2024-04-17T22:31:03.115 DEBUG:teuthology.orchestra.run.smithi190:> assert(len(files) == 1) 2024-04-17T22:31:03.115 DEBUG:teuthology.orchestra.run.smithi190:> return files[0] 2024-04-17T22:31:03.115 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T22:31:03.115 DEBUG:teuthology.orchestra.run.smithi190:> for f in files: 2024-04-17T22:31:03.116 DEBUG:teuthology.orchestra.run.smithi190:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-17T22:31:03.116 DEBUG:teuthology.orchestra.run.smithi190:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-17T22:31:03.116 DEBUG:teuthology.orchestra.run.smithi190:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-17T22:31:03.116 DEBUG:teuthology.orchestra.run.smithi190:> contents = proc_f.read() 2024-04-17T22:31:03.116 DEBUG:teuthology.orchestra.run.smithi190:> if mountpoint in contents: 2024-04-17T22:31:03.116 DEBUG:teuthology.orchestra.run.smithi190:> return f 2024-04-17T22:31:03.116 DEBUG:teuthology.orchestra.run.smithi190:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-17T22:31:03.116 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T22:31:03.116 DEBUG:teuthology.orchestra.run.smithi190:> print(_find_admin_socket("client.0")) 2024-04-17T22:31:03.116 DEBUG:teuthology.orchestra.run.smithi190:> ' 2024-04-17T22:31:03.237 INFO:teuthology.orchestra.run.smithi190.stdout:/var/run/ceph/ceph-client.0.42143.asok 2024-04-17T22:31:03.240 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T22:31:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T22:31:03.246 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.42143.asok 2024-04-17T22:31:03.246 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T22:31:03.247 DEBUG:teuthology.orchestra.run.smithi190:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.42143.asok status 2024-04-17T22:31:03.397 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-04-17T22:31:03.397 INFO:teuthology.orchestra.run.smithi190.stdout: "metadata": { 2024-04-17T22:31:03.397 INFO:teuthology.orchestra.run.smithi190.stdout: "ceph_sha1": "618f440892089921c3e944a991122ddc44e60516", 2024-04-17T22:31:03.397 INFO:teuthology.orchestra.run.smithi190.stdout: "ceph_version": "ceph version 16.2.15 (618f440892089921c3e944a991122ddc44e60516) pacific (stable)", 2024-04-17T22:31:03.397 INFO:teuthology.orchestra.run.smithi190.stdout: "entity_id": "0", 2024-04-17T22:31:03.397 INFO:teuthology.orchestra.run.smithi190.stdout: "hostname": "smithi190", 2024-04-17T22:31:03.397 INFO:teuthology.orchestra.run.smithi190.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-17T22:31:03.397 INFO:teuthology.orchestra.run.smithi190.stdout: "pid": "42143", 2024-04-17T22:31:03.398 INFO:teuthology.orchestra.run.smithi190.stdout: "root": "/" 2024-04-17T22:31:03.398 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-04-17T22:31:03.398 INFO:teuthology.orchestra.run.smithi190.stdout: "dentry_count": 0, 2024-04-17T22:31:03.398 INFO:teuthology.orchestra.run.smithi190.stdout: "dentry_pinned_count": 0, 2024-04-17T22:31:03.398 INFO:teuthology.orchestra.run.smithi190.stdout: "id": 4959, 2024-04-17T22:31:03.398 INFO:teuthology.orchestra.run.smithi190.stdout: "inst": { 2024-04-17T22:31:03.398 INFO:teuthology.orchestra.run.smithi190.stdout: "name": { 2024-04-17T22:31:03.398 INFO:teuthology.orchestra.run.smithi190.stdout: "type": "client", 2024-04-17T22:31:03.398 INFO:teuthology.orchestra.run.smithi190.stdout: "num": 4959 2024-04-17T22:31:03.398 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-04-17T22:31:03.398 INFO:teuthology.orchestra.run.smithi190.stdout: "addr": { 2024-04-17T22:31:03.398 INFO:teuthology.orchestra.run.smithi190.stdout: "type": "v1", 2024-04-17T22:31:03.399 INFO:teuthology.orchestra.run.smithi190.stdout: "addr": "172.21.15.190:0", 2024-04-17T22:31:03.399 INFO:teuthology.orchestra.run.smithi190.stdout: "nonce": 1903263087 2024-04-17T22:31:03.399 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-04-17T22:31:03.399 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-04-17T22:31:03.399 INFO:teuthology.orchestra.run.smithi190.stdout: "addr": { 2024-04-17T22:31:03.399 INFO:teuthology.orchestra.run.smithi190.stdout: "type": "v1", 2024-04-17T22:31:03.399 INFO:teuthology.orchestra.run.smithi190.stdout: "addr": "172.21.15.190:0", 2024-04-17T22:31:03.399 INFO:teuthology.orchestra.run.smithi190.stdout: "nonce": 1903263087 2024-04-17T22:31:03.399 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-04-17T22:31:03.399 INFO:teuthology.orchestra.run.smithi190.stdout: "inst_str": "client.4959 172.21.15.190:0/1903263087", 2024-04-17T22:31:03.399 INFO:teuthology.orchestra.run.smithi190.stdout: "addr_str": "172.21.15.190:0/1903263087", 2024-04-17T22:31:03.399 INFO:teuthology.orchestra.run.smithi190.stdout: "inode_count": 1, 2024-04-17T22:31:03.399 INFO:teuthology.orchestra.run.smithi190.stdout: "mds_epoch": 5, 2024-04-17T22:31:03.399 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_epoch": 17, 2024-04-17T22:31:03.399 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_epoch_barrier": 0, 2024-04-17T22:31:03.399 INFO:teuthology.orchestra.run.smithi190.stdout: "blocklisted": false, 2024-04-17T22:31:03.400 INFO:teuthology.orchestra.run.smithi190.stdout: "fs_name": "cephfs" 2024-04-17T22:31:03.400 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-04-17T22:31:03.407 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T22:31:03.407 DEBUG:teuthology.orchestra.run.smithi190:> stat --file-system '--printf=%T 2024-04-17T22:31:03.407 DEBUG:teuthology.orchestra.run.smithi190:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-17T22:31:03.425 INFO:teuthology.orchestra.run.smithi190.stdout:fuseblk 2024-04-17T22:31:03.425 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-17T22:31:03.426 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T22:31:03.426 DEBUG:teuthology.orchestra.run.smithi190:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-17T22:31:03.492 INFO:teuthology.run_tasks:Running task print... 2024-04-17T22:31:03.503 INFO:teuthology.task.print:**** done remount client.0 2024-04-17T22:31:03.503 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2024-04-17T22:31:03.547 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T22:31:03.689 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:03.701 7fef7dfac700 1 Processor -- start 2024-04-17T22:31:03.689 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:03.701 7fef7dfac700 1 -- start start 2024-04-17T22:31:03.689 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:03.701 7fef7dfac700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fef780f5860 0x7fef7810b030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:03.689 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:03.701 7fef7dfac700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fef7810bab0 0x7fef781a0310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:03.690 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:03.701 7fef7dfac700 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef78103b60 con 0x7fef7810b660 2024-04-17T22:31:03.690 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:03.701 7fef7dfac700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef7810bf00 con 0x7fef780f5860 2024-04-17T22:31:03.690 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:03.701 7fef7dfac700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fef780fa390 con 0x7fef7810bab0 2024-04-17T22:31:03.690 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:03.701 7fef77573700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fef7810bab0 0x7fef781a0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:03.690 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:03.701 7fef6ffff700 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7fef7810b660 0x7fef7819fb10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:38644/0 (socket says 172.21.15.102:38644) 2024-04-17T22:31:03.690 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:03.701 7fef6ffff700 1 -- 172.21.15.102:0/2851182191 learned_addr learned my addr 172.21.15.102:0/2851182191 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-17T22:31:03.690 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:03.701 7fef76d72700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fef780f5860 0x7fef7810b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:03.690 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:03.702 7fef76571700 1 -- 172.21.15.102:0/2851182191 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4141374250 0 0) 0x7fef60001110 con 0x7fef7810b660 2024-04-17T22:31:03.690 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:03.702 7fef76571700 1 -- 172.21.15.102:0/2851182191 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fef58003270 con 0x7fef7810b660 2024-04-17T22:31:03.690 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:03.702 7fef76d72700 1 -- 172.21.15.102:0/2851182191 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fef7810bab0 msgr2=0x7fef781a0310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:31:03.690 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:03.702 7fef76d72700 1 --2- 172.21.15.102:0/2851182191 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fef7810bab0 0x7fef781a0310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:03.690 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:03.702 7fef76d72700 1 -- 172.21.15.102:0/2851182191 >> v1:172.21.15.102:6789/0 conn(0x7fef7810b660 legacy=0x7fef7819fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:03.690 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:03.702 7fef76d72700 1 -- 172.21.15.102:0/2851182191 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef781a0940 con 0x7fef780f5860 2024-04-17T22:31:03.691 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:03.702 7fef77573700 1 --2- 172.21.15.102:0/2851182191 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fef7810bab0 0x7fef781a0310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:03.691 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:03.702 7fef76d72700 1 --2- 172.21.15.102:0/2851182191 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fef780f5860 0x7fef7810b030 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 rx=0x7fef78103d80 tx=0x7fef6800ba50).ready entity=mon.1 client_cookie=cb3316df617c5c07 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:03.691 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:03.702 7fef76571700 1 -- 172.21.15.102:0/2851182191 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fef781a0940 con 0x7fef780f5860 2024-04-17T22:31:03.691 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:03.702 7fef76571700 1 -- 172.21.15.102:0/2851182191 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef6800d570 con 0x7fef780f5860 2024-04-17T22:31:03.691 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:03.702 7fef76571700 1 -- 172.21.15.102:0/2851182191 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fef6800d940 con 0x7fef780f5860 2024-04-17T22:31:03.691 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:03.702 7fef7dfac700 1 -- 172.21.15.102:0/2851182191 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fef780f5860 msgr2=0x7fef7810b030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:03.691 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:03.702 7fef7dfac700 1 --2- 172.21.15.102:0/2851182191 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fef780f5860 0x7fef7810b030 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 rx=0x7fef78103d80 tx=0x7fef6800ba50).stop 2024-04-17T22:31:03.691 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:03.703 7fef7dfac700 1 -- 172.21.15.102:0/2851182191 shutdown_connections 2024-04-17T22:31:03.691 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:03.703 7fef7dfac700 1 --2- 172.21.15.102:0/2851182191 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fef7810bab0 0x7fef781a0310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:03.691 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:03.703 7fef7dfac700 1 --2- 172.21.15.102:0/2851182191 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fef780f5860 0x7fef7810b030 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:03.691 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:03.703 7fef7dfac700 1 -- 172.21.15.102:0/2851182191 >> 172.21.15.102:0/2851182191 conn(0x7fef781138c0 msgr2=0x7fef780f4f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:03.691 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:03.703 7fef7dfac700 1 -- 172.21.15.102:0/2851182191 shutdown_connections 2024-04-17T22:31:03.691 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:03.703 7fef7dfac700 1 -- 172.21.15.102:0/2851182191 wait complete. 2024-04-17T22:31:03.713 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:03.726 7fef7dfac700 1 Processor -- start 2024-04-17T22:31:03.714 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:03.726 7fef7dfac700 1 -- start start 2024-04-17T22:31:03.714 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:03.726 7fef7dfac700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fef781a6070 0x7fef781ad850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:03.714 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:03.726 7fef7dfac700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fef781ade80 0x7fef780c6b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:03.714 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:03.726 7fef7dfac700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fef780c7150 0x7fef781bb250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:03.714 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:03.726 7fef7dfac700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef781ae2d0 con 0x7fef781ade80 2024-04-17T22:31:03.714 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:03.726 7fef7dfac700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef781a6cb0 con 0x7fef780c7150 2024-04-17T22:31:03.714 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:03.726 7fef7dfac700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fef781a64c0 con 0x7fef781a6070 2024-04-17T22:31:03.714 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:03.726 7fef6ffff700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fef781ade80 0x7fef780c6b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:03.714 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:03.726 7fef77573700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fef780c7150 0x7fef781bb250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:03.714 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:03.726 7fef6ffff700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fef781ade80 0x7fef780c6b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:53900/0 (socket says 172.21.15.102:53900) 2024-04-17T22:31:03.715 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:03.726 7fef6ffff700 1 -- 172.21.15.102:0/3866500930 learned_addr learned my addr 172.21.15.102:0/3866500930 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:31:03.715 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:03.726 7fef76d72700 1 --2- 172.21.15.102:0/3866500930 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fef781a6070 0x7fef781ad850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:03.715 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:03.726 7fef6ffff700 1 -- 172.21.15.102:0/3866500930 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fef781a6070 msgr2=0x7fef781ad850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:03.715 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:03.726 7fef6ffff700 1 --2- 172.21.15.102:0/3866500930 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fef781a6070 0x7fef781ad850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:03.715 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:03.726 7fef6ffff700 1 -- 172.21.15.102:0/3866500930 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fef780c7150 msgr2=0x7fef781bb250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:03.715 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:03.726 7fef6ffff700 1 --2- 172.21.15.102:0/3866500930 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fef780c7150 0x7fef781bb250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:03.715 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:03.726 7fef6ffff700 1 -- 172.21.15.102:0/3866500930 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef780c75a0 con 0x7fef781ade80 2024-04-17T22:31:03.715 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:03.726 7fef77573700 1 --2- 172.21.15.102:0/3866500930 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fef780c7150 0x7fef781bb250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:03.715 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:03.726 7fef76d72700 1 --2- 172.21.15.102:0/3866500930 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fef781a6070 0x7fef781ad850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:03.715 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:03.726 7fef6ffff700 1 --2- 172.21.15.102:0/3866500930 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fef781ade80 0x7fef780c6b20 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 rx=0x7fef780d8220 tx=0x7fef6000c810).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:03.715 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:03.727 7fef74d6e700 1 -- 172.21.15.102:0/3866500930 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fef780c75a0 con 0x7fef781ade80 2024-04-17T22:31:03.715 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:03.727 7fef74d6e700 1 -- 172.21.15.102:0/3866500930 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef6000ce20 con 0x7fef781ade80 2024-04-17T22:31:03.715 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:03.727 7fef7dfac700 1 -- 172.21.15.102:0/3866500930 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef780c78c0 con 0x7fef781ade80 2024-04-17T22:31:03.715 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:03.727 7fef74d6e700 1 -- 172.21.15.102:0/3866500930 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fef60005620 con 0x7fef781ade80 2024-04-17T22:31:03.715 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:03.727 7fef7dfac700 1 -- 172.21.15.102:0/3866500930 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef781bba60 con 0x7fef781ade80 2024-04-17T22:31:03.716 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:03.727 7fef74d6e700 1 -- 172.21.15.102:0/3866500930 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7fef60005820 con 0x7fef781ade80 2024-04-17T22:31:03.716 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:03.727 7fef74d6e700 1 -- 172.21.15.102:0/3866500930 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3194+0+0 (secure 0 0 0) 0x7fef60014800 con 0x7fef781ade80 2024-04-17T22:31:03.716 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:03.728 7fef7dfac700 1 -- 172.21.15.102:0/3866500930 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef780f0820 con 0x7fef781ade80 2024-04-17T22:31:03.717 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:03.731 7fef74d6e700 1 -- 172.21.15.102:0/3866500930 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+110455 (secure 0 0 0) 0x7fef600070d0 con 0x7fef781ade80 2024-04-17T22:31:03.821 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:03.835 7fef7dfac700 1 -- 172.21.15.102:0/3866500930 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fef781ae4b0 con 0x7fef781ade80 2024-04-17T22:31:03.822 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:03.836 7fef74d6e700 1 -- 172.21.15.102:0/3866500930 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) v1 ==== 74+0+8764 (secure 0 0 0) 0x7fef780f1460 con 0x7fef781ade80 2024-04-17T22:31:03.822 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-17T22:31:03.822 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":17,"fsid":"cb63e789-091d-49b1-b120-55f9db526871","created":"2024-04-17 22:28:48.209635","modified":"2024-04-17 22:29:23.257387","last_up_change":"2024-04-17 22:28:54.236215","last_in_change":"2024-04-17 22:28:54.236215","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":4,"require_min_compat_client":"nautilus","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-04-17 22:28:57.478545","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-17 22:29:06.144708","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-17 22:29:06.748924","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3351ef85-ade4-4b90-a8f4-213f4e00ba01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6812","nonce":42955}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6813","nonce":42955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6815","nonce":42955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6814","nonce":42955}]},"public_addr":"172.21.15.102:6812/42955","cluster_addr":"172.21.15.102:6813/42955","heartbeat_back_addr":"172.21.15.102:6815/42955","heartbeat_front_addr":"172.21.15.102:6814/42955","state":["exists","up"]},{"osd":1,"uuid":"04f67eac-f36b-48f5-982d-4cfc776596d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6808","nonce":42958}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6809","nonce":42958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6811","nonce":42958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6810","nonce":42958}]},"public_addr":"172.21.15.102:6808/42958","cluster_addr":"172.21.15.102:6809/42958","heartbeat_back_addr":"172.21.15.102:6811/42958","heartbeat_front_addr":"172.21.15.102:6810/42958","state":["exists","up"]},{"osd":2,"uuid":"d524204b-70b2-40a8-8f73-f9e7d44318af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6804","nonce":42954}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6805","nonce":42954}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6807","nonce":42954}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6806","nonce":42954}]},"public_addr":"172.21.15.102:6804/42954","cluster_addr":"172.21.15.102:6805/42954","heartbeat_back_addr":"172.21.15.102:6807/42954","heartbeat_front_addr":"172.21.15.102:6806/42954","state":["exists","up"]},{"osd":3,"uuid":"4fc9d20c-1ad0-484f-aafa-2c9a7574245b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6800","nonce":42953}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6801","nonce":42953}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6803","nonce":42953}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6802","nonce":42953}]},"public_addr":"172.21.15.102:6800/42953","cluster_addr":"172.21.15.102:6801/42953","heartbeat_back_addr":"172.21.15.102:6803/42953","heartbeat_front_addr":"172.21.15.102:6802/42953","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2024-04-17T22:31:03.823 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:03.837 7fef7dfac700 1 -- 172.21.15.102:0/3866500930 >> v1:172.21.15.102:6816/42360 conn(0x7fef5c0145a0 legacy=0x7fef5c016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:03.823 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:03.837 7fef7dfac700 1 -- 172.21.15.102:0/3866500930 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fef781ade80 msgr2=0x7fef780c6b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:03.823 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:03.837 7fef7dfac700 1 --2- 172.21.15.102:0/3866500930 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fef781ade80 0x7fef780c6b20 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 rx=0x7fef780d8220 tx=0x7fef6000c810).stop 2024-04-17T22:31:03.824 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:03.837 7fef7dfac700 1 -- 172.21.15.102:0/3866500930 shutdown_connections 2024-04-17T22:31:03.824 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:03.837 7fef7dfac700 1 --2- 172.21.15.102:0/3866500930 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fef780c7150 0x7fef781bb250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:03.824 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:03.837 7fef7dfac700 1 --2- 172.21.15.102:0/3866500930 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fef781a6070 0x7fef781ad850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:03.824 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:03.837 7fef7dfac700 1 --2- 172.21.15.102:0/3866500930 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fef781ade80 0x7fef780c6b20 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:03.824 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:03.837 7fef7dfac700 1 -- 172.21.15.102:0/3866500930 >> 172.21.15.102:0/3866500930 conn(0x7fef780f1880 msgr2=0x7fef780c9100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:03.824 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:03.837 7fef7dfac700 1 -- 172.21.15.102:0/3866500930 shutdown_connections 2024-04-17T22:31:03.824 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:03.837 7fef7dfac700 1 -- 172.21.15.102:0/3866500930 wait complete. 2024-04-17T22:31:03.834 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2024-04-17 22:28:57.478545', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '12', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-17 22:29:06.144708', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-17 22:29:06.748924', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-17T22:31:03.834 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2024-04-17T22:31:03.963 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:03.975 7facd6722700 1 Processor -- start 2024-04-17T22:31:03.963 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:03.975 7facd6722700 1 -- start start 2024-04-17T22:31:03.963 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:03.975 7facd6722700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7facd01a6690 0x7facd01a8af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:03.964 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:03.975 7facd6722700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7facd01a9120 0x7facd01ab580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:03.964 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:03.975 7facd6722700 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7facd00ad030 con 0x7facd019fde0 2024-04-17T22:31:03.964 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:03.975 7facd6722700 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7facd00f9d40 con 0x7facd01a9120 2024-04-17T22:31:03.964 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:03.975 7facd6722700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7facd00f2c90 con 0x7facd01a6690 2024-04-17T22:31:03.964 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:03.976 7faccf3b0700 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7facd019fde0 0x7facd01a0230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:38654/0 (socket says 172.21.15.102:38654) 2024-04-17T22:31:03.964 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:03.976 7facc7fff700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7facd01a6690 0x7facd01a8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:03.964 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:03.976 7faccf3b0700 1 -- 172.21.15.102:0/1311737825 learned_addr learned my addr 172.21.15.102:0/1311737825 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-17T22:31:03.964 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:03.976 7faccfbb1700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7facd01a9120 0x7facd01ab580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:03.964 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:03.976 7faccebaf700 1 -- 172.21.15.102:0/1311737825 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4052988771 0 0) 0x7facc0001110 con 0x7facd019fde0 2024-04-17T22:31:03.964 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:03.976 7faccebaf700 1 -- 172.21.15.102:0/1311737825 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7facb0003270 con 0x7facd019fde0 2024-04-17T22:31:03.964 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:03.976 7facc7fff700 1 -- 172.21.15.102:0/1311737825 >> v2:172.21.15.102:3300/0 conn(0x7facd01a9120 msgr2=0x7facd01ab580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:31:03.964 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:03.976 7facc7fff700 1 --2- 172.21.15.102:0/1311737825 >> v2:172.21.15.102:3300/0 conn(0x7facd01a9120 0x7facd01ab580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:03.964 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:03.976 7facc7fff700 1 -- 172.21.15.102:0/1311737825 >> v1:172.21.15.102:6789/0 conn(0x7facd019fde0 legacy=0x7facd01a0230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:03.964 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:03.976 7facc7fff700 1 -- 172.21.15.102:0/1311737825 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facd01a4030 con 0x7facd01a6690 2024-04-17T22:31:03.965 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:03.976 7facc7fff700 1 --2- 172.21.15.102:0/1311737825 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7facd01a6690 0x7facd01a8af0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0x7facb8002f70 tx=0x7facb800b970).ready entity=mon.2 client_cookie=3a7d698595409062 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:03.965 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:03.976 7faccebaf700 1 -- 172.21.15.102:0/1311737825 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7facd01a4030 con 0x7facd01a6690 2024-04-17T22:31:03.965 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:03.976 7faccebaf700 1 -- 172.21.15.102:0/1311737825 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facb800d5c0 con 0x7facd01a6690 2024-04-17T22:31:03.965 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:03.976 7faccebaf700 1 -- 172.21.15.102:0/1311737825 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7facb800d990 con 0x7facd01a6690 2024-04-17T22:31:03.965 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:03.976 7facd6722700 1 -- 172.21.15.102:0/1311737825 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7facd01a6690 msgr2=0x7facd01a8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:03.965 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:03.976 7facd6722700 1 --2- 172.21.15.102:0/1311737825 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7facd01a6690 0x7facd01a8af0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0x7facb8002f70 tx=0x7facb800b970).stop 2024-04-17T22:31:03.965 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:03.977 7facd6722700 1 -- 172.21.15.102:0/1311737825 shutdown_connections 2024-04-17T22:31:03.965 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:03.977 7facd6722700 1 --2- 172.21.15.102:0/1311737825 >> v2:172.21.15.102:3300/0 conn(0x7facd01a9120 0x7facd01ab580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:03.965 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:03.977 7facd6722700 1 --2- 172.21.15.102:0/1311737825 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7facd01a6690 0x7facd01a8af0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:03.965 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:03.977 7facd6722700 1 -- 172.21.15.102:0/1311737825 >> 172.21.15.102:0/1311737825 conn(0x7facd0113000 msgr2=0x7facd0113450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:03.965 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:03.977 7facd6722700 1 -- 172.21.15.102:0/1311737825 shutdown_connections 2024-04-17T22:31:03.965 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:03.977 7facd6722700 1 -- 172.21.15.102:0/1311737825 wait complete. 2024-04-17T22:31:03.987 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:03.999 7facd6722700 1 Processor -- start 2024-04-17T22:31:03.988 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:03.999 7facd6722700 1 -- start start 2024-04-17T22:31:03.988 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.000 7facd6722700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7facd01a30a0 0x7facd00d8130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:03.988 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.000 7facd6722700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7facd00d8760 0x7facd019ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:03.988 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.000 7facd6722700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7facd00d8bb0 0x7facd019e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:03.988 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.000 7facd6722700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facd00abb70 con 0x7facd00d8bb0 2024-04-17T22:31:03.988 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.000 7facd6722700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7facd00f27e0 con 0x7facd00d8760 2024-04-17T22:31:03.988 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.000 7facd6722700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7facd00d9670 con 0x7facd01a30a0 2024-04-17T22:31:03.988 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.000 7faccf3b0700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7facd01a30a0 0x7facd00d8130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:03.988 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.000 7faccf3b0700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7facd01a30a0 0x7facd00d8130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:54674/0 (socket says 172.21.15.102:54674) 2024-04-17T22:31:03.988 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.000 7faccf3b0700 1 -- 172.21.15.102:0/3447355199 learned_addr learned my addr 172.21.15.102:0/3447355199 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:31:03.988 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.000 7faccfbb1700 1 --2- 172.21.15.102:0/3447355199 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7facd00d8bb0 0x7facd019e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:03.988 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.000 7facc7fff700 1 --2- 172.21.15.102:0/3447355199 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7facd00d8760 0x7facd019ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:03.988 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.000 7faccf3b0700 1 -- 172.21.15.102:0/3447355199 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7facd00d8760 msgr2=0x7facd019ddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:03.989 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.000 7faccf3b0700 1 --2- 172.21.15.102:0/3447355199 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7facd00d8760 0x7facd019ddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:03.989 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.000 7faccf3b0700 1 -- 172.21.15.102:0/3447355199 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7facd00d8bb0 msgr2=0x7facd019e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:03.989 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.000 7faccf3b0700 1 --2- 172.21.15.102:0/3447355199 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7facd00d8bb0 0x7facd019e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:03.989 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.000 7faccf3b0700 1 -- 172.21.15.102:0/3447355199 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facd019ea20 con 0x7facd01a30a0 2024-04-17T22:31:03.989 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.000 7faccfbb1700 1 --2- 172.21.15.102:0/3447355199 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7facd00d8bb0 0x7facd019e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:03.989 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.000 7faccf3b0700 1 --2- 172.21.15.102:0/3447355199 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7facd01a30a0 0x7facd00d8130 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 rx=0x7facd00d2020 tx=0x7facc000c790).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:03.989 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.000 7faccd3ac700 1 -- 172.21.15.102:0/3447355199 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7facd019ea20 con 0x7facd01a30a0 2024-04-17T22:31:03.989 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.000 7faccd3ac700 1 -- 172.21.15.102:0/3447355199 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facc00072e0 con 0x7facd01a30a0 2024-04-17T22:31:03.989 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.000 7facd6722700 1 -- 172.21.15.102:0/3447355199 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7facd01bb710 con 0x7facd01a30a0 2024-04-17T22:31:03.989 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.000 7faccd3ac700 1 -- 172.21.15.102:0/3447355199 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7facc0005810 con 0x7facd01a30a0 2024-04-17T22:31:03.989 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.000 7facd6722700 1 -- 172.21.15.102:0/3447355199 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7facd01bbaf0 con 0x7facd01a30a0 2024-04-17T22:31:03.989 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.001 7faccd3ac700 1 -- 172.21.15.102:0/3447355199 <== mon.2 v2:172.21.15.102:3302/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7facc001b070 con 0x7facd01a30a0 2024-04-17T22:31:03.989 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.001 7faccd3ac700 1 -- 172.21.15.102:0/3447355199 <== mon.2 v2:172.21.15.102:3302/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3194+0+0 (secure 0 0 0) 0x7facc0004cc0 con 0x7facd01a30a0 2024-04-17T22:31:03.989 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.001 7facd6722700 1 -- 172.21.15.102:0/3447355199 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7facd00f09f0 con 0x7facd01a30a0 2024-04-17T22:31:03.990 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.004 7faccd3ac700 1 -- 172.21.15.102:0/3447355199 <== mon.2 v2:172.21.15.102:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+110455 (secure 0 0 0) 0x7facc00070d0 con 0x7facd01a30a0 2024-04-17T22:31:04.096 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.110 7facd6722700 1 -- 172.21.15.102:0/3447355199 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7facd00f00c0 con 0x7facd01a30a0 2024-04-17T22:31:04.096 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.110 7faccd3ac700 1 -- 172.21.15.102:0/3447355199 <== mon.2 v2:172.21.15.102:3302/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v17) v1 ==== 92+0+10 (secure 0 0 0) 0x7facd00fe760 con 0x7facd01a30a0 2024-04-17T22:31:04.096 INFO:teuthology.orchestra.run.smithi102.stdout:pg_num: 8 2024-04-17T22:31:04.098 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.111 7facd6722700 1 -- 172.21.15.102:0/3447355199 >> v1:172.21.15.102:6816/42360 conn(0x7facb40145a0 legacy=0x7facb4016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:04.098 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.111 7facd6722700 1 -- 172.21.15.102:0/3447355199 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7facd01a30a0 msgr2=0x7facd00d8130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:04.098 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.111 7facd6722700 1 --2- 172.21.15.102:0/3447355199 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7facd01a30a0 0x7facd00d8130 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 rx=0x7facd00d2020 tx=0x7facc000c790).stop 2024-04-17T22:31:04.098 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.111 7facd6722700 1 -- 172.21.15.102:0/3447355199 shutdown_connections 2024-04-17T22:31:04.098 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.111 7facd6722700 1 --2- 172.21.15.102:0/3447355199 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7facd00d8bb0 0x7facd019e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:04.098 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.111 7facd6722700 1 --2- 172.21.15.102:0/3447355199 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7facd01a30a0 0x7facd00d8130 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:04.098 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.111 7facd6722700 1 --2- 172.21.15.102:0/3447355199 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7facd00d8760 0x7facd019ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:04.098 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.111 7facd6722700 1 -- 172.21.15.102:0/3447355199 >> 172.21.15.102:0/3447355199 conn(0x7facd00e0e90 msgr2=0x7facd01a0780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:04.098 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.111 7facd6722700 1 -- 172.21.15.102:0/3447355199 shutdown_connections 2024-04-17T22:31:04.099 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.112 7facd6722700 1 -- 172.21.15.102:0/3447355199 wait complete. 2024-04-17T22:31:04.108 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-17T22:31:04.237 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.249 7f5e2c414700 1 Processor -- start 2024-04-17T22:31:04.237 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.249 7f5e2c414700 1 -- start start 2024-04-17T22:31:04.237 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.250 7f5e2c414700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5e2410e240 0x7f5e2410e690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:04.237 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.250 7f5e2c414700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5e240f1f30 0x7f5e241ab4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:04.237 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.250 7f5e2c414700 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e240f2380 con 0x7f5e2410ecc0 2024-04-17T22:31:04.238 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.250 7f5e2c414700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e240f9f10 con 0x7f5e240f1f30 2024-04-17T22:31:04.238 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.250 7f5e2c414700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5e240f9030 con 0x7f5e2410e240 2024-04-17T22:31:04.238 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.250 7f5e29a10700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5e240f1f30 0x7f5e241ab4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:04.238 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.250 7f5e2920f700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5e2410e240 0x7f5e2410e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:04.238 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.250 7f5e28a0e700 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f5e2410ecc0 0x7f5e240f1730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:38670/0 (socket says 172.21.15.102:38670) 2024-04-17T22:31:04.238 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.250 7f5e28a0e700 1 -- 172.21.15.102:0/2216477817 learned_addr learned my addr 172.21.15.102:0/2216477817 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-17T22:31:04.238 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.250 7f5e2920f700 1 -- 172.21.15.102:0/2216477817 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5e240f1f30 msgr2=0x7f5e241ab4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:31:04.238 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.250 7f5e2920f700 1 --2- 172.21.15.102:0/2216477817 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5e240f1f30 0x7f5e241ab4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:04.238 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.250 7f5e2920f700 1 -- 172.21.15.102:0/2216477817 >> v1:172.21.15.102:6789/0 conn(0x7f5e2410ecc0 legacy=0x7f5e240f1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:04.238 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.250 7f5e13fff700 1 -- 172.21.15.102:0/2216477817 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2486586739 0 0) 0x7f5e20001110 con 0x7f5e2410ecc0 2024-04-17T22:31:04.238 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.250 7f5e2920f700 1 -- 172.21.15.102:0/2216477817 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e240f25a0 con 0x7f5e2410e240 2024-04-17T22:31:04.238 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.250 7f5e29a10700 1 --2- 172.21.15.102:0/2216477817 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5e240f1f30 0x7f5e241ab4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:04.238 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.250 7f5e2920f700 1 --2- 172.21.15.102:0/2216477817 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5e2410e240 0x7f5e2410e690 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 rx=0x7f5e240f4470 tx=0x7f5e1400b9d0).ready entity=mon.2 client_cookie=90b4ca71d7dd3aee server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:04.239 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.250 7f5e13fff700 1 -- 172.21.15.102:0/2216477817 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f5e240f25a0 con 0x7f5e2410e240 2024-04-17T22:31:04.239 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.250 7f5e13fff700 1 -- 172.21.15.102:0/2216477817 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e1400d570 con 0x7f5e2410e240 2024-04-17T22:31:04.239 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.250 7f5e13fff700 1 -- 172.21.15.102:0/2216477817 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f5e1400d940 con 0x7f5e2410e240 2024-04-17T22:31:04.239 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.250 7f5e2c414700 1 -- 172.21.15.102:0/2216477817 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5e2410e240 msgr2=0x7f5e2410e690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:04.239 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.250 7f5e2c414700 1 --2- 172.21.15.102:0/2216477817 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5e2410e240 0x7f5e2410e690 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 rx=0x7f5e240f4470 tx=0x7f5e1400b9d0).stop 2024-04-17T22:31:04.239 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.251 7f5e2c414700 1 -- 172.21.15.102:0/2216477817 shutdown_connections 2024-04-17T22:31:04.239 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.251 7f5e2c414700 1 --2- 172.21.15.102:0/2216477817 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5e240f1f30 0x7f5e241ab4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:04.239 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.251 7f5e2c414700 1 --2- 172.21.15.102:0/2216477817 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5e2410e240 0x7f5e2410e690 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:04.239 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.251 7f5e2c414700 1 -- 172.21.15.102:0/2216477817 >> 172.21.15.102:0/2216477817 conn(0x7f5e241138a0 msgr2=0x7f5e240f4f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:04.239 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.251 7f5e2c414700 1 -- 172.21.15.102:0/2216477817 shutdown_connections 2024-04-17T22:31:04.239 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.251 7f5e2c414700 1 -- 172.21.15.102:0/2216477817 wait complete. 2024-04-17T22:31:04.260 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.274 7f5e2c414700 1 Processor -- start 2024-04-17T22:31:04.260 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.274 7f5e2c414700 1 -- start start 2024-04-17T22:31:04.262 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.274 7f5e2c414700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5e240dde90 0x7f5e241ad740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:04.262 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.274 7f5e2c414700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5e241add70 0x7f5e241b41d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:04.262 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.274 7f5e2c414700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5e241ae1c0 0x7f5e241b4800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:04.262 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.274 7f5e2c414700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e240e2c80 con 0x7f5e241add70 2024-04-17T22:31:04.262 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.274 7f5e2c414700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e240fa1b0 con 0x7f5e240dde90 2024-04-17T22:31:04.262 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.274 7f5e2c414700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5e240c47e0 con 0x7f5e241ae1c0 2024-04-17T22:31:04.263 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.274 7f5e2920f700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5e240dde90 0x7f5e241ad740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:04.263 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.274 7f5e28a0e700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5e241add70 0x7f5e241b41d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:04.263 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.274 7f5e29a10700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5e241ae1c0 0x7f5e241b4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:04.263 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.274 7f5e2920f700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5e240dde90 0x7f5e241ad740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:51842/0 (socket says 172.21.15.102:51842) 2024-04-17T22:31:04.263 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.274 7f5e28a0e700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5e241add70 0x7f5e241b41d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:53918/0 (socket says 172.21.15.102:53918) 2024-04-17T22:31:04.263 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.274 7f5e2920f700 1 -- 172.21.15.102:0/3261591652 learned_addr learned my addr 172.21.15.102:0/3261591652 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:31:04.263 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.275 7f5e2920f700 1 -- 172.21.15.102:0/3261591652 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5e241ae1c0 msgr2=0x7f5e241b4800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:04.263 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.275 7f5e2920f700 1 --2- 172.21.15.102:0/3261591652 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5e241ae1c0 0x7f5e241b4800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:04.263 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.275 7f5e2920f700 1 -- 172.21.15.102:0/3261591652 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5e241add70 msgr2=0x7f5e241b41d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:04.263 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.275 7f5e2920f700 1 --2- 172.21.15.102:0/3261591652 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5e241add70 0x7f5e241b41d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:04.263 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.275 7f5e2920f700 1 -- 172.21.15.102:0/3261591652 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e240d9050 con 0x7f5e240dde90 2024-04-17T22:31:04.263 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.275 7f5e29a10700 1 --2- 172.21.15.102:0/3261591652 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5e241ae1c0 0x7f5e241b4800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:04.263 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.275 7f5e28a0e700 1 --2- 172.21.15.102:0/3261591652 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5e241add70 0x7f5e241b41d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:04.263 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.275 7f5e2920f700 1 --2- 172.21.15.102:0/3261591652 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5e240dde90 0x7f5e241ad740 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 rx=0x7f5e240d78b0 tx=0x7f5e140068c0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:04.264 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.275 7f5e127fc700 1 -- 172.21.15.102:0/3261591652 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f5e240d9050 con 0x7f5e240dde90 2024-04-17T22:31:04.264 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.275 7f5e127fc700 1 -- 172.21.15.102:0/3261591652 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e140118b0 con 0x7f5e240dde90 2024-04-17T22:31:04.264 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.275 7f5e2c414700 1 -- 172.21.15.102:0/3261591652 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e241b4e30 con 0x7f5e240dde90 2024-04-17T22:31:04.264 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.275 7f5e127fc700 1 -- 172.21.15.102:0/3261591652 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f5e14017040 con 0x7f5e240dde90 2024-04-17T22:31:04.264 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.275 7f5e2c414700 1 -- 172.21.15.102:0/3261591652 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e241bb8c0 con 0x7f5e240dde90 2024-04-17T22:31:04.264 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.276 7f5e127fc700 1 -- 172.21.15.102:0/3261591652 <== mon.1 v2:172.21.15.102:3301/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7f5e14011c80 con 0x7f5e240dde90 2024-04-17T22:31:04.264 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.276 7f5e127fc700 1 -- 172.21.15.102:0/3261591652 <== mon.1 v2:172.21.15.102:3301/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3194+0+0 (secure 0 0 0) 0x7f5e14018070 con 0x7f5e240dde90 2024-04-17T22:31:04.264 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.276 7f5e2c414700 1 -- 172.21.15.102:0/3261591652 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e240f0800 con 0x7f5e240dde90 2024-04-17T22:31:04.265 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.279 7f5e127fc700 1 -- 172.21.15.102:0/3261591652 <== mon.1 v2:172.21.15.102:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+110455 (secure 0 0 0) 0x7f5e140116a0 con 0x7f5e240dde90 2024-04-17T22:31:04.371 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.384 7f5e2c414700 1 -- 172.21.15.102:0/3261591652 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f5e241ae8f0 con 0x7f5e240dde90 2024-04-17T22:31:04.371 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.384 7f5e127fc700 1 -- 172.21.15.102:0/3261591652 <== mon.1 v2:172.21.15.102:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v17) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5e1400d360 con 0x7f5e240dde90 2024-04-17T22:31:04.371 INFO:teuthology.orchestra.run.smithi102.stdout:pg_num: 64 2024-04-17T22:31:04.373 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.386 7f5e2c414700 1 -- 172.21.15.102:0/3261591652 >> v1:172.21.15.102:6816/42360 conn(0x7f5e040145a0 legacy=0x7f5e04016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:04.373 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.386 7f5e2c414700 1 -- 172.21.15.102:0/3261591652 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5e240dde90 msgr2=0x7f5e241ad740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:04.374 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.386 7f5e2c414700 1 --2- 172.21.15.102:0/3261591652 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5e240dde90 0x7f5e241ad740 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 rx=0x7f5e240d78b0 tx=0x7f5e140068c0).stop 2024-04-17T22:31:04.374 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.386 7f5e2c414700 1 -- 172.21.15.102:0/3261591652 shutdown_connections 2024-04-17T22:31:04.374 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.386 7f5e2c414700 1 --2- 172.21.15.102:0/3261591652 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5e241ae1c0 0x7f5e241b4800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:04.374 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.386 7f5e2c414700 1 --2- 172.21.15.102:0/3261591652 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5e240dde90 0x7f5e241ad740 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:04.374 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.386 7f5e2c414700 1 --2- 172.21.15.102:0/3261591652 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5e241add70 0x7f5e241b41d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:04.374 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.386 7f5e2c414700 1 -- 172.21.15.102:0/3261591652 >> 172.21.15.102:0/3261591652 conn(0x7f5e240c6af0 msgr2=0x7f5e241a84e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:04.374 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.386 7f5e2c414700 1 -- 172.21.15.102:0/3261591652 shutdown_connections 2024-04-17T22:31:04.374 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.386 7f5e2c414700 1 -- 172.21.15.102:0/3261591652 wait complete. 2024-04-17T22:31:04.383 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-17T22:31:04.510 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.524 7ff01e183700 1 Processor -- start 2024-04-17T22:31:04.510 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.524 7ff01e183700 1 -- start start 2024-04-17T22:31:04.513 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.524 7ff01e183700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7ff01810f040 0x7ff0181a83c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:04.513 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.524 7ff01e183700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7ff0181a89f0 0x7ff0181aae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:04.513 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.524 7ff01e183700 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff018103670 con 0x7ff0180f5be0 2024-04-17T22:31:04.513 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.524 7ff01e183700 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff018110e50 con 0x7ff01810f040 2024-04-17T22:31:04.513 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.524 7ff01e183700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff0181ac490 con 0x7ff0181a89f0 2024-04-17T22:31:04.513 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.525 7ff0165c5700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7ff01810f040 0x7ff0181a83c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:04.513 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.525 7ff0175c7700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7ff0181a89f0 0x7ff0181aae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:04.514 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.525 7ff0165c5700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7ff01810f040 0x7ff0181a83c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:53930/0 (socket says 172.21.15.102:53930) 2024-04-17T22:31:04.514 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.525 7ff0165c5700 1 -- 172.21.15.102:0/901089748 learned_addr learned my addr 172.21.15.102:0/901089748 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:31:04.514 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.525 7ff0165c5700 1 -- 172.21.15.102:0/901089748 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7ff0181a89f0 msgr2=0x7ff0181aae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:31:04.514 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.525 7ff0165c5700 1 --2- 172.21.15.102:0/901089748 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7ff0181a89f0 0x7ff0181aae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:04.514 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.525 7ff0165c5700 1 -- 172.21.15.102:0/901089748 >> v1:172.21.15.102:6789/0 conn(0x7ff0180f5be0 legacy=0x7ff01810e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:04.514 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.525 7ff015dc4700 1 -- 172.21.15.102:0/901089748 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3057769089 0 0) 0x7ff00c001110 con 0x7ff0180f5be0 2024-04-17T22:31:04.514 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.525 7ff0165c5700 1 -- 172.21.15.102:0/901089748 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0181ac640 con 0x7ff01810f040 2024-04-17T22:31:04.514 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.525 7ff0175c7700 1 --2- 172.21.15.102:0/901089748 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7ff0181a89f0 0x7ff0181aae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:04.514 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.525 7ff0165c5700 1 --2- 172.21.15.102:0/901089748 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff01810f040 0x7ff0181a83c0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 rx=0x7ff018113420 tx=0x7ff008009a80).ready entity=mon.0 client_cookie=beec85a0b8c2d492 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:04.514 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.525 7ff015dc4700 1 -- 172.21.15.102:0/901089748 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff0181ac640 con 0x7ff01810f040 2024-04-17T22:31:04.514 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.525 7ff015dc4700 1 -- 172.21.15.102:0/901089748 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff00800f510 con 0x7ff01810f040 2024-04-17T22:31:04.514 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.525 7ff015dc4700 1 -- 172.21.15.102:0/901089748 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff00800f6f0 con 0x7ff01810f040 2024-04-17T22:31:04.514 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.525 7ff01e183700 1 -- 172.21.15.102:0/901089748 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff01810f040 msgr2=0x7ff0181a83c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:04.514 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.525 7ff01e183700 1 --2- 172.21.15.102:0/901089748 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff01810f040 0x7ff0181a83c0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 rx=0x7ff018113420 tx=0x7ff008009a80).stop 2024-04-17T22:31:04.515 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.526 7ff01e183700 1 -- 172.21.15.102:0/901089748 shutdown_connections 2024-04-17T22:31:04.515 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.526 7ff01e183700 1 --2- 172.21.15.102:0/901089748 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7ff0181a89f0 0x7ff0181aae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:04.515 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.526 7ff01e183700 1 --2- 172.21.15.102:0/901089748 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff01810f040 0x7ff0181a83c0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:04.515 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.526 7ff01e183700 1 -- 172.21.15.102:0/901089748 >> 172.21.15.102:0/901089748 conn(0x7ff018113a90 msgr2=0x7ff0180f5140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:04.515 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.526 7ff01e183700 1 -- 172.21.15.102:0/901089748 shutdown_connections 2024-04-17T22:31:04.515 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.526 7ff01e183700 1 -- 172.21.15.102:0/901089748 wait complete. 2024-04-17T22:31:04.536 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.548 7ff01e183700 1 Processor -- start 2024-04-17T22:31:04.536 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.548 7ff01e183700 1 -- start start 2024-04-17T22:31:04.537 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.549 7ff01e183700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7ff0180c50c0 0x7ff0180ce3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:04.537 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.549 7ff01e183700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff0180ce9d0 0x7ff0180d2ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:04.537 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.549 7ff01e183700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ff0180cee20 0x7ff0180d3100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:04.537 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.549 7ff01e183700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0180d4e90 con 0x7ff0180ce9d0 2024-04-17T22:31:04.537 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.549 7ff01e183700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0180d3730 con 0x7ff0180cee20 2024-04-17T22:31:04.537 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.549 7ff01e183700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff0180d3960 con 0x7ff0180c50c0 2024-04-17T22:31:04.537 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.549 7ff016dc6700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7ff0180c50c0 0x7ff0180ce3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:04.537 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.549 7ff0165c5700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff0180ce9d0 0x7ff0180d2ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:04.537 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.549 7ff0175c7700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ff0180cee20 0x7ff0180d3100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:04.537 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.549 7ff0165c5700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff0180ce9d0 0x7ff0180d2ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:53946/0 (socket says 172.21.15.102:53946) 2024-04-17T22:31:04.537 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.549 7ff016dc6700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7ff0180c50c0 0x7ff0180ce3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:54726/0 (socket says 172.21.15.102:54726) 2024-04-17T22:31:04.537 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.549 7ff0165c5700 1 -- 172.21.15.102:0/2670710347 learned_addr learned my addr 172.21.15.102:0/2670710347 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:31:04.537 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.549 7ff0175c7700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ff0180cee20 0x7ff0180d3100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:51858/0 (socket says 172.21.15.102:51858) 2024-04-17T22:31:04.537 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.549 7ff016dc6700 1 -- 172.21.15.102:0/2670710347 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ff0180cee20 msgr2=0x7ff0180d3100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:04.538 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.549 7ff016dc6700 1 --2- 172.21.15.102:0/2670710347 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ff0180cee20 0x7ff0180d3100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:04.538 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.549 7ff016dc6700 1 -- 172.21.15.102:0/2670710347 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff0180ce9d0 msgr2=0x7ff0180d2ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:04.538 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.549 7ff016dc6700 1 --2- 172.21.15.102:0/2670710347 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff0180ce9d0 0x7ff0180d2ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:04.538 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.549 7ff016dc6700 1 -- 172.21.15.102:0/2670710347 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0181bb5e0 con 0x7ff0180c50c0 2024-04-17T22:31:04.538 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.549 7ff0165c5700 1 --2- 172.21.15.102:0/2670710347 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff0180ce9d0 0x7ff0180d2ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:04.538 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.549 7ff016dc6700 1 --2- 172.21.15.102:0/2670710347 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7ff0180c50c0 0x7ff0180ce3a0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0x7ff00c000bc0 tx=0x7ff00c00c790).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:04.538 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.549 7ff007fff700 1 -- 172.21.15.102:0/2670710347 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff0181bb5e0 con 0x7ff0180c50c0 2024-04-17T22:31:04.538 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.549 7ff007fff700 1 -- 172.21.15.102:0/2670710347 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff00c007030 con 0x7ff0180c50c0 2024-04-17T22:31:04.538 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.550 7ff01e183700 1 -- 172.21.15.102:0/2670710347 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0181bb7e0 con 0x7ff0180c50c0 2024-04-17T22:31:04.538 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.550 7ff007fff700 1 -- 172.21.15.102:0/2670710347 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff00c006520 con 0x7ff0180c50c0 2024-04-17T22:31:04.538 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.550 7ff01e183700 1 -- 172.21.15.102:0/2670710347 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0181bbbc0 con 0x7ff0180c50c0 2024-04-17T22:31:04.538 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.550 7ff007fff700 1 -- 172.21.15.102:0/2670710347 <== mon.2 v2:172.21.15.102:3302/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7ff00c01a020 con 0x7ff0180c50c0 2024-04-17T22:31:04.538 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.550 7ff007fff700 1 -- 172.21.15.102:0/2670710347 <== mon.2 v2:172.21.15.102:3302/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3194+0+0 (secure 0 0 0) 0x7ff00c003c20 con 0x7ff0180c50c0 2024-04-17T22:31:04.538 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.551 7ff01e183700 1 -- 172.21.15.102:0/2670710347 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0180f09f0 con 0x7ff0180c50c0 2024-04-17T22:31:04.539 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.554 7ff007fff700 1 -- 172.21.15.102:0/2670710347 <== mon.2 v2:172.21.15.102:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+110455 (secure 0 0 0) 0x7ff00c007dd0 con 0x7ff0180c50c0 2024-04-17T22:31:04.645 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.659 7ff01e183700 1 -- 172.21.15.102:0/2670710347 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff0181ae660 con 0x7ff0180c50c0 2024-04-17T22:31:04.645 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.659 7ff007fff700 1 -- 172.21.15.102:0/2670710347 <== mon.2 v2:172.21.15.102:3302/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v17) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff00c000c30 con 0x7ff0180c50c0 2024-04-17T22:31:04.646 INFO:teuthology.orchestra.run.smithi102.stdout:pg_num: 64 2024-04-17T22:31:04.647 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.660 7ff01e183700 1 -- 172.21.15.102:0/2670710347 >> v1:172.21.15.102:6816/42360 conn(0x7feff40145a0 legacy=0x7feff4016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:04.647 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.660 7ff01e183700 1 -- 172.21.15.102:0/2670710347 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7ff0180c50c0 msgr2=0x7ff0180ce3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:04.647 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.660 7ff01e183700 1 --2- 172.21.15.102:0/2670710347 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7ff0180c50c0 0x7ff0180ce3a0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0x7ff00c000bc0 tx=0x7ff00c00c790).stop 2024-04-17T22:31:04.647 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.660 7ff01e183700 1 -- 172.21.15.102:0/2670710347 shutdown_connections 2024-04-17T22:31:04.647 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.660 7ff01e183700 1 --2- 172.21.15.102:0/2670710347 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ff0180cee20 0x7ff0180d3100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:04.647 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.660 7ff01e183700 1 --2- 172.21.15.102:0/2670710347 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7ff0180c50c0 0x7ff0180ce3a0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:04.647 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.661 7ff01e183700 1 --2- 172.21.15.102:0/2670710347 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff0180ce9d0 0x7ff0180d2ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:04.647 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.661 7ff01e183700 1 -- 172.21.15.102:0/2670710347 >> 172.21.15.102:0/2670710347 conn(0x7ff0180c5a00 msgr2=0x7ff0180bd930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:04.647 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.661 7ff01e183700 1 -- 172.21.15.102:0/2670710347 shutdown_connections 2024-04-17T22:31:04.648 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.661 7ff01e183700 1 -- 172.21.15.102:0/2670710347 wait complete. 2024-04-17T22:31:04.657 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T22:31:04.782 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.796 7f0539993700 1 Processor -- start 2024-04-17T22:31:04.782 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.796 7f0539993700 1 -- start start 2024-04-17T22:31:04.784 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.796 7f0539993700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f05340f5770 0x7f053410c110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:04.784 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.796 7f0539993700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f053410c740 0x7f05341a0c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:04.785 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.796 7f0539993700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f053410cb90 0x7f05341a1230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:04.785 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.796 7f0539993700 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05340f93e0 con 0x7f053410cb90 2024-04-17T22:31:04.785 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.796 7f0539993700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05341a1860 con 0x7f05340f5770 2024-04-17T22:31:04.785 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.796 7f0539993700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f05341a1a90 con 0x7f053410c740 2024-04-17T22:31:04.785 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.796 7f05325c5700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f05340f5770 0x7f053410c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:04.785 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.796 7f0531dc4700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f053410c740 0x7f05341a0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:04.785 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.796 7f0532dc6700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f053410cb90 0x7f05341a1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:04.785 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.796 7f05325c5700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f05340f5770 0x7f053410c110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:51868/0 (socket says 172.21.15.102:51868) 2024-04-17T22:31:04.785 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.796 7f05325c5700 1 -- 172.21.15.102:0/628192278 learned_addr learned my addr 172.21.15.102:0/628192278 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:31:04.785 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.796 7f0531dc4700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f053410c740 0x7f05341a0c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:54732/0 (socket says 172.21.15.102:54732) 2024-04-17T22:31:04.785 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.796 7f0532dc6700 1 --2- 172.21.15.102:0/628192278 >> v2:172.21.15.102:3300/0 conn(0x7f053410cb90 0x7f05341a1230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:53950/0 (socket says 172.21.15.102:53950) 2024-04-17T22:31:04.785 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.797 7f0532dc6700 1 -- 172.21.15.102:0/628192278 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f053410c740 msgr2=0x7f05341a0c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:31:04.785 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.797 7f0532dc6700 1 --2- 172.21.15.102:0/628192278 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f053410c740 0x7f05341a0c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:04.785 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.797 7f0532dc6700 1 -- 172.21.15.102:0/628192278 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f05340f5770 msgr2=0x7f053410c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:31:04.786 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.797 7f0532dc6700 1 --2- 172.21.15.102:0/628192278 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f05340f5770 0x7f053410c110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:04.786 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.797 7f0532dc6700 1 -- 172.21.15.102:0/628192278 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05341a8310 con 0x7f053410cb90 2024-04-17T22:31:04.786 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.797 7f05325c5700 1 --2- 172.21.15.102:0/628192278 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f05340f5770 0x7f053410c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:04.786 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.797 7f0531dc4700 1 --2- 172.21.15.102:0/628192278 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f053410c740 0x7f05341a0c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:04.786 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.797 7f0532dc6700 1 --2- 172.21.15.102:0/628192278 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f053410cb90 0x7f05341a1230 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 rx=0x7f052c0032f0 tx=0x7f052c009d10).ready entity=mon.0 client_cookie=fcb9cc67a152236f server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:04.786 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.797 7f05315c3700 1 -- 172.21.15.102:0/628192278 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f05341a8310 con 0x7f053410cb90 2024-04-17T22:31:04.786 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.797 7f05315c3700 1 -- 172.21.15.102:0/628192278 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f052c00b800 con 0x7f053410cb90 2024-04-17T22:31:04.786 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.797 7f05315c3700 1 -- 172.21.15.102:0/628192278 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f052c00b9e0 con 0x7f053410cb90 2024-04-17T22:31:04.786 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.797 7f0539993700 1 -- 172.21.15.102:0/628192278 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f053410cb90 msgr2=0x7f05341a1230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:04.786 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.797 7f0539993700 1 --2- 172.21.15.102:0/628192278 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f053410cb90 0x7f05341a1230 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 rx=0x7f052c0032f0 tx=0x7f052c009d10).stop 2024-04-17T22:31:04.786 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.798 7f0539993700 1 -- 172.21.15.102:0/628192278 shutdown_connections 2024-04-17T22:31:04.786 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.798 7f0539993700 1 --2- 172.21.15.102:0/628192278 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f053410cb90 0x7f05341a1230 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:04.786 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.798 7f0539993700 1 --2- 172.21.15.102:0/628192278 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f05340f5770 0x7f053410c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:04.787 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.798 7f0539993700 1 --2- 172.21.15.102:0/628192278 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f053410c740 0x7f05341a0c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:04.787 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.798 7f0539993700 1 -- 172.21.15.102:0/628192278 >> 172.21.15.102:0/628192278 conn(0x7f05341138c0 msgr2=0x7f05340f4f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:04.787 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.798 7f0539993700 1 -- 172.21.15.102:0/628192278 shutdown_connections 2024-04-17T22:31:04.787 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.798 7f0539993700 1 -- 172.21.15.102:0/628192278 wait complete. 2024-04-17T22:31:04.808 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.820 7f0539993700 1 Processor -- start 2024-04-17T22:31:04.808 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.820 7f0539993700 1 -- start start 2024-04-17T22:31:04.809 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.820 7f0539993700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f05341a98c0 0x7f05341a9d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:04.809 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.820 7f0539993700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f05341aa340 0x7f05340ddff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:04.809 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.820 7f0539993700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f05340de620 0x7f05340dea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:04.809 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.820 7f0539993700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05340cafd0 con 0x7f05341a98c0 2024-04-17T22:31:04.809 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.820 7f0539993700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05340c3fa0 con 0x7f05340de620 2024-04-17T22:31:04.809 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.820 7f0539993700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f05340da550 con 0x7f05341aa340 2024-04-17T22:31:04.809 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.820 7f05325c5700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f05341a98c0 0x7f05341a9d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:04.809 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.821 7f0532dc6700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f05340de620 0x7f05340dea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:04.809 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.821 7f05325c5700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f05341a98c0 0x7f05341a9d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:53954/0 (socket says 172.21.15.102:53954) 2024-04-17T22:31:04.809 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.821 7f05325c5700 1 -- 172.21.15.102:0/3904498901 learned_addr learned my addr 172.21.15.102:0/3904498901 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:31:04.809 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.821 7f0531dc4700 1 --2- 172.21.15.102:0/3904498901 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f05341aa340 0x7f05340ddff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:04.809 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.821 7f05325c5700 1 -- 172.21.15.102:0/3904498901 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f05341aa340 msgr2=0x7f05340ddff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:04.809 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.821 7f05325c5700 1 --2- 172.21.15.102:0/3904498901 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f05341aa340 0x7f05340ddff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:04.809 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.821 7f05325c5700 1 -- 172.21.15.102:0/3904498901 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f05340de620 msgr2=0x7f05340dea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:04.810 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.821 7f05325c5700 1 --2- 172.21.15.102:0/3904498901 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f05340de620 0x7f05340dea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:04.810 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.821 7f05325c5700 1 -- 172.21.15.102:0/3904498901 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05340df500 con 0x7f05341a98c0 2024-04-17T22:31:04.810 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.821 7f0532dc6700 1 --2- 172.21.15.102:0/3904498901 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f05340de620 0x7f05340dea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:04.810 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.821 7f05325c5700 1 --2- 172.21.15.102:0/3904498901 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f05341a98c0 0x7f05341a9d10 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 rx=0x7f05340cf690 tx=0x7f052800ed00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:04.810 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.821 7f05237fe700 1 -- 172.21.15.102:0/3904498901 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f05340df500 con 0x7f05341a98c0 2024-04-17T22:31:04.810 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.821 7f05237fe700 1 -- 172.21.15.102:0/3904498901 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f052800ee40 con 0x7f05341a98c0 2024-04-17T22:31:04.810 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.821 7f0539993700 1 -- 172.21.15.102:0/3904498901 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05341baf50 con 0x7f05341a98c0 2024-04-17T22:31:04.810 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.821 7f05237fe700 1 -- 172.21.15.102:0/3904498901 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f052800f040 con 0x7f05341a98c0 2024-04-17T22:31:04.810 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.821 7f0539993700 1 -- 172.21.15.102:0/3904498901 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05341bb330 con 0x7f05341a98c0 2024-04-17T22:31:04.810 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.822 7f05237fe700 1 -- 172.21.15.102:0/3904498901 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7f0528011070 con 0x7f05341a98c0 2024-04-17T22:31:04.810 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.822 7f05237fe700 1 -- 172.21.15.102:0/3904498901 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3194+0+0 (secure 0 0 0) 0x7f0528010040 con 0x7f05341a98c0 2024-04-17T22:31:04.810 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.822 7f0539993700 1 -- 172.21.15.102:0/3904498901 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05340f0820 con 0x7f05341a98c0 2024-04-17T22:31:04.811 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.825 7f05237fe700 1 -- 172.21.15.102:0/3904498901 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+110455 (secure 0 0 0) 0x7f052800c5a0 con 0x7f05341a98c0 2024-04-17T22:31:04.918 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.932 7f0539993700 1 -- 172.21.15.102:0/3904498901 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f05341adf80 con 0x7f05341a98c0 2024-04-17T22:31:04.919 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.933 7f05237fe700 1 -- 172.21.15.102:0/3904498901 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) v1 ==== 74+0+8764 (secure 0 0 0) 0x7f0528000c30 con 0x7f05341a98c0 2024-04-17T22:31:04.919 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-17T22:31:04.919 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":17,"fsid":"cb63e789-091d-49b1-b120-55f9db526871","created":"2024-04-17 22:28:48.209635","modified":"2024-04-17 22:29:23.257387","last_up_change":"2024-04-17 22:28:54.236215","last_in_change":"2024-04-17 22:28:54.236215","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":4,"require_min_compat_client":"nautilus","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-04-17 22:28:57.478545","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-17 22:29:06.144708","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-17 22:29:06.748924","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3351ef85-ade4-4b90-a8f4-213f4e00ba01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6812","nonce":42955}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6813","nonce":42955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6815","nonce":42955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6814","nonce":42955}]},"public_addr":"172.21.15.102:6812/42955","cluster_addr":"172.21.15.102:6813/42955","heartbeat_back_addr":"172.21.15.102:6815/42955","heartbeat_front_addr":"172.21.15.102:6814/42955","state":["exists","up"]},{"osd":1,"uuid":"04f67eac-f36b-48f5-982d-4cfc776596d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6808","nonce":42958}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6809","nonce":42958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6811","nonce":42958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6810","nonce":42958}]},"public_addr":"172.21.15.102:6808/42958","cluster_addr":"172.21.15.102:6809/42958","heartbeat_back_addr":"172.21.15.102:6811/42958","heartbeat_front_addr":"172.21.15.102:6810/42958","state":["exists","up"]},{"osd":2,"uuid":"d524204b-70b2-40a8-8f73-f9e7d44318af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6804","nonce":42954}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6805","nonce":42954}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6807","nonce":42954}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6806","nonce":42954}]},"public_addr":"172.21.15.102:6804/42954","cluster_addr":"172.21.15.102:6805/42954","heartbeat_back_addr":"172.21.15.102:6807/42954","heartbeat_front_addr":"172.21.15.102:6806/42954","state":["exists","up"]},{"osd":3,"uuid":"4fc9d20c-1ad0-484f-aafa-2c9a7574245b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6800","nonce":42953}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6801","nonce":42953}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6803","nonce":42953}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6802","nonce":42953}]},"public_addr":"172.21.15.102:6800/42953","cluster_addr":"172.21.15.102:6801/42953","heartbeat_back_addr":"172.21.15.102:6803/42953","heartbeat_front_addr":"172.21.15.102:6802/42953","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2024-04-17T22:31:04.920 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.934 7f0539993700 1 -- 172.21.15.102:0/3904498901 >> v1:172.21.15.102:6816/42360 conn(0x7f05100145a0 legacy=0x7f0510016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:04.920 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.934 7f0539993700 1 -- 172.21.15.102:0/3904498901 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f05341a98c0 msgr2=0x7f05341a9d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:04.920 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.934 7f0539993700 1 --2- 172.21.15.102:0/3904498901 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f05341a98c0 0x7f05341a9d10 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 rx=0x7f05340cf690 tx=0x7f052800ed00).stop 2024-04-17T22:31:04.920 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.934 7f0539993700 1 -- 172.21.15.102:0/3904498901 shutdown_connections 2024-04-17T22:31:04.920 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.934 7f0539993700 1 --2- 172.21.15.102:0/3904498901 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f05340de620 0x7f05340dea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:04.920 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.934 7f0539993700 1 --2- 172.21.15.102:0/3904498901 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f05341a98c0 0x7f05341a9d10 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:04.921 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.934 7f0539993700 1 --2- 172.21.15.102:0/3904498901 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f05341aa340 0x7f05340ddff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:04.921 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.934 7f0539993700 1 -- 172.21.15.102:0/3904498901 >> 172.21.15.102:0/3904498901 conn(0x7f05340c4ee0 msgr2=0x7f05340d6760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:04.921 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.934 7f0539993700 1 -- 172.21.15.102:0/3904498901 shutdown_connections 2024-04-17T22:31:04.921 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:04.934 7f0539993700 1 -- 172.21.15.102:0/3904498901 wait complete. 2024-04-17T22:31:04.931 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2024-04-17 22:28:57.478545', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '12', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-17 22:29:06.144708', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-17 22:29:06.748924', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-17T22:31:04.931 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2024-04-17T22:31:05.060 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.071 7fd582780700 1 Processor -- start 2024-04-17T22:31:05.060 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.072 7fd582780700 1 -- start start 2024-04-17T22:31:05.060 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.072 7fd582780700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd57c0f5840 0x7fd57c10b010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:05.060 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.072 7fd582780700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd57c10b640 0x7fd57c19faf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:05.060 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.072 7fd582780700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fd57c10ba90 0x7fd57c1a0120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:05.060 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.072 7fd582780700 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd57c10bee0 con 0x7fd57c10ba90 2024-04-17T22:31:05.060 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.072 7fd582780700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd57c0fa370 con 0x7fd57c0f5840 2024-04-17T22:31:05.060 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.072 7fd582780700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd57c103b40 con 0x7fd57c10b640 2024-04-17T22:31:05.060 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.072 7fd573fff700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd57c10b640 0x7fd57c19faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:05.061 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.072 7fd57b3b0700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd57c0f5840 0x7fd57c10b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:05.061 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.072 7fd573fff700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd57c10b640 0x7fd57c19faf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:54740/0 (socket says 172.21.15.102:54740) 2024-04-17T22:31:05.061 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.072 7fd57b3b0700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd57c0f5840 0x7fd57c10b010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:51886/0 (socket says 172.21.15.102:51886) 2024-04-17T22:31:05.061 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.072 7fd573fff700 1 -- 172.21.15.102:0/3963753332 learned_addr learned my addr 172.21.15.102:0/3963753332 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:31:05.061 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.072 7fd57bbb1700 1 --2- 172.21.15.102:0/3963753332 >> v2:172.21.15.102:3300/0 conn(0x7fd57c10ba90 0x7fd57c1a0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:05.061 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.072 7fd57b3b0700 1 -- 172.21.15.102:0/3963753332 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd57c10b640 msgr2=0x7fd57c19faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:31:05.061 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.072 7fd57b3b0700 1 --2- 172.21.15.102:0/3963753332 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd57c10b640 0x7fd57c19faf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:05.061 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.072 7fd57b3b0700 1 -- 172.21.15.102:0/3963753332 >> v2:172.21.15.102:3300/0 conn(0x7fd57c10ba90 msgr2=0x7fd57c1a0120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:31:05.061 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.072 7fd57b3b0700 1 --2- 172.21.15.102:0/3963753332 >> v2:172.21.15.102:3300/0 conn(0x7fd57c10ba90 0x7fd57c1a0120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:05.061 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.072 7fd57b3b0700 1 -- 172.21.15.102:0/3963753332 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd57c1a0750 con 0x7fd57c0f5840 2024-04-17T22:31:05.061 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.072 7fd57bbb1700 1 --2- 172.21.15.102:0/3963753332 >> v2:172.21.15.102:3300/0 conn(0x7fd57c10ba90 0x7fd57c1a0120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:05.061 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.073 7fd57b3b0700 1 --2- 172.21.15.102:0/3963753332 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd57c0f5840 0x7fd57c10b010 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 rx=0x7fd56c002f70 tx=0x7fd56c00b9f0).ready entity=mon.1 client_cookie=b6e2966bd07b8801 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:05.061 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.073 7fd57abaf700 1 -- 172.21.15.102:0/3963753332 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd57c1a0750 con 0x7fd57c0f5840 2024-04-17T22:31:05.061 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.073 7fd57abaf700 1 -- 172.21.15.102:0/3963753332 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd56c00d570 con 0x7fd57c0f5840 2024-04-17T22:31:05.062 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.073 7fd57abaf700 1 -- 172.21.15.102:0/3963753332 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd56c00d940 con 0x7fd57c0f5840 2024-04-17T22:31:05.062 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.073 7fd582780700 1 -- 172.21.15.102:0/3963753332 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd57c0f5840 msgr2=0x7fd57c10b010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:05.062 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.073 7fd582780700 1 --2- 172.21.15.102:0/3963753332 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd57c0f5840 0x7fd57c10b010 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 rx=0x7fd56c002f70 tx=0x7fd56c00b9f0).stop 2024-04-17T22:31:05.062 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.073 7fd582780700 1 -- 172.21.15.102:0/3963753332 shutdown_connections 2024-04-17T22:31:05.062 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.073 7fd582780700 1 --2- 172.21.15.102:0/3963753332 >> v2:172.21.15.102:3300/0 conn(0x7fd57c10ba90 0x7fd57c1a0120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:05.062 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.073 7fd582780700 1 --2- 172.21.15.102:0/3963753332 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd57c0f5840 0x7fd57c10b010 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:05.062 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.073 7fd582780700 1 --2- 172.21.15.102:0/3963753332 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd57c10b640 0x7fd57c19faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:05.062 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.073 7fd582780700 1 -- 172.21.15.102:0/3963753332 >> 172.21.15.102:0/3963753332 conn(0x7fd57c1138a0 msgr2=0x7fd57c0f4f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:05.062 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.073 7fd582780700 1 -- 172.21.15.102:0/3963753332 shutdown_connections 2024-04-17T22:31:05.062 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.073 7fd582780700 1 -- 172.21.15.102:0/3963753332 wait complete. 2024-04-17T22:31:05.084 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.096 7fd582780700 1 Processor -- start 2024-04-17T22:31:05.084 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.096 7fd582780700 1 -- start start 2024-04-17T22:31:05.084 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.096 7fd582780700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd57c0d3490 0x7fd57c1ad4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:05.085 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.096 7fd582780700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd57c1adb00 0x7fd57c1b3f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:05.085 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.096 7fd582780700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd57c1adf50 0x7fd57c1b4540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:05.085 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.096 7fd582780700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd57c0d9610 con 0x7fd57c0d3490 2024-04-17T22:31:05.085 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.096 7fd582780700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd57c0d98d0 con 0x7fd57c1adf50 2024-04-17T22:31:05.085 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.096 7fd582780700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd57c1b4b70 con 0x7fd57c1adb00 2024-04-17T22:31:05.085 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.096 7fd573fff700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd57c1adb00 0x7fd57c1b3f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:05.085 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.096 7fd57bbb1700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd57c1adf50 0x7fd57c1b4540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:05.085 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.096 7fd57b3b0700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd57c0d3490 0x7fd57c1ad4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:05.085 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.096 7fd573fff700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd57c1adb00 0x7fd57c1b3f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:54744/0 (socket says 172.21.15.102:54744) 2024-04-17T22:31:05.086 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.096 7fd57b3b0700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd57c0d3490 0x7fd57c1ad4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:53968/0 (socket says 172.21.15.102:53968) 2024-04-17T22:31:05.086 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.096 7fd573fff700 1 -- 172.21.15.102:0/463508508 learned_addr learned my addr 172.21.15.102:0/463508508 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:31:05.086 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.096 7fd57bbb1700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd57c1adf50 0x7fd57c1b4540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:51888/0 (socket says 172.21.15.102:51888) 2024-04-17T22:31:05.086 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.097 7fd57bbb1700 1 -- 172.21.15.102:0/463508508 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd57c1adb00 msgr2=0x7fd57c1b3f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:05.086 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.097 7fd57bbb1700 1 --2- 172.21.15.102:0/463508508 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd57c1adb00 0x7fd57c1b3f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:05.086 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.097 7fd57bbb1700 1 -- 172.21.15.102:0/463508508 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd57c0d3490 msgr2=0x7fd57c1ad4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:05.086 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.097 7fd57bbb1700 1 --2- 172.21.15.102:0/463508508 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd57c0d3490 0x7fd57c1ad4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:05.086 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.097 7fd57bbb1700 1 -- 172.21.15.102:0/463508508 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd57c1b4d50 con 0x7fd57c1adf50 2024-04-17T22:31:05.086 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.097 7fd57b3b0700 1 --2- 172.21.15.102:0/463508508 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd57c0d3490 0x7fd57c1ad4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:05.086 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.097 7fd573fff700 1 --2- 172.21.15.102:0/463508508 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd57c1adb00 0x7fd57c1b3f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:05.086 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.097 7fd57bbb1700 1 --2- 172.21.15.102:0/463508508 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd57c1adf50 0x7fd57c1b4540 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0x7fd57c0d6280 tx=0x7fd57400ed50).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:05.087 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.097 7fd5793ac700 1 -- 172.21.15.102:0/463508508 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd57c1b4d50 con 0x7fd57c1adf50 2024-04-17T22:31:05.087 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.097 7fd5793ac700 1 -- 172.21.15.102:0/463508508 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd57400c7b0 con 0x7fd57c1adf50 2024-04-17T22:31:05.087 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.097 7fd582780700 1 -- 172.21.15.102:0/463508508 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd57c1bb680 con 0x7fd57c1adf50 2024-04-17T22:31:05.087 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.097 7fd5793ac700 1 -- 172.21.15.102:0/463508508 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd57400f040 con 0x7fd57c1adf50 2024-04-17T22:31:05.087 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.097 7fd582780700 1 -- 172.21.15.102:0/463508508 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd57c1bba60 con 0x7fd57c1adf50 2024-04-17T22:31:05.087 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.097 7fd5793ac700 1 -- 172.21.15.102:0/463508508 <== mon.1 v2:172.21.15.102:3301/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7fd574011070 con 0x7fd57c1adf50 2024-04-17T22:31:05.087 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.097 7fd5793ac700 1 -- 172.21.15.102:0/463508508 <== mon.1 v2:172.21.15.102:3301/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3194+0+0 (secure 0 0 0) 0x7fd574010040 con 0x7fd57c1adf50 2024-04-17T22:31:05.087 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.098 7fd582780700 1 -- 172.21.15.102:0/463508508 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd57c0f0800 con 0x7fd57c1adf50 2024-04-17T22:31:05.087 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.101 7fd5793ac700 1 -- 172.21.15.102:0/463508508 <== mon.1 v2:172.21.15.102:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+110455 (secure 0 0 0) 0x7fd57400c5a0 con 0x7fd57c1adf50 2024-04-17T22:31:05.191 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.205 7fd582780700 1 -- 172.21.15.102:0/463508508 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fd57c1ae630 con 0x7fd57c1adf50 2024-04-17T22:31:05.192 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.206 7fd5793ac700 1 -- 172.21.15.102:0/463508508 <== mon.1 v2:172.21.15.102:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v17) v1 ==== 92+0+10 (secure 0 0 0) 0x7fd57c114420 con 0x7fd57c1adf50 2024-04-17T22:31:05.192 INFO:teuthology.orchestra.run.smithi102.stdout:pg_num: 8 2024-04-17T22:31:05.193 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.207 7fd582780700 1 -- 172.21.15.102:0/463508508 >> v1:172.21.15.102:6816/42360 conn(0x7fd5600145a0 legacy=0x7fd560016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:05.194 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.207 7fd582780700 1 -- 172.21.15.102:0/463508508 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd57c1adf50 msgr2=0x7fd57c1b4540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:05.194 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.207 7fd582780700 1 --2- 172.21.15.102:0/463508508 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd57c1adf50 0x7fd57c1b4540 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0x7fd57c0d6280 tx=0x7fd57400ed50).stop 2024-04-17T22:31:05.194 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.207 7fd582780700 1 -- 172.21.15.102:0/463508508 shutdown_connections 2024-04-17T22:31:05.194 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.207 7fd582780700 1 --2- 172.21.15.102:0/463508508 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd57c1adf50 0x7fd57c1b4540 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:05.194 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.207 7fd582780700 1 --2- 172.21.15.102:0/463508508 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd57c0d3490 0x7fd57c1ad4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:05.194 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.207 7fd582780700 1 --2- 172.21.15.102:0/463508508 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd57c1adb00 0x7fd57c1b3f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:05.194 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.207 7fd582780700 1 -- 172.21.15.102:0/463508508 >> 172.21.15.102:0/463508508 conn(0x7fd57c0e0f40 msgr2=0x7fd57c0dfe90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:05.194 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.207 7fd582780700 1 -- 172.21.15.102:0/463508508 shutdown_connections 2024-04-17T22:31:05.194 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.207 7fd582780700 1 -- 172.21.15.102:0/463508508 wait complete. 2024-04-17T22:31:05.204 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-17T22:31:05.334 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.345 7f8f3f0ec700 1 Processor -- start 2024-04-17T22:31:05.334 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.345 7f8f3f0ec700 1 -- start start 2024-04-17T22:31:05.334 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.346 7f8f3f0ec700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f8f381055c0 0x7f8f3810b130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:05.334 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.346 7f8f3f0ec700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f8f380f59b0 0x7f8f3810b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:05.334 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.346 7f8f3f0ec700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f8f3819fc80 0x7f8f381a00d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:05.334 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.346 7f8f3f0ec700 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f3810bd90 con 0x7f8f380f59b0 2024-04-17T22:31:05.334 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.346 7f8f3f0ec700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f3810bf40 con 0x7f8f3819fc80 2024-04-17T22:31:05.335 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.346 7f8f3f0ec700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8f381a0700 con 0x7f8f381055c0 2024-04-17T22:31:05.335 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.346 7f8f36ffd700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f8f380f59b0 0x7f8f3810b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:05.335 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.346 7f8f37fff700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f8f3819fc80 0x7f8f381a00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:05.335 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.346 7f8f377fe700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f8f381055c0 0x7f8f3810b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:05.335 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.346 7f8f36ffd700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f8f380f59b0 0x7f8f3810b760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:53976/0 (socket says 172.21.15.102:53976) 2024-04-17T22:31:05.335 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.346 7f8f36ffd700 1 -- 172.21.15.102:0/239991258 learned_addr learned my addr 172.21.15.102:0/239991258 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:31:05.335 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.346 7f8f377fe700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f8f381055c0 0x7f8f3810b130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:54750/0 (socket says 172.21.15.102:54750) 2024-04-17T22:31:05.335 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.346 7f8f36ffd700 1 -- 172.21.15.102:0/239991258 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f8f381055c0 msgr2=0x7f8f3810b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:31:05.335 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.346 7f8f36ffd700 1 --2- 172.21.15.102:0/239991258 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f8f381055c0 0x7f8f3810b130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:05.335 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.346 7f8f36ffd700 1 -- 172.21.15.102:0/239991258 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f8f3819fc80 msgr2=0x7f8f381a00d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:31:05.335 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.346 7f8f36ffd700 1 --2- 172.21.15.102:0/239991258 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f8f3819fc80 0x7f8f381a00d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:05.335 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.346 7f8f36ffd700 1 -- 172.21.15.102:0/239991258 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f381a0930 con 0x7f8f380f59b0 2024-04-17T22:31:05.335 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.346 7f8f377fe700 1 --2- 172.21.15.102:0/239991258 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f8f381055c0 0x7f8f3810b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:05.335 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.346 7f8f37fff700 1 --2- 172.21.15.102:0/239991258 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f8f3819fc80 0x7f8f381a00d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:05.336 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.346 7f8f36ffd700 1 --2- 172.21.15.102:0/239991258 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8f380f59b0 0x7f8f3810b760 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 rx=0x7f8f380f4660 tx=0x7f8f20009a80).ready entity=mon.0 client_cookie=d5994413ac27555f server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:05.336 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.347 7f8f367fc700 1 -- 172.21.15.102:0/239991258 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8f381a0930 con 0x7f8f380f59b0 2024-04-17T22:31:05.336 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.347 7f8f367fc700 1 -- 172.21.15.102:0/239991258 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f2000f510 con 0x7f8f380f59b0 2024-04-17T22:31:05.336 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.347 7f8f367fc700 1 -- 172.21.15.102:0/239991258 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8f2000f6f0 con 0x7f8f380f59b0 2024-04-17T22:31:05.336 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.347 7f8f3f0ec700 1 -- 172.21.15.102:0/239991258 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8f380f59b0 msgr2=0x7f8f3810b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:05.336 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.347 7f8f3f0ec700 1 --2- 172.21.15.102:0/239991258 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8f380f59b0 0x7f8f3810b760 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 rx=0x7f8f380f4660 tx=0x7f8f20009a80).stop 2024-04-17T22:31:05.336 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.347 7f8f3f0ec700 1 -- 172.21.15.102:0/239991258 shutdown_connections 2024-04-17T22:31:05.336 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.347 7f8f3f0ec700 1 --2- 172.21.15.102:0/239991258 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f8f3819fc80 0x7f8f381a00d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:05.336 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.347 7f8f3f0ec700 1 --2- 172.21.15.102:0/239991258 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f8f381055c0 0x7f8f3810b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:05.336 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.347 7f8f3f0ec700 1 --2- 172.21.15.102:0/239991258 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8f380f59b0 0x7f8f3810b760 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:05.336 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.347 7f8f3f0ec700 1 -- 172.21.15.102:0/239991258 >> 172.21.15.102:0/239991258 conn(0x7f8f38113a90 msgr2=0x7f8f380f5140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:05.336 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.347 7f8f3f0ec700 1 -- 172.21.15.102:0/239991258 shutdown_connections 2024-04-17T22:31:05.336 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.347 7f8f3f0ec700 1 -- 172.21.15.102:0/239991258 wait complete. 2024-04-17T22:31:05.356 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.370 7f8f3f0ec700 1 Processor -- start 2024-04-17T22:31:05.356 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.370 7f8f3f0ec700 1 -- start start 2024-04-17T22:31:05.359 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.370 7f8f3f0ec700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8f380d5040 0x7f8f380d5490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:05.359 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.370 7f8f3f0ec700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f8f380d5ac0 0x7f8f381afd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:05.359 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.370 7f8f3f0ec700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f8f381b0380 0x7f8f381b07d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:05.359 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.371 7f8f3f0ec700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f381a0f80 con 0x7f8f380d5040 2024-04-17T22:31:05.359 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.371 7f8f3f0ec700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f380c3520 con 0x7f8f381b0380 2024-04-17T22:31:05.359 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.371 7f8f3f0ec700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8f381a47f0 con 0x7f8f380d5ac0 2024-04-17T22:31:05.359 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.371 7f8f377fe700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8f380d5040 0x7f8f380d5490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:05.359 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.371 7f8f37fff700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f8f381b0380 0x7f8f381b07d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:05.359 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.371 7f8f37fff700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f8f381b0380 0x7f8f381b07d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:51910/0 (socket says 172.21.15.102:51910) 2024-04-17T22:31:05.359 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.371 7f8f377fe700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8f380d5040 0x7f8f380d5490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:53980/0 (socket says 172.21.15.102:53980) 2024-04-17T22:31:05.359 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.371 7f8f37fff700 1 -- 172.21.15.102:0/149198630 learned_addr learned my addr 172.21.15.102:0/149198630 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:31:05.359 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.371 7f8f36ffd700 1 --2- 172.21.15.102:0/149198630 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f8f380d5ac0 0x7f8f381afd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:05.360 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.371 7f8f377fe700 1 -- 172.21.15.102:0/149198630 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f8f380d5ac0 msgr2=0x7f8f381afd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:05.360 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.371 7f8f377fe700 1 --2- 172.21.15.102:0/149198630 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f8f380d5ac0 0x7f8f381afd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:05.360 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.371 7f8f377fe700 1 -- 172.21.15.102:0/149198630 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f8f381b0380 msgr2=0x7f8f381b07d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:05.360 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.371 7f8f377fe700 1 --2- 172.21.15.102:0/149198630 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f8f381b0380 0x7f8f381b07d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:05.360 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.371 7f8f377fe700 1 -- 172.21.15.102:0/149198630 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f380d95d0 con 0x7f8f380d5040 2024-04-17T22:31:05.360 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.371 7f8f37fff700 1 --2- 172.21.15.102:0/149198630 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f8f381b0380 0x7f8f381b07d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:05.360 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.371 7f8f36ffd700 1 --2- 172.21.15.102:0/149198630 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f8f380d5ac0 0x7f8f381afd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:05.360 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.371 7f8f377fe700 1 --2- 172.21.15.102:0/149198630 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8f380d5040 0x7f8f380d5490 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 rx=0x7f8f380d6580 tx=0x7f8f2800eeb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:05.360 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.371 7f8f34ff9700 1 -- 172.21.15.102:0/149198630 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8f380d95d0 con 0x7f8f380d5040 2024-04-17T22:31:05.360 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.371 7f8f34ff9700 1 -- 172.21.15.102:0/149198630 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f2800c950 con 0x7f8f380d5040 2024-04-17T22:31:05.360 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.371 7f8f3f0ec700 1 -- 172.21.15.102:0/149198630 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f381b74c0 con 0x7f8f380d5040 2024-04-17T22:31:05.360 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.371 7f8f34ff9700 1 -- 172.21.15.102:0/149198630 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8f2800f040 con 0x7f8f380d5040 2024-04-17T22:31:05.360 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.371 7f8f3f0ec700 1 -- 172.21.15.102:0/149198630 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f381b76c0 con 0x7f8f380d5040 2024-04-17T22:31:05.360 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.372 7f8f34ff9700 1 -- 172.21.15.102:0/149198630 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7f8f28022060 con 0x7f8f380d5040 2024-04-17T22:31:05.360 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.372 7f8f34ff9700 1 -- 172.21.15.102:0/149198630 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3194+0+0 (secure 0 0 0) 0x7f8f28010040 con 0x7f8f380d5040 2024-04-17T22:31:05.361 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.372 7f8f3f0ec700 1 -- 172.21.15.102:0/149198630 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f380f09f0 con 0x7f8f380d5040 2024-04-17T22:31:05.361 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.375 7f8f34ff9700 1 -- 172.21.15.102:0/149198630 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+110455 (secure 0 0 0) 0x7f8f2800c740 con 0x7f8f380d5040 2024-04-17T22:31:05.468 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.482 7f8f3f0ec700 1 -- 172.21.15.102:0/149198630 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f8f380ad030 con 0x7f8f380d5040 2024-04-17T22:31:05.469 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.482 7f8f34ff9700 1 -- 172.21.15.102:0/149198630 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v17) v1 ==== 104+0+11 (secure 0 0 0) 0x7f8f28000fb0 con 0x7f8f380d5040 2024-04-17T22:31:05.469 INFO:teuthology.orchestra.run.smithi102.stdout:pg_num: 64 2024-04-17T22:31:05.470 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.483 7f8f3f0ec700 1 -- 172.21.15.102:0/149198630 >> v1:172.21.15.102:6816/42360 conn(0x7f8f1c0145a0 legacy=0x7f8f1c016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:05.470 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.483 7f8f3f0ec700 1 -- 172.21.15.102:0/149198630 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8f380d5040 msgr2=0x7f8f380d5490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:05.470 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.483 7f8f3f0ec700 1 --2- 172.21.15.102:0/149198630 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8f380d5040 0x7f8f380d5490 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 rx=0x7f8f380d6580 tx=0x7f8f2800eeb0).stop 2024-04-17T22:31:05.470 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.483 7f8f3f0ec700 1 -- 172.21.15.102:0/149198630 shutdown_connections 2024-04-17T22:31:05.470 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.483 7f8f3f0ec700 1 --2- 172.21.15.102:0/149198630 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f8f381b0380 0x7f8f381b07d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:05.470 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.483 7f8f3f0ec700 1 --2- 172.21.15.102:0/149198630 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8f380d5040 0x7f8f380d5490 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:05.470 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.483 7f8f3f0ec700 1 --2- 172.21.15.102:0/149198630 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f8f380d5ac0 0x7f8f381afd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:05.471 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.483 7f8f3f0ec700 1 -- 172.21.15.102:0/149198630 >> 172.21.15.102:0/149198630 conn(0x7f8f380e0100 msgr2=0x7f8f380c1ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:05.471 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.484 7f8f3f0ec700 1 -- 172.21.15.102:0/149198630 shutdown_connections 2024-04-17T22:31:05.471 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.484 7f8f3f0ec700 1 -- 172.21.15.102:0/149198630 wait complete. 2024-04-17T22:31:05.480 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-17T22:31:05.610 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.621 7f4d12fdd700 1 Processor -- start 2024-04-17T22:31:05.610 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.622 7f4d12fdd700 1 -- start start 2024-04-17T22:31:05.610 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.622 7f4d12fdd700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f4d0c0f5be0 0x7f4d0c10e840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:05.610 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.622 7f4d12fdd700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4d0c10ee70 0x7f4d0c0f1920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:05.610 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.622 7f4d12fdd700 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d0c103670 con 0x7f4d0c0f1f50 2024-04-17T22:31:05.610 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.622 7f4d12fdd700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d0c10f2c0 con 0x7f4d0c10ee70 2024-04-17T22:31:05.610 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.622 7f4d12fdd700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4d0c110e50 con 0x7f4d0c0f5be0 2024-04-17T22:31:05.611 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.622 7f4d0b7fe700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f4d0c0f5be0 0x7f4d0c10e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:05.611 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.622 7f4d0affd700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4d0c10ee70 0x7f4d0c0f1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:05.611 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.622 7f4d0b7fe700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f4d0c0f5be0 0x7f4d0c10e840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:54766/0 (socket says 172.21.15.102:54766) 2024-04-17T22:31:05.611 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.622 7f4d0b7fe700 1 -- 172.21.15.102:0/190983465 learned_addr learned my addr 172.21.15.102:0/190983465 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:31:05.611 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.622 7f4d0bfff700 1 --1- 172.21.15.102:0/190983465 >> v1:172.21.15.102:6789/0 conn(0x7f4d0c0f1f50 0x7f4d0c1aa3d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:38698/0 (socket says 172.21.15.102:38698) 2024-04-17T22:31:05.611 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.622 7f4d0affd700 1 -- 172.21.15.102:0/190983465 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f4d0c0f5be0 msgr2=0x7f4d0c10e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:31:05.611 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.622 7f4d0affd700 1 --2- 172.21.15.102:0/190983465 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f4d0c0f5be0 0x7f4d0c10e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:05.611 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.622 7f4d0affd700 1 -- 172.21.15.102:0/190983465 >> v1:172.21.15.102:6789/0 conn(0x7f4d0c0f1f50 legacy=0x7f4d0c1aa3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:05.611 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.622 7f4d0a7fc700 1 -- 172.21.15.102:0/190983465 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2572414234 0 0) 0x7f4d04001490 con 0x7f4d0c0f1f50 2024-04-17T22:31:05.611 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.622 7f4d0affd700 1 -- 172.21.15.102:0/190983465 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d0c0f23a0 con 0x7f4d0c10ee70 2024-04-17T22:31:05.611 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.622 7f4d0b7fe700 1 --2- 172.21.15.102:0/190983465 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f4d0c0f5be0 0x7f4d0c10e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:05.611 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.623 7f4d0affd700 1 --2- 172.21.15.102:0/190983465 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4d0c10ee70 0x7f4d0c0f1920 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 rx=0x7f4cf4002f70 tx=0x7f4cf400ba20).ready entity=mon.1 client_cookie=4ff5168b87633f70 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:05.611 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.623 7f4d0a7fc700 1 -- 172.21.15.102:0/190983465 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f4d0c0f23a0 con 0x7f4d0c10ee70 2024-04-17T22:31:05.612 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.623 7f4d0a7fc700 1 -- 172.21.15.102:0/190983465 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cf400d570 con 0x7f4d0c10ee70 2024-04-17T22:31:05.612 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.623 7f4d0a7fc700 1 -- 172.21.15.102:0/190983465 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f4cf400d940 con 0x7f4d0c10ee70 2024-04-17T22:31:05.612 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.623 7f4d12fdd700 1 -- 172.21.15.102:0/190983465 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4d0c10ee70 msgr2=0x7f4d0c0f1920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:05.612 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.623 7f4d12fdd700 1 --2- 172.21.15.102:0/190983465 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4d0c10ee70 0x7f4d0c0f1920 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 rx=0x7f4cf4002f70 tx=0x7f4cf400ba20).stop 2024-04-17T22:31:05.612 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.623 7f4d12fdd700 1 -- 172.21.15.102:0/190983465 shutdown_connections 2024-04-17T22:31:05.612 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.623 7f4d12fdd700 1 --2- 172.21.15.102:0/190983465 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f4d0c0f5be0 0x7f4d0c10e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:05.612 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.623 7f4d12fdd700 1 --2- 172.21.15.102:0/190983465 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4d0c10ee70 0x7f4d0c0f1920 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:05.612 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.623 7f4d12fdd700 1 -- 172.21.15.102:0/190983465 >> 172.21.15.102:0/190983465 conn(0x7f4d0c113a90 msgr2=0x7f4d0c0f5140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:05.612 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.623 7f4d12fdd700 1 -- 172.21.15.102:0/190983465 shutdown_connections 2024-04-17T22:31:05.612 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.623 7f4d12fdd700 1 -- 172.21.15.102:0/190983465 wait complete. 2024-04-17T22:31:05.632 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.646 7f4d12fdd700 1 Processor -- start 2024-04-17T22:31:05.632 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.646 7f4d12fdd700 1 -- start start 2024-04-17T22:31:05.635 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.647 7f4d12fdd700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f4d0c1adb30 0x7f4d0c1adf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:05.635 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.647 7f4d12fdd700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4d0c1ae5b0 0x7f4d0c1b4440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:05.635 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.647 7f4d12fdd700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4d0c1b4a70 0x7f4d0c1b4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:05.635 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.647 7f4d12fdd700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d0c0c4d40 con 0x7f4d0c1ae5b0 2024-04-17T22:31:05.635 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.647 7f4d12fdd700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d0c0cbd90 con 0x7f4d0c1b4a70 2024-04-17T22:31:05.635 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.647 7f4d12fdd700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4d0c0cc310 con 0x7f4d0c1adb30 2024-04-17T22:31:05.635 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.647 7f4d0b7fe700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f4d0c1adb30 0x7f4d0c1adf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:05.635 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.647 7f4d0b7fe700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f4d0c1adb30 0x7f4d0c1adf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:54776/0 (socket says 172.21.15.102:54776) 2024-04-17T22:31:05.635 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.647 7f4d0b7fe700 1 -- 172.21.15.102:0/3923929892 learned_addr learned my addr 172.21.15.102:0/3923929892 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:31:05.635 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.647 7f4d0affd700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4d0c1ae5b0 0x7f4d0c1b4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:05.635 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.647 7f4d0bfff700 1 --2- 172.21.15.102:0/3923929892 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4d0c1b4a70 0x7f4d0c1b4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:05.636 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.647 7f4d0b7fe700 1 -- 172.21.15.102:0/3923929892 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4d0c1b4a70 msgr2=0x7f4d0c1b4ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:05.636 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.647 7f4d0b7fe700 1 --2- 172.21.15.102:0/3923929892 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4d0c1b4a70 0x7f4d0c1b4ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:05.636 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.647 7f4d0b7fe700 1 -- 172.21.15.102:0/3923929892 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4d0c1ae5b0 msgr2=0x7f4d0c1b4440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:05.636 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.647 7f4d0b7fe700 1 --2- 172.21.15.102:0/3923929892 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4d0c1ae5b0 0x7f4d0c1b4440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:05.636 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.647 7f4d0b7fe700 1 -- 172.21.15.102:0/3923929892 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d0c0cc9d0 con 0x7f4d0c1adb30 2024-04-17T22:31:05.636 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.647 7f4d0affd700 1 --2- 172.21.15.102:0/3923929892 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4d0c1ae5b0 0x7f4d0c1b4440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:05.636 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.647 7f4d0b7fe700 1 --2- 172.21.15.102:0/3923929892 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f4d0c1adb30 0x7f4d0c1adf80 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 rx=0x7f4d0c1a9540 tx=0x7f4cfc00ecb0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:05.636 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.647 7f4d08ff9700 1 -- 172.21.15.102:0/3923929892 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f4d0c0cc9d0 con 0x7f4d0c1adb30 2024-04-17T22:31:05.636 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.647 7f4d08ff9700 1 -- 172.21.15.102:0/3923929892 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cfc00c620 con 0x7f4d0c1adb30 2024-04-17T22:31:05.636 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.648 7f4d12fdd700 1 -- 172.21.15.102:0/3923929892 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d0c1bbc40 con 0x7f4d0c1adb30 2024-04-17T22:31:05.636 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.648 7f4d08ff9700 1 -- 172.21.15.102:0/3923929892 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f4cfc00f040 con 0x7f4d0c1adb30 2024-04-17T22:31:05.636 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.648 7f4d12fdd700 1 -- 172.21.15.102:0/3923929892 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d0c1bbe40 con 0x7f4d0c1adb30 2024-04-17T22:31:05.636 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.648 7f4d08ff9700 1 -- 172.21.15.102:0/3923929892 <== mon.2 v2:172.21.15.102:3302/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7f4cfc022060 con 0x7f4d0c1adb30 2024-04-17T22:31:05.636 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.648 7f4d08ff9700 1 -- 172.21.15.102:0/3923929892 <== mon.2 v2:172.21.15.102:3302/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3194+0+0 (secure 0 0 0) 0x7f4cfc010040 con 0x7f4d0c1adb30 2024-04-17T22:31:05.636 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.649 7f4d12fdd700 1 -- 172.21.15.102:0/3923929892 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d0c0ad030 con 0x7f4d0c1adb30 2024-04-17T22:31:05.638 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.652 7f4d08ff9700 1 -- 172.21.15.102:0/3923929892 <== mon.2 v2:172.21.15.102:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+110455 (secure 0 0 0) 0x7f4cfc00edf0 con 0x7f4d0c1adb30 2024-04-17T22:31:05.744 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.758 7f4d12fdd700 1 -- 172.21.15.102:0/3923929892 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f4d0c0ac930 con 0x7f4d0c1adb30 2024-04-17T22:31:05.744 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.758 7f4d08ff9700 1 -- 172.21.15.102:0/3923929892 <== mon.2 v2:172.21.15.102:3302/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v17) v1 ==== 100+0+11 (secure 0 0 0) 0x7f4cfc000c30 con 0x7f4d0c1adb30 2024-04-17T22:31:05.744 INFO:teuthology.orchestra.run.smithi102.stdout:pg_num: 64 2024-04-17T22:31:05.746 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.759 7f4d12fdd700 1 -- 172.21.15.102:0/3923929892 >> v1:172.21.15.102:6816/42360 conn(0x7f4ce40145a0 legacy=0x7f4ce4016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:05.746 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.759 7f4d12fdd700 1 -- 172.21.15.102:0/3923929892 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f4d0c1adb30 msgr2=0x7f4d0c1adf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:05.746 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.759 7f4d12fdd700 1 --2- 172.21.15.102:0/3923929892 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f4d0c1adb30 0x7f4d0c1adf80 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 rx=0x7f4d0c1a9540 tx=0x7f4cfc00ecb0).stop 2024-04-17T22:31:05.746 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.759 7f4d12fdd700 1 -- 172.21.15.102:0/3923929892 shutdown_connections 2024-04-17T22:31:05.746 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.759 7f4d12fdd700 1 --2- 172.21.15.102:0/3923929892 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4d0c1b4a70 0x7f4d0c1b4ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:05.746 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.759 7f4d12fdd700 1 --2- 172.21.15.102:0/3923929892 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f4d0c1adb30 0x7f4d0c1adf80 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:05.746 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.759 7f4d12fdd700 1 --2- 172.21.15.102:0/3923929892 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4d0c1ae5b0 0x7f4d0c1b4440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:05.746 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.759 7f4d12fdd700 1 -- 172.21.15.102:0/3923929892 >> 172.21.15.102:0/3923929892 conn(0x7f4d0c0c8270 msgr2=0x7f4d0c0cf400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:05.746 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.760 7f4d12fdd700 1 -- 172.21.15.102:0/3923929892 shutdown_connections 2024-04-17T22:31:05.746 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.760 7f4d12fdd700 1 -- 172.21.15.102:0/3923929892 wait complete. 2024-04-17T22:31:05.756 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T22:31:05.885 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.896 7f0ecd378700 1 Processor -- start 2024-04-17T22:31:05.885 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.897 7f0ecd378700 1 -- start start 2024-04-17T22:31:05.885 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.897 7f0ecd378700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f0ec810e260 0x7f0ec810e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:05.885 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.897 7f0ecd378700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f0ec80f1f50 0x7f0ec81ab500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:05.885 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.897 7f0ecd378700 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ec80f23a0 con 0x7f0ec810ece0 2024-04-17T22:31:05.885 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.897 7f0ecd378700 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ec80f9f30 con 0x7f0ec810e260 2024-04-17T22:31:05.885 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.897 7f0ecd378700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ec80f9050 con 0x7f0ec80f1f50 2024-04-17T22:31:05.885 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.897 7f0ec685d700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f0ec80f1f50 0x7f0ec81ab500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:05.885 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.897 7f0ec605c700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f0ec810e260 0x7f0ec810e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:05.886 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.897 7f0ec685d700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f0ec80f1f50 0x7f0ec81ab500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:51922/0 (socket says 172.21.15.102:51922) 2024-04-17T22:31:05.886 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.897 7f0ec685d700 1 -- 172.21.15.102:0/2972236360 learned_addr learned my addr 172.21.15.102:0/2972236360 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:31:05.886 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.897 7f0ec605c700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f0ec810e260 0x7f0ec810e6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:53998/0 (socket says 172.21.15.102:53998) 2024-04-17T22:31:05.886 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.897 7f0ebffff700 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f0ec810ece0 0x7f0ec80f1750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:38700/0 (socket says 172.21.15.102:38700) 2024-04-17T22:31:05.886 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.897 7f0ec605c700 1 -- 172.21.15.102:0/2972236360 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f0ec80f1f50 msgr2=0x7f0ec81ab500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:31:05.886 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.897 7f0ec605c700 1 --2- 172.21.15.102:0/2972236360 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f0ec80f1f50 0x7f0ec81ab500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:05.886 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.897 7f0ec605c700 1 -- 172.21.15.102:0/2972236360 >> v1:172.21.15.102:6789/0 conn(0x7f0ec810ece0 legacy=0x7f0ec80f1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:05.886 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.897 7f0ec605c700 1 -- 172.21.15.102:0/2972236360 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ec80f25c0 con 0x7f0ec810e260 2024-04-17T22:31:05.886 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.897 7f0ec585b700 1 -- 172.21.15.102:0/2972236360 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1531559510 0 0) 0x7f0eb0001110 con 0x7f0ec810ece0 2024-04-17T22:31:05.886 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.897 7f0ec685d700 1 --2- 172.21.15.102:0/2972236360 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f0ec80f1f50 0x7f0ec81ab500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:05.887 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.897 7f0ec605c700 1 --2- 172.21.15.102:0/2972236360 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0ec810e260 0x7f0ec810e6b0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 rx=0x7f0ec80f46c0 tx=0x7f0eb800ba20).ready entity=mon.0 client_cookie=42cf1c3ecb5370b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:05.887 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.898 7f0ec585b700 1 -- 172.21.15.102:0/2972236360 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f0ec80f25c0 con 0x7f0ec810e260 2024-04-17T22:31:05.887 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.898 7f0ec585b700 1 -- 172.21.15.102:0/2972236360 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0eb800d5c0 con 0x7f0ec810e260 2024-04-17T22:31:05.887 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.898 7f0ec585b700 1 -- 172.21.15.102:0/2972236360 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f0eb800d990 con 0x7f0ec810e260 2024-04-17T22:31:05.887 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.898 7f0ecd378700 1 -- 172.21.15.102:0/2972236360 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0ec810e260 msgr2=0x7f0ec810e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:05.887 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.898 7f0ecd378700 1 --2- 172.21.15.102:0/2972236360 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0ec810e260 0x7f0ec810e6b0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 rx=0x7f0ec80f46c0 tx=0x7f0eb800ba20).stop 2024-04-17T22:31:05.887 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.898 7f0ecd378700 1 -- 172.21.15.102:0/2972236360 shutdown_connections 2024-04-17T22:31:05.887 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.898 7f0ecd378700 1 --2- 172.21.15.102:0/2972236360 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f0ec80f1f50 0x7f0ec81ab500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:05.887 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.898 7f0ecd378700 1 --2- 172.21.15.102:0/2972236360 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0ec810e260 0x7f0ec810e6b0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:05.887 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.898 7f0ecd378700 1 -- 172.21.15.102:0/2972236360 >> 172.21.15.102:0/2972236360 conn(0x7f0ec81138c0 msgr2=0x7f0ec80f4f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:05.887 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.898 7f0ecd378700 1 -- 172.21.15.102:0/2972236360 shutdown_connections 2024-04-17T22:31:05.887 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.898 7f0ecd378700 1 -- 172.21.15.102:0/2972236360 wait complete. 2024-04-17T22:31:05.909 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.921 7f0ecd378700 1 Processor -- start 2024-04-17T22:31:05.909 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.921 7f0ecd378700 1 -- start start 2024-04-17T22:31:05.909 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.921 7f0ecd378700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f0ec81adac0 0x7f0ec81adf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:05.910 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.921 7f0ecd378700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f0ec81ae540 0x7f0ec81b4590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:05.910 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.921 7f0ecd378700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0ec81b4bc0 0x7f0ec81b5010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:05.910 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.921 7f0ecd378700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ec8113a50 con 0x7f0ec81b4bc0 2024-04-17T22:31:05.910 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.921 7f0ecd378700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ec80fa1d0 con 0x7f0ec81ae540 2024-04-17T22:31:05.910 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.921 7f0ecd378700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0ec80c4a60 con 0x7f0ec81adac0 2024-04-17T22:31:05.910 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.921 7f0ec685d700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0ec81b4bc0 0x7f0ec81b5010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:05.910 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.921 7f0ec605c700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f0ec81adac0 0x7f0ec81adf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:05.910 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.921 7f0ebffff700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f0ec81ae540 0x7f0ec81b4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:05.910 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.921 7f0ec685d700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0ec81b4bc0 0x7f0ec81b5010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:54010/0 (socket says 172.21.15.102:54010) 2024-04-17T22:31:05.910 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.921 7f0ec605c700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f0ec81adac0 0x7f0ec81adf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:54780/0 (socket says 172.21.15.102:54780) 2024-04-17T22:31:05.910 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.921 7f0ec685d700 1 -- 172.21.15.102:0/2417055759 learned_addr learned my addr 172.21.15.102:0/2417055759 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:31:05.910 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.922 7f0ebffff700 1 -- 172.21.15.102:0/2417055759 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f0ec81adac0 msgr2=0x7f0ec81adf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:05.910 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.922 7f0ebffff700 1 --2- 172.21.15.102:0/2417055759 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f0ec81adac0 0x7f0ec81adf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:05.911 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.922 7f0ebffff700 1 -- 172.21.15.102:0/2417055759 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0ec81b4bc0 msgr2=0x7f0ec81b5010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:05.911 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.922 7f0ebffff700 1 --2- 172.21.15.102:0/2417055759 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0ec81b4bc0 0x7f0ec81b5010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:05.911 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.922 7f0ebffff700 1 -- 172.21.15.102:0/2417055759 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ec80db670 con 0x7f0ec81ae540 2024-04-17T22:31:05.911 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.922 7f0ec685d700 1 --2- 172.21.15.102:0/2417055759 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0ec81b4bc0 0x7f0ec81b5010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:05.911 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.922 7f0ec605c700 1 --2- 172.21.15.102:0/2417055759 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f0ec81adac0 0x7f0ec81adf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:05.911 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.922 7f0ebffff700 1 --2- 172.21.15.102:0/2417055759 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f0ec81ae540 0x7f0ec81b4590 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0x7f0ec80c6240 tx=0x7f0eb000c860).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:05.911 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.922 7f0ebf7fe700 1 -- 172.21.15.102:0/2417055759 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f0ec80db670 con 0x7f0ec81ae540 2024-04-17T22:31:05.911 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.922 7f0ebf7fe700 1 -- 172.21.15.102:0/2417055759 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0eb00070d0 con 0x7f0ec81ae540 2024-04-17T22:31:05.911 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.922 7f0ecd378700 1 -- 172.21.15.102:0/2417055759 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ec80e0590 con 0x7f0ec81ae540 2024-04-17T22:31:05.911 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.922 7f0ebf7fe700 1 -- 172.21.15.102:0/2417055759 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f0eb0005560 con 0x7f0ec81ae540 2024-04-17T22:31:05.911 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.922 7f0ecd378700 1 -- 172.21.15.102:0/2417055759 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ec81bbc80 con 0x7f0ec81ae540 2024-04-17T22:31:05.911 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.922 7f0ebf7fe700 1 -- 172.21.15.102:0/2417055759 <== mon.1 v2:172.21.15.102:3301/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7f0eb0005970 con 0x7f0ec81ae540 2024-04-17T22:31:05.911 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.922 7f0ebf7fe700 1 -- 172.21.15.102:0/2417055759 <== mon.1 v2:172.21.15.102:3301/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3194+0+0 (secure 0 0 0) 0x7f0eb0014730 con 0x7f0ec81ae540 2024-04-17T22:31:05.911 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.923 7f0ecd378700 1 -- 172.21.15.102:0/2417055759 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ec80f0820 con 0x7f0ec81ae540 2024-04-17T22:31:05.912 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:05.926 7f0ebf7fe700 1 -- 172.21.15.102:0/2417055759 <== mon.1 v2:172.21.15.102:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+110455 (secure 0 0 0) 0x7f0eb0005760 con 0x7f0ec81ae540 2024-04-17T22:31:06.017 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.031 7f0ecd378700 1 -- 172.21.15.102:0/2417055759 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0ec81aecb0 con 0x7f0ec81ae540 2024-04-17T22:31:06.018 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.032 7f0ebf7fe700 1 -- 172.21.15.102:0/2417055759 <== mon.1 v2:172.21.15.102:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) v1 ==== 74+0+8764 (secure 0 0 0) 0x7f0eb0000c30 con 0x7f0ec81ae540 2024-04-17T22:31:06.018 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-17T22:31:06.018 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":17,"fsid":"cb63e789-091d-49b1-b120-55f9db526871","created":"2024-04-17 22:28:48.209635","modified":"2024-04-17 22:29:23.257387","last_up_change":"2024-04-17 22:28:54.236215","last_in_change":"2024-04-17 22:28:54.236215","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":4,"require_min_compat_client":"nautilus","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-04-17 22:28:57.478545","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-17 22:29:06.144708","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-17 22:29:06.748924","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3351ef85-ade4-4b90-a8f4-213f4e00ba01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6812","nonce":42955}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6813","nonce":42955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6815","nonce":42955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6814","nonce":42955}]},"public_addr":"172.21.15.102:6812/42955","cluster_addr":"172.21.15.102:6813/42955","heartbeat_back_addr":"172.21.15.102:6815/42955","heartbeat_front_addr":"172.21.15.102:6814/42955","state":["exists","up"]},{"osd":1,"uuid":"04f67eac-f36b-48f5-982d-4cfc776596d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6808","nonce":42958}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6809","nonce":42958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6811","nonce":42958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6810","nonce":42958}]},"public_addr":"172.21.15.102:6808/42958","cluster_addr":"172.21.15.102:6809/42958","heartbeat_back_addr":"172.21.15.102:6811/42958","heartbeat_front_addr":"172.21.15.102:6810/42958","state":["exists","up"]},{"osd":2,"uuid":"d524204b-70b2-40a8-8f73-f9e7d44318af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6804","nonce":42954}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6805","nonce":42954}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6807","nonce":42954}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6806","nonce":42954}]},"public_addr":"172.21.15.102:6804/42954","cluster_addr":"172.21.15.102:6805/42954","heartbeat_back_addr":"172.21.15.102:6807/42954","heartbeat_front_addr":"172.21.15.102:6806/42954","state":["exists","up"]},{"osd":3,"uuid":"4fc9d20c-1ad0-484f-aafa-2c9a7574245b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6800","nonce":42953}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6801","nonce":42953}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6803","nonce":42953}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6802","nonce":42953}]},"public_addr":"172.21.15.102:6800/42953","cluster_addr":"172.21.15.102:6801/42953","heartbeat_back_addr":"172.21.15.102:6803/42953","heartbeat_front_addr":"172.21.15.102:6802/42953","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2024-04-17T22:31:06.020 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.033 7f0ecd378700 1 -- 172.21.15.102:0/2417055759 >> v1:172.21.15.102:6816/42360 conn(0x7f0eac0145a0 legacy=0x7f0eac016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:06.020 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.033 7f0ecd378700 1 -- 172.21.15.102:0/2417055759 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f0ec81ae540 msgr2=0x7f0ec81b4590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:06.020 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.033 7f0ecd378700 1 --2- 172.21.15.102:0/2417055759 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f0ec81ae540 0x7f0ec81b4590 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0x7f0ec80c6240 tx=0x7f0eb000c860).stop 2024-04-17T22:31:06.020 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.033 7f0ecd378700 1 -- 172.21.15.102:0/2417055759 shutdown_connections 2024-04-17T22:31:06.020 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.033 7f0ecd378700 1 --2- 172.21.15.102:0/2417055759 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0ec81b4bc0 0x7f0ec81b5010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:06.020 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.033 7f0ecd378700 1 --2- 172.21.15.102:0/2417055759 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f0ec81adac0 0x7f0ec81adf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:06.020 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.033 7f0ecd378700 1 --2- 172.21.15.102:0/2417055759 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f0ec81ae540 0x7f0ec81b4590 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:06.020 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.033 7f0ecd378700 1 -- 172.21.15.102:0/2417055759 >> 172.21.15.102:0/2417055759 conn(0x7f0ec80c6d20 msgr2=0x7f0ec81a8580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:06.020 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.033 7f0ecd378700 1 -- 172.21.15.102:0/2417055759 shutdown_connections 2024-04-17T22:31:06.020 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.033 7f0ecd378700 1 -- 172.21.15.102:0/2417055759 wait complete. 2024-04-17T22:31:06.030 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2024-04-17 22:28:57.478545', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '12', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-17 22:29:06.144708', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-17 22:29:06.748924', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-17T22:31:06.031 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2024-04-17T22:31:06.159 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.171 7f1e8c082700 1 Processor -- start 2024-04-17T22:31:06.159 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.171 7f1e8c082700 1 -- start start 2024-04-17T22:31:06.159 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.171 7f1e8c082700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f1e8410e240 0x7f1e8410e690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:06.160 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.172 7f1e8c082700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f1e840f1f30 0x7f1e841ab4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:06.160 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.172 7f1e8c082700 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e840f2380 con 0x7f1e8410ecc0 2024-04-17T22:31:06.160 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.172 7f1e8c082700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e840f9f10 con 0x7f1e8410e240 2024-04-17T22:31:06.160 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.172 7f1e8c082700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1e840f9030 con 0x7f1e840f1f30 2024-04-17T22:31:06.160 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.172 7f1e8967e700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f1e840f1f30 0x7f1e841ab4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:06.160 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.172 7f1e83fff700 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f1e8410ecc0 0x7f1e840f1730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:38708/0 (socket says 172.21.15.102:38708) 2024-04-17T22:31:06.160 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.172 7f1e8967e700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f1e840f1f30 0x7f1e841ab4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:54786/0 (socket says 172.21.15.102:54786) 2024-04-17T22:31:06.160 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.172 7f1e88e7d700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f1e8410e240 0x7f1e8410e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:06.160 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.172 7f1e83fff700 1 -- 172.21.15.102:0/2271543654 learned_addr learned my addr 172.21.15.102:0/2271543654 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-17T22:31:06.160 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.172 7f1e8967e700 1 -- 172.21.15.102:0/2271543654 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f1e8410e240 msgr2=0x7f1e8410e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:31:06.160 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.172 7f1e8967e700 1 --2- 172.21.15.102:0/2271543654 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f1e8410e240 0x7f1e8410e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:06.160 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.172 7f1e837fe700 1 -- 172.21.15.102:0/2271543654 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1424175809 0 0) 0x7f1e70001110 con 0x7f1e8410ecc0 2024-04-17T22:31:06.160 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.172 7f1e8967e700 1 -- 172.21.15.102:0/2271543654 >> v1:172.21.15.102:6789/0 conn(0x7f1e8410ecc0 legacy=0x7f1e840f1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:06.161 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.172 7f1e8967e700 1 -- 172.21.15.102:0/2271543654 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e840f25a0 con 0x7f1e840f1f30 2024-04-17T22:31:06.161 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.172 7f1e88e7d700 1 --2- 172.21.15.102:0/2271543654 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f1e8410e240 0x7f1e8410e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:06.161 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.172 7f1e8967e700 1 --2- 172.21.15.102:0/2271543654 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f1e840f1f30 0x7f1e841ab4e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 rx=0x7f1e840f4470 tx=0x7f1e7c009cf0).ready entity=mon.2 client_cookie=92f57b2198fa1e78 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:06.161 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.172 7f1e837fe700 1 -- 172.21.15.102:0/2271543654 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f1e840f25a0 con 0x7f1e840f1f30 2024-04-17T22:31:06.161 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.172 7f1e837fe700 1 -- 172.21.15.102:0/2271543654 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e7c00b840 con 0x7f1e840f1f30 2024-04-17T22:31:06.161 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.172 7f1e837fe700 1 -- 172.21.15.102:0/2271543654 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f1e7c00ba20 con 0x7f1e840f1f30 2024-04-17T22:31:06.161 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.172 7f1e8c082700 1 -- 172.21.15.102:0/2271543654 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f1e840f1f30 msgr2=0x7f1e841ab4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:06.161 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.172 7f1e8c082700 1 --2- 172.21.15.102:0/2271543654 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f1e840f1f30 0x7f1e841ab4e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 rx=0x7f1e840f4470 tx=0x7f1e7c009cf0).stop 2024-04-17T22:31:06.161 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.173 7f1e8c082700 1 -- 172.21.15.102:0/2271543654 shutdown_connections 2024-04-17T22:31:06.161 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.173 7f1e8c082700 1 --2- 172.21.15.102:0/2271543654 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f1e840f1f30 0x7f1e841ab4e0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:06.161 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.173 7f1e8c082700 1 --2- 172.21.15.102:0/2271543654 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f1e8410e240 0x7f1e8410e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:06.161 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.173 7f1e8c082700 1 -- 172.21.15.102:0/2271543654 >> 172.21.15.102:0/2271543654 conn(0x7f1e841138a0 msgr2=0x7f1e840f4f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:06.161 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.173 7f1e8c082700 1 -- 172.21.15.102:0/2271543654 shutdown_connections 2024-04-17T22:31:06.161 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.173 7f1e8c082700 1 -- 172.21.15.102:0/2271543654 wait complete. 2024-04-17T22:31:06.182 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.196 7f1e8c082700 1 Processor -- start 2024-04-17T22:31:06.182 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.196 7f1e8c082700 1 -- start start 2024-04-17T22:31:06.184 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.196 7f1e8c082700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1e840d5280 0x7f1e840d56d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:06.185 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.196 7f1e8c082700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f1e840d5d00 0x7f1e841b4150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:06.185 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.196 7f1e8c082700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f1e841b4780 0x7f1e841b4bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:06.185 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.196 7f1e8c082700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e840fa1b0 con 0x7f1e840d5280 2024-04-17T22:31:06.185 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.196 7f1e8c082700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e840cb3e0 con 0x7f1e841b4780 2024-04-17T22:31:06.185 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.196 7f1e8c082700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1e840c88b0 con 0x7f1e840d5d00 2024-04-17T22:31:06.185 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.196 7f1e8967e700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f1e841b4780 0x7f1e841b4bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:06.185 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.196 7f1e88e7d700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1e840d5280 0x7f1e840d56d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:06.185 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.196 7f1e88e7d700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1e840d5280 0x7f1e840d56d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:54016/0 (socket says 172.21.15.102:54016) 2024-04-17T22:31:06.185 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.196 7f1e8967e700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f1e841b4780 0x7f1e841b4bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:51954/0 (socket says 172.21.15.102:51954) 2024-04-17T22:31:06.185 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.196 7f1e88e7d700 1 -- 172.21.15.102:0/10092865 learned_addr learned my addr 172.21.15.102:0/10092865 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:31:06.185 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.196 7f1e83fff700 1 --2- 172.21.15.102:0/10092865 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f1e840d5d00 0x7f1e841b4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:06.185 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.197 7f1e83fff700 1 -- 172.21.15.102:0/10092865 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f1e841b4780 msgr2=0x7f1e841b4bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:06.186 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.197 7f1e83fff700 1 --2- 172.21.15.102:0/10092865 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f1e841b4780 0x7f1e841b4bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:06.186 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.197 7f1e83fff700 1 -- 172.21.15.102:0/10092865 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1e840d5280 msgr2=0x7f1e840d56d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:06.186 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.197 7f1e83fff700 1 --2- 172.21.15.102:0/10092865 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1e840d5280 0x7f1e840d56d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:06.186 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.197 7f1e83fff700 1 -- 172.21.15.102:0/10092865 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e840ce960 con 0x7f1e840d5d00 2024-04-17T22:31:06.186 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.197 7f1e8967e700 1 --2- 172.21.15.102:0/10092865 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f1e841b4780 0x7f1e841b4bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:06.186 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.197 7f1e88e7d700 1 --2- 172.21.15.102:0/10092865 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1e840d5280 0x7f1e840d56d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:06.186 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.197 7f1e83fff700 1 --2- 172.21.15.102:0/10092865 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f1e840d5d00 0x7f1e841b4150 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 rx=0x7f1e840c71c0 tx=0x7f1e7000c7b0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:06.186 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.197 7f1e81ffb700 1 -- 172.21.15.102:0/10092865 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f1e840ce960 con 0x7f1e840d5d00 2024-04-17T22:31:06.186 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.197 7f1e81ffb700 1 -- 172.21.15.102:0/10092865 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e700070d0 con 0x7f1e840d5d00 2024-04-17T22:31:06.186 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.197 7f1e8c082700 1 -- 172.21.15.102:0/10092865 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e840c90c0 con 0x7f1e840d5d00 2024-04-17T22:31:06.186 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.197 7f1e81ffb700 1 -- 172.21.15.102:0/10092865 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f1e70005620 con 0x7f1e840d5d00 2024-04-17T22:31:06.186 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.197 7f1e8c082700 1 -- 172.21.15.102:0/10092865 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e841bb840 con 0x7f1e840d5d00 2024-04-17T22:31:06.187 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.197 7f1e81ffb700 1 -- 172.21.15.102:0/10092865 <== mon.2 v2:172.21.15.102:3302/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7f1e70005820 con 0x7f1e840d5d00 2024-04-17T22:31:06.187 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.197 7f1e81ffb700 1 -- 172.21.15.102:0/10092865 <== mon.2 v2:172.21.15.102:3302/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3194+0+0 (secure 0 0 0) 0x7f1e700146a0 con 0x7f1e840d5d00 2024-04-17T22:31:06.187 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.198 7f1e8c082700 1 -- 172.21.15.102:0/10092865 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e840f0800 con 0x7f1e840d5d00 2024-04-17T22:31:06.187 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.201 7f1e81ffb700 1 -- 172.21.15.102:0/10092865 <== mon.2 v2:172.21.15.102:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+110455 (secure 0 0 0) 0x7f1e7000cda0 con 0x7f1e840d5d00 2024-04-17T22:31:06.293 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.307 7f1e8c082700 1 -- 172.21.15.102:0/10092865 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f1e841ae890 con 0x7f1e840d5d00 2024-04-17T22:31:06.293 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.307 7f1e81ffb700 1 -- 172.21.15.102:0/10092865 <== mon.2 v2:172.21.15.102:3302/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v17) v1 ==== 92+0+10 (secure 0 0 0) 0x7f1e70000c30 con 0x7f1e840d5d00 2024-04-17T22:31:06.293 INFO:teuthology.orchestra.run.smithi102.stdout:pg_num: 8 2024-04-17T22:31:06.295 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.308 7f1e8c082700 1 -- 172.21.15.102:0/10092865 >> v1:172.21.15.102:6816/42360 conn(0x7f1e6c0145a0 legacy=0x7f1e6c016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:06.295 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.308 7f1e8c082700 1 -- 172.21.15.102:0/10092865 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f1e840d5d00 msgr2=0x7f1e841b4150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:06.295 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.308 7f1e8c082700 1 --2- 172.21.15.102:0/10092865 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f1e840d5d00 0x7f1e841b4150 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 rx=0x7f1e840c71c0 tx=0x7f1e7000c7b0).stop 2024-04-17T22:31:06.295 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.309 7f1e8c082700 1 -- 172.21.15.102:0/10092865 shutdown_connections 2024-04-17T22:31:06.295 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.309 7f1e8c082700 1 --2- 172.21.15.102:0/10092865 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f1e841b4780 0x7f1e841b4bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:06.295 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.309 7f1e8c082700 1 --2- 172.21.15.102:0/10092865 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1e840d5280 0x7f1e840d56d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:06.295 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.309 7f1e8c082700 1 --2- 172.21.15.102:0/10092865 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f1e840d5d00 0x7f1e841b4150 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:06.295 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.309 7f1e8c082700 1 -- 172.21.15.102:0/10092865 >> 172.21.15.102:0/10092865 conn(0x7f1e840c7cf0 msgr2=0x7f1e840c6160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:06.295 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.309 7f1e8c082700 1 -- 172.21.15.102:0/10092865 shutdown_connections 2024-04-17T22:31:06.296 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.309 7f1e8c082700 1 -- 172.21.15.102:0/10092865 wait complete. 2024-04-17T22:31:06.305 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-17T22:31:06.437 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.448 7fcadf09a700 1 Processor -- start 2024-04-17T22:31:06.437 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.448 7fcadf09a700 1 -- start start 2024-04-17T22:31:06.437 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.449 7fcadf09a700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fcad80f5840 0x7fcad810b010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:06.437 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.449 7fcadf09a700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fcad810ba90 0x7fcad81a02f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:06.437 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.449 7fcadf09a700 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcad8103b40 con 0x7fcad810b640 2024-04-17T22:31:06.437 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.449 7fcadf09a700 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcad810bee0 con 0x7fcad80f5840 2024-04-17T22:31:06.437 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.449 7fcadf09a700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcad80fa370 con 0x7fcad810ba90 2024-04-17T22:31:06.437 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.449 7fcad77fe700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fcad80f5840 0x7fcad810b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:06.437 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.449 7fcad7fff700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fcad810ba90 0x7fcad81a02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:06.437 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.449 7fcacffff700 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7fcad810b640 0x7fcad819faf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:38724/0 (socket says 172.21.15.102:38724) 2024-04-17T22:31:06.438 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.449 7fcad77fe700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fcad80f5840 0x7fcad810b010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:54030/0 (socket says 172.21.15.102:54030) 2024-04-17T22:31:06.438 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.449 7fcacffff700 1 -- 172.21.15.102:0/189286030 learned_addr learned my addr 172.21.15.102:0/189286030 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-17T22:31:06.438 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.449 7fcad7fff700 1 -- 172.21.15.102:0/189286030 >> v2:172.21.15.102:3300/0 conn(0x7fcad80f5840 msgr2=0x7fcad810b010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:31:06.438 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.449 7fcad7fff700 1 --2- 172.21.15.102:0/189286030 >> v2:172.21.15.102:3300/0 conn(0x7fcad80f5840 0x7fcad810b010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:06.438 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.449 7fcad7fff700 1 -- 172.21.15.102:0/189286030 >> v1:172.21.15.102:6789/0 conn(0x7fcad810b640 legacy=0x7fcad819faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:06.438 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.449 7fcad7fff700 1 -- 172.21.15.102:0/189286030 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcad81a0920 con 0x7fcad810ba90 2024-04-17T22:31:06.438 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.449 7fcad6ffd700 1 -- 172.21.15.102:0/189286030 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 20162927 0 0) 0x7fcac0001110 con 0x7fcad810b640 2024-04-17T22:31:06.438 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.449 7fcad77fe700 1 --2- 172.21.15.102:0/189286030 >> v2:172.21.15.102:3300/0 conn(0x7fcad80f5840 0x7fcad810b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:06.438 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.449 7fcad7fff700 1 --2- 172.21.15.102:0/189286030 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fcad810ba90 0x7fcad81a02f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 rx=0x7fcad80f4470 tx=0x7fcad0009cc0).ready entity=mon.2 client_cookie=1ff862ddd864909e server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:06.438 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.449 7fcad6ffd700 1 -- 172.21.15.102:0/189286030 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fcad81a0920 con 0x7fcad810ba90 2024-04-17T22:31:06.438 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.449 7fcad6ffd700 1 -- 172.21.15.102:0/189286030 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcad000b840 con 0x7fcad810ba90 2024-04-17T22:31:06.439 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.449 7fcad6ffd700 1 -- 172.21.15.102:0/189286030 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fcad000ba20 con 0x7fcad810ba90 2024-04-17T22:31:06.439 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.449 7fcadf09a700 1 -- 172.21.15.102:0/189286030 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fcad810ba90 msgr2=0x7fcad81a02f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:06.439 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.449 7fcadf09a700 1 --2- 172.21.15.102:0/189286030 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fcad810ba90 0x7fcad81a02f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 rx=0x7fcad80f4470 tx=0x7fcad0009cc0).stop 2024-04-17T22:31:06.439 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.450 7fcadf09a700 1 -- 172.21.15.102:0/189286030 shutdown_connections 2024-04-17T22:31:06.439 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.450 7fcadf09a700 1 --2- 172.21.15.102:0/189286030 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fcad810ba90 0x7fcad81a02f0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:06.439 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.450 7fcadf09a700 1 --2- 172.21.15.102:0/189286030 >> v2:172.21.15.102:3300/0 conn(0x7fcad80f5840 0x7fcad810b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:06.439 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.450 7fcadf09a700 1 -- 172.21.15.102:0/189286030 >> 172.21.15.102:0/189286030 conn(0x7fcad81138a0 msgr2=0x7fcad80f4f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:06.439 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.450 7fcadf09a700 1 -- 172.21.15.102:0/189286030 shutdown_connections 2024-04-17T22:31:06.439 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.450 7fcadf09a700 1 -- 172.21.15.102:0/189286030 wait complete. 2024-04-17T22:31:06.461 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.473 7fcadf09a700 1 Processor -- start 2024-04-17T22:31:06.461 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.473 7fcadf09a700 1 -- start start 2024-04-17T22:31:06.461 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.473 7fcadf09a700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fcad80cc170 0x7fcad80f17b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:06.461 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.473 7fcadf09a700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fcad80f1de0 0x7fcad81ad7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:06.461 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.473 7fcadf09a700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcad80f2230 0x7fcad81addd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:06.461 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.473 7fcadf09a700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcad80cfac0 con 0x7fcad80f2230 2024-04-17T22:31:06.461 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.473 7fcadf09a700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcad81ae400 con 0x7fcad80cc170 2024-04-17T22:31:06.461 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.473 7fcadf09a700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcad81bb9a0 con 0x7fcad80f1de0 2024-04-17T22:31:06.461 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.473 7fcacffff700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fcad80f1de0 0x7fcad81ad7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:06.462 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.473 7fcad7fff700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcad80f2230 0x7fcad81addd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:06.462 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.473 7fcacffff700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fcad80f1de0 0x7fcad81ad7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:54806/0 (socket says 172.21.15.102:54806) 2024-04-17T22:31:06.462 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.473 7fcad7fff700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcad80f2230 0x7fcad81addd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:54036/0 (socket says 172.21.15.102:54036) 2024-04-17T22:31:06.462 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.473 7fcacffff700 1 -- 172.21.15.102:0/277273437 learned_addr learned my addr 172.21.15.102:0/277273437 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:31:06.462 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.473 7fcad77fe700 1 --2- 172.21.15.102:0/277273437 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fcad80cc170 0x7fcad80f17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:06.462 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.473 7fcad77fe700 1 -- 172.21.15.102:0/277273437 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fcad80f1de0 msgr2=0x7fcad81ad7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:06.462 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.473 7fcad77fe700 1 --2- 172.21.15.102:0/277273437 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fcad80f1de0 0x7fcad81ad7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:06.462 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.474 7fcad77fe700 1 -- 172.21.15.102:0/277273437 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcad80f2230 msgr2=0x7fcad81addd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:06.462 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.474 7fcad77fe700 1 --2- 172.21.15.102:0/277273437 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcad80f2230 0x7fcad81addd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:06.462 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.474 7fcad77fe700 1 -- 172.21.15.102:0/277273437 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcad81bbb50 con 0x7fcad80cc170 2024-04-17T22:31:06.462 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.474 7fcacffff700 1 --2- 172.21.15.102:0/277273437 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fcad80f1de0 0x7fcad81ad7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:06.462 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.474 7fcad7fff700 1 --2- 172.21.15.102:0/277273437 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcad80f2230 0x7fcad81addd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:06.462 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.474 7fcad77fe700 1 --2- 172.21.15.102:0/277273437 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fcad80cc170 0x7fcad80f17b0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 rx=0x7fcad80d7480 tx=0x7fcac800ed00).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:06.463 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.474 7fcad57fa700 1 -- 172.21.15.102:0/277273437 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fcad81bbb50 con 0x7fcad80cc170 2024-04-17T22:31:06.463 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.474 7fcad57fa700 1 -- 172.21.15.102:0/277273437 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcac800c7b0 con 0x7fcad80cc170 2024-04-17T22:31:06.463 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.474 7fcadf09a700 1 -- 172.21.15.102:0/277273437 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcad81bbd50 con 0x7fcad80cc170 2024-04-17T22:31:06.463 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.474 7fcad57fa700 1 -- 172.21.15.102:0/277273437 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fcac800f040 con 0x7fcad80cc170 2024-04-17T22:31:06.463 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.474 7fcadf09a700 1 -- 172.21.15.102:0/277273437 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcad81bc130 con 0x7fcad80cc170 2024-04-17T22:31:06.463 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.474 7fcad57fa700 1 -- 172.21.15.102:0/277273437 <== mon.1 v2:172.21.15.102:3301/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7fcac8011070 con 0x7fcad80cc170 2024-04-17T22:31:06.463 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.474 7fcad57fa700 1 -- 172.21.15.102:0/277273437 <== mon.1 v2:172.21.15.102:3301/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3194+0+0 (secure 0 0 0) 0x7fcac8010040 con 0x7fcad80cc170 2024-04-17T22:31:06.463 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.475 7fcadf09a700 1 -- 172.21.15.102:0/277273437 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcad81ae630 con 0x7fcad80cc170 2024-04-17T22:31:06.464 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.478 7fcad57fa700 1 -- 172.21.15.102:0/277273437 <== mon.1 v2:172.21.15.102:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+110455 (secure 0 0 0) 0x7fcac800c5a0 con 0x7fcad80cc170 2024-04-17T22:31:06.568 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.583 7fcadf09a700 1 -- 172.21.15.102:0/277273437 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fcad80f0800 con 0x7fcad80cc170 2024-04-17T22:31:06.569 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.583 7fcad57fa700 1 -- 172.21.15.102:0/277273437 <== mon.1 v2:172.21.15.102:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v17) v1 ==== 104+0+11 (secure 0 0 0) 0x7fcac8000c30 con 0x7fcad80cc170 2024-04-17T22:31:06.569 INFO:teuthology.orchestra.run.smithi102.stdout:pg_num: 64 2024-04-17T22:31:06.571 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.584 7fcadf09a700 1 -- 172.21.15.102:0/277273437 >> v1:172.21.15.102:6816/42360 conn(0x7fcabc0145a0 legacy=0x7fcabc016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:06.571 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.584 7fcadf09a700 1 -- 172.21.15.102:0/277273437 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fcad80cc170 msgr2=0x7fcad80f17b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:06.571 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.584 7fcadf09a700 1 --2- 172.21.15.102:0/277273437 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fcad80cc170 0x7fcad80f17b0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 rx=0x7fcad80d7480 tx=0x7fcac800ed00).stop 2024-04-17T22:31:06.571 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.584 7fcadf09a700 1 -- 172.21.15.102:0/277273437 shutdown_connections 2024-04-17T22:31:06.571 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.584 7fcadf09a700 1 --2- 172.21.15.102:0/277273437 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcad80f2230 0x7fcad81addd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:06.571 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.584 7fcadf09a700 1 --2- 172.21.15.102:0/277273437 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fcad80cc170 0x7fcad80f17b0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:06.571 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.584 7fcadf09a700 1 --2- 172.21.15.102:0/277273437 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fcad80f1de0 0x7fcad81ad7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:06.571 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.584 7fcadf09a700 1 -- 172.21.15.102:0/277273437 >> 172.21.15.102:0/277273437 conn(0x7fcad80c3840 msgr2=0x7fcad80d76e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:06.571 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.584 7fcadf09a700 1 -- 172.21.15.102:0/277273437 shutdown_connections 2024-04-17T22:31:06.571 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.584 7fcadf09a700 1 -- 172.21.15.102:0/277273437 wait complete. 2024-04-17T22:31:06.581 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-17T22:31:06.709 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.721 7f3e2e9f6700 1 Processor -- start 2024-04-17T22:31:06.710 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.721 7f3e2e9f6700 1 -- start start 2024-04-17T22:31:06.710 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.722 7f3e2e9f6700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f3e281a64b0 0x7f3e281a8910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:06.710 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.722 7f3e2e9f6700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f3e281a8f40 0x7f3e281ab3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:06.710 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.722 7f3e2e9f6700 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e2810ee90 con 0x7f3e2810e240 2024-04-17T22:31:06.710 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.722 7f3e2e9f6700 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e280f9f10 con 0x7f3e281a8f40 2024-04-17T22:31:06.710 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.722 7f3e2e9f6700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e280f9030 con 0x7f3e281a64b0 2024-04-17T22:31:06.710 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.722 7f3e27dec700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f3e281a8f40 0x7f3e281ab3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:06.710 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.722 7f3e26dea700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f3e281a64b0 0x7f3e281a8910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:06.710 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.722 7f3e275eb700 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f3e2810e240 0x7f3e2810e690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:38726/0 (socket says 172.21.15.102:38726) 2024-04-17T22:31:06.710 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.722 7f3e27dec700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f3e281a8f40 0x7f3e281ab3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:54044/0 (socket says 172.21.15.102:54044) 2024-04-17T22:31:06.711 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.722 7f3e26dea700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f3e281a64b0 0x7f3e281a8910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:51976/0 (socket says 172.21.15.102:51976) 2024-04-17T22:31:06.711 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.722 7f3e275eb700 1 -- 172.21.15.102:0/1094055367 learned_addr learned my addr 172.21.15.102:0/1094055367 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-17T22:31:06.711 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.722 7f3e265e9700 1 -- 172.21.15.102:0/1094055367 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3407279296 0 0) 0x7f3e1c001110 con 0x7f3e2810e240 2024-04-17T22:31:06.711 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.722 7f3e265e9700 1 -- 172.21.15.102:0/1094055367 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3e14003270 con 0x7f3e2810e240 2024-04-17T22:31:06.711 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.722 7f3e27dec700 1 -- 172.21.15.102:0/1094055367 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f3e281a64b0 msgr2=0x7f3e281a8910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:31:06.711 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.722 7f3e27dec700 1 --2- 172.21.15.102:0/1094055367 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f3e281a64b0 0x7f3e281a8910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:06.711 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.722 7f3e27dec700 1 -- 172.21.15.102:0/1094055367 >> v1:172.21.15.102:6789/0 conn(0x7f3e2810e240 legacy=0x7f3e2810e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:06.711 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.722 7f3e27dec700 1 -- 172.21.15.102:0/1094055367 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e281ac9e0 con 0x7f3e281a8f40 2024-04-17T22:31:06.711 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.722 7f3e27dec700 1 --2- 172.21.15.102:0/1094055367 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3e281a8f40 0x7f3e281ab3a0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 rx=0x7f3e280f46a0 tx=0x7f3e20009d40).ready entity=mon.0 client_cookie=dc3050e1df5e79ad server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:06.711 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.722 7f3e265e9700 1 -- 172.21.15.102:0/1094055367 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f3e281ac9e0 con 0x7f3e281a8f40 2024-04-17T22:31:06.711 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.723 7f3e265e9700 1 -- 172.21.15.102:0/1094055367 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e2000b7c0 con 0x7f3e281a8f40 2024-04-17T22:31:06.711 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.723 7f3e265e9700 1 -- 172.21.15.102:0/1094055367 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f3e2000b9f0 con 0x7f3e281a8f40 2024-04-17T22:31:06.711 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.723 7f3e2e9f6700 1 -- 172.21.15.102:0/1094055367 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3e281a8f40 msgr2=0x7f3e281ab3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:06.711 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.723 7f3e2e9f6700 1 --2- 172.21.15.102:0/1094055367 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3e281a8f40 0x7f3e281ab3a0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 rx=0x7f3e280f46a0 tx=0x7f3e20009d40).stop 2024-04-17T22:31:06.711 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.723 7f3e2e9f6700 1 -- 172.21.15.102:0/1094055367 shutdown_connections 2024-04-17T22:31:06.712 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.723 7f3e2e9f6700 1 --2- 172.21.15.102:0/1094055367 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3e281a8f40 0x7f3e281ab3a0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:06.712 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.723 7f3e2e9f6700 1 --2- 172.21.15.102:0/1094055367 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f3e281a64b0 0x7f3e281a8910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:06.712 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.723 7f3e2e9f6700 1 -- 172.21.15.102:0/1094055367 >> 172.21.15.102:0/1094055367 conn(0x7f3e281138a0 msgr2=0x7f3e280f4f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:06.712 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.723 7f3e2e9f6700 1 -- 172.21.15.102:0/1094055367 shutdown_connections 2024-04-17T22:31:06.712 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.723 7f3e2e9f6700 1 -- 172.21.15.102:0/1094055367 wait complete. 2024-04-17T22:31:06.731 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.745 7f3e2e9f6700 1 Processor -- start 2024-04-17T22:31:06.731 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.746 7f3e2e9f6700 1 -- start start 2024-04-17T22:31:06.733 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.746 7f3e2e9f6700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f3e280cdbd0 0x7f3e280d6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:06.734 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.746 7f3e2e9f6700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3e280d6870 0x7f3e281b3f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:06.734 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.746 7f3e2e9f6700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f3e281b4530 0x7f3e281b4980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:06.734 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.746 7f3e2e9f6700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e28113a40 con 0x7f3e280d6870 2024-04-17T22:31:06.734 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.746 7f3e2e9f6700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e280fa1b0 con 0x7f3e280cdbd0 2024-04-17T22:31:06.734 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.746 7f3e2e9f6700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3e280d6cc0 con 0x7f3e281b4530 2024-04-17T22:31:06.734 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.746 7f3e26dea700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3e280d6870 0x7f3e281b3f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:06.734 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.746 7f3e27dec700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f3e281b4530 0x7f3e281b4980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:06.734 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.746 7f3e26dea700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3e280d6870 0x7f3e281b3f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:54046/0 (socket says 172.21.15.102:54046) 2024-04-17T22:31:06.734 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.746 7f3e26dea700 1 -- 172.21.15.102:0/2846779500 learned_addr learned my addr 172.21.15.102:0/2846779500 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:31:06.734 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.746 7f3e275eb700 1 --2- 172.21.15.102:0/2846779500 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f3e280cdbd0 0x7f3e280d6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:06.734 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.746 7f3e26dea700 1 -- 172.21.15.102:0/2846779500 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f3e281b4530 msgr2=0x7f3e281b4980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:06.734 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.746 7f3e26dea700 1 --2- 172.21.15.102:0/2846779500 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f3e281b4530 0x7f3e281b4980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:06.734 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.746 7f3e26dea700 1 -- 172.21.15.102:0/2846779500 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f3e280cdbd0 msgr2=0x7f3e280d6240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:06.735 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.746 7f3e26dea700 1 --2- 172.21.15.102:0/2846779500 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f3e280cdbd0 0x7f3e280d6240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:06.735 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.746 7f3e26dea700 1 -- 172.21.15.102:0/2846779500 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e281bb3e0 con 0x7f3e280d6870 2024-04-17T22:31:06.735 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.746 7f3e275eb700 1 --2- 172.21.15.102:0/2846779500 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f3e280cdbd0 0x7f3e280d6240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:06.735 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.746 7f3e27dec700 1 --2- 172.21.15.102:0/2846779500 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f3e281b4530 0x7f3e281b4980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:06.735 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.747 7f3e26dea700 1 --2- 172.21.15.102:0/2846779500 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3e280d6870 0x7f3e281b3f00 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 rx=0x7f3e280c5430 tx=0x7f3e1800ecd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:06.735 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.747 7f3e24de6700 1 -- 172.21.15.102:0/2846779500 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f3e281bb3e0 con 0x7f3e280d6870 2024-04-17T22:31:06.735 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.747 7f3e24de6700 1 -- 172.21.15.102:0/2846779500 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e1800c9a0 con 0x7f3e280d6870 2024-04-17T22:31:06.735 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.747 7f3e2e9f6700 1 -- 172.21.15.102:0/2846779500 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e281bb5e0 con 0x7f3e280d6870 2024-04-17T22:31:06.735 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.747 7f3e24de6700 1 -- 172.21.15.102:0/2846779500 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f3e1800f040 con 0x7f3e280d6870 2024-04-17T22:31:06.735 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.747 7f3e2e9f6700 1 -- 172.21.15.102:0/2846779500 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e281bb7e0 con 0x7f3e280d6870 2024-04-17T22:31:06.735 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.747 7f3e24de6700 1 -- 172.21.15.102:0/2846779500 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7f3e18011070 con 0x7f3e280d6870 2024-04-17T22:31:06.735 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.747 7f3e24de6700 1 -- 172.21.15.102:0/2846779500 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3194+0+0 (secure 0 0 0) 0x7f3e18010040 con 0x7f3e280d6870 2024-04-17T22:31:06.735 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.748 7f3e2e9f6700 1 -- 172.21.15.102:0/2846779500 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e280f0800 con 0x7f3e280d6870 2024-04-17T22:31:06.737 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.751 7f3e24de6700 1 -- 172.21.15.102:0/2846779500 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+110455 (secure 0 0 0) 0x7f3e1800c5a0 con 0x7f3e280d6870 2024-04-17T22:31:06.841 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.855 7f3e2e9f6700 1 -- 172.21.15.102:0/2846779500 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f3e281ae600 con 0x7f3e280d6870 2024-04-17T22:31:06.842 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.856 7f3e24de6700 1 -- 172.21.15.102:0/2846779500 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v17) v1 ==== 100+0+11 (secure 0 0 0) 0x7f3e18000c30 con 0x7f3e280d6870 2024-04-17T22:31:06.842 INFO:teuthology.orchestra.run.smithi102.stdout:pg_num: 64 2024-04-17T22:31:06.843 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.857 7f3e2e9f6700 1 -- 172.21.15.102:0/2846779500 >> v1:172.21.15.102:6816/42360 conn(0x7f3e080145a0 legacy=0x7f3e08016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:06.843 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.857 7f3e2e9f6700 1 -- 172.21.15.102:0/2846779500 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3e280d6870 msgr2=0x7f3e281b3f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:06.844 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.857 7f3e2e9f6700 1 --2- 172.21.15.102:0/2846779500 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3e280d6870 0x7f3e281b3f00 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 rx=0x7f3e280c5430 tx=0x7f3e1800ecd0).stop 2024-04-17T22:31:06.844 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.857 7f3e2e9f6700 1 -- 172.21.15.102:0/2846779500 shutdown_connections 2024-04-17T22:31:06.844 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.857 7f3e2e9f6700 1 --2- 172.21.15.102:0/2846779500 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f3e281b4530 0x7f3e281b4980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:06.844 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.857 7f3e2e9f6700 1 --2- 172.21.15.102:0/2846779500 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f3e280cdbd0 0x7f3e280d6240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:06.844 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.857 7f3e2e9f6700 1 --2- 172.21.15.102:0/2846779500 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3e280d6870 0x7f3e281b3f00 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:06.844 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.857 7f3e2e9f6700 1 -- 172.21.15.102:0/2846779500 >> 172.21.15.102:0/2846779500 conn(0x7f3e280e0f70 msgr2=0x7f3e280dac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:06.844 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.857 7f3e2e9f6700 1 -- 172.21.15.102:0/2846779500 shutdown_connections 2024-04-17T22:31:06.844 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.857 7f3e2e9f6700 1 -- 172.21.15.102:0/2846779500 wait complete. 2024-04-17T22:31:06.854 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T22:31:06.981 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.993 7fd2296b4700 1 Processor -- start 2024-04-17T22:31:06.981 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.993 7fd2296b4700 1 -- start start 2024-04-17T22:31:06.981 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.993 7fd2296b4700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd22410e1d0 0x7fd22410e620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:06.981 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.993 7fd2296b4700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fd22410ec50 0x7fd2240f16c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:06.981 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.993 7fd2296b4700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd2240f1cf0 0x7fd2241aa460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:06.981 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.993 7fd2296b4700 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2240f9ea0 con 0x7fd22410ec50 2024-04-17T22:31:06.981 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.993 7fd2296b4700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2240f8fc0 con 0x7fd2240f1cf0 2024-04-17T22:31:06.981 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.993 7fd2296b4700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd2240fa140 con 0x7fd22410e1d0 2024-04-17T22:31:06.982 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.993 7fd222ab2700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd2240f1cf0 0x7fd2241aa460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:06.982 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.993 7fd21bfff700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fd22410ec50 0x7fd2240f16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:06.982 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.993 7fd2222b1700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd22410e1d0 0x7fd22410e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:06.982 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.993 7fd222ab2700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd2240f1cf0 0x7fd2241aa460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:51996/0 (socket says 172.21.15.102:51996) 2024-04-17T22:31:06.982 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.993 7fd21bfff700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fd22410ec50 0x7fd2240f16c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:54056/0 (socket says 172.21.15.102:54056) 2024-04-17T22:31:06.982 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.993 7fd222ab2700 1 -- 172.21.15.102:0/2513866855 learned_addr learned my addr 172.21.15.102:0/2513866855 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:31:06.982 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.994 7fd2222b1700 1 -- 172.21.15.102:0/2513866855 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd2240f1cf0 msgr2=0x7fd2241aa460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:31:06.982 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.994 7fd2222b1700 1 --2- 172.21.15.102:0/2513866855 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd2240f1cf0 0x7fd2241aa460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:06.982 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.994 7fd2222b1700 1 -- 172.21.15.102:0/2513866855 >> v2:172.21.15.102:3300/0 conn(0x7fd22410ec50 msgr2=0x7fd2240f16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:31:06.982 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.994 7fd2222b1700 1 --2- 172.21.15.102:0/2513866855 >> v2:172.21.15.102:3300/0 conn(0x7fd22410ec50 0x7fd2240f16c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:06.982 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.994 7fd2222b1700 1 -- 172.21.15.102:0/2513866855 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2240f2140 con 0x7fd22410e1d0 2024-04-17T22:31:06.982 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.994 7fd222ab2700 1 --2- 172.21.15.102:0/2513866855 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd2240f1cf0 0x7fd2241aa460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:06.982 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.994 7fd21bfff700 1 --2- 172.21.15.102:0/2513866855 >> v2:172.21.15.102:3300/0 conn(0x7fd22410ec50 0x7fd2240f16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:06.982 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.994 7fd2222b1700 1 --2- 172.21.15.102:0/2513866855 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd22410e1d0 0x7fd22410e620 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 rx=0x7fd2240f4400 tx=0x7fd21400b9d0).ready entity=mon.2 client_cookie=dbf6d539e479c2e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:06.982 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.994 7fd221ab0700 1 -- 172.21.15.102:0/2513866855 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd2240f2140 con 0x7fd22410e1d0 2024-04-17T22:31:06.983 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.994 7fd221ab0700 1 -- 172.21.15.102:0/2513866855 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd21400d570 con 0x7fd22410e1d0 2024-04-17T22:31:06.983 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.994 7fd221ab0700 1 -- 172.21.15.102:0/2513866855 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd21400d940 con 0x7fd22410e1d0 2024-04-17T22:31:06.983 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.994 7fd2296b4700 1 -- 172.21.15.102:0/2513866855 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd22410e1d0 msgr2=0x7fd22410e620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:06.983 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.994 7fd2296b4700 1 --2- 172.21.15.102:0/2513866855 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd22410e1d0 0x7fd22410e620 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 rx=0x7fd2240f4400 tx=0x7fd21400b9d0).stop 2024-04-17T22:31:06.983 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.995 7fd2296b4700 1 -- 172.21.15.102:0/2513866855 shutdown_connections 2024-04-17T22:31:06.983 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.995 7fd2296b4700 1 --2- 172.21.15.102:0/2513866855 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd2240f1cf0 0x7fd2241aa460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:06.983 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.995 7fd2296b4700 1 --2- 172.21.15.102:0/2513866855 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd22410e1d0 0x7fd22410e620 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:06.983 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.995 7fd2296b4700 1 --2- 172.21.15.102:0/2513866855 >> v2:172.21.15.102:3300/0 conn(0x7fd22410ec50 0x7fd2240f16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:06.983 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.995 7fd2296b4700 1 -- 172.21.15.102:0/2513866855 >> 172.21.15.102:0/2513866855 conn(0x7fd224113830 msgr2=0x7fd2240f4ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:06.983 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.995 7fd2296b4700 1 -- 172.21.15.102:0/2513866855 shutdown_connections 2024-04-17T22:31:06.983 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:06.995 7fd2296b4700 1 -- 172.21.15.102:0/2513866855 wait complete. 2024-04-17T22:31:07.005 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.017 7fd2296b4700 1 Processor -- start 2024-04-17T22:31:07.005 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.017 7fd2296b4700 1 -- start start 2024-04-17T22:31:07.005 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.018 7fd2296b4700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd2240d5f40 0x7fd2240d6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:07.005 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.018 7fd2296b4700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd2240d69c0 0x7fd2241b3d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:07.006 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.018 7fd2296b4700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd2241b4340 0x7fd2241b4790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:07.006 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.018 7fd2296b4700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd224113a40 con 0x7fd2240d69c0 2024-04-17T22:31:07.006 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.018 7fd2296b4700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2240cd9a0 con 0x7fd2241b4340 2024-04-17T22:31:07.006 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.018 7fd2296b4700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd2240cd350 con 0x7fd2240d5f40 2024-04-17T22:31:07.006 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.018 7fd21bfff700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd2240d69c0 0x7fd2241b3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:07.006 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.018 7fd2222b1700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd2240d5f40 0x7fd2240d6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:07.006 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.018 7fd222ab2700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd2241b4340 0x7fd2241b4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:07.006 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.018 7fd21bfff700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd2240d69c0 0x7fd2241b3d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:54064/0 (socket says 172.21.15.102:54064) 2024-04-17T22:31:07.006 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.018 7fd21bfff700 1 -- 172.21.15.102:0/894382340 learned_addr learned my addr 172.21.15.102:0/894382340 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:31:07.006 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.018 7fd2222b1700 1 -- 172.21.15.102:0/894382340 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd2241b4340 msgr2=0x7fd2241b4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:07.006 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.018 7fd2222b1700 1 --2- 172.21.15.102:0/894382340 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd2241b4340 0x7fd2241b4790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:07.006 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.018 7fd2222b1700 1 -- 172.21.15.102:0/894382340 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd2240d69c0 msgr2=0x7fd2241b3d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:07.006 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.018 7fd2222b1700 1 --2- 172.21.15.102:0/894382340 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd2240d69c0 0x7fd2241b3d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:07.006 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.018 7fd2222b1700 1 -- 172.21.15.102:0/894382340 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2240c43c0 con 0x7fd2240d5f40 2024-04-17T22:31:07.006 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.018 7fd222ab2700 1 --2- 172.21.15.102:0/894382340 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd2241b4340 0x7fd2241b4790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:07.007 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.018 7fd2222b1700 1 --2- 172.21.15.102:0/894382340 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd2240d5f40 0x7fd2240d6390 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0x7fd2240c5bb0 tx=0x7fd2140068c0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:07.007 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.018 7fd21b7fe700 1 -- 172.21.15.102:0/894382340 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd214017040 con 0x7fd2240d5f40 2024-04-17T22:31:07.007 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.018 7fd21b7fe700 1 -- 172.21.15.102:0/894382340 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd214011980 con 0x7fd2240d5f40 2024-04-17T22:31:07.007 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.018 7fd2296b4700 1 -- 172.21.15.102:0/894382340 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2240cd650 con 0x7fd2240d5f40 2024-04-17T22:31:07.007 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.019 7fd21b7fe700 1 -- 172.21.15.102:0/894382340 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd214011d50 con 0x7fd2240d5f40 2024-04-17T22:31:07.007 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.019 7fd2296b4700 1 -- 172.21.15.102:0/894382340 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2241bb510 con 0x7fd2240d5f40 2024-04-17T22:31:07.007 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.019 7fd21b7fe700 1 -- 172.21.15.102:0/894382340 <== mon.2 v2:172.21.15.102:3302/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7fd2140273b0 con 0x7fd2240d5f40 2024-04-17T22:31:07.007 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.019 7fd21b7fe700 1 -- 172.21.15.102:0/894382340 <== mon.2 v2:172.21.15.102:3302/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3194+0+0 (secure 0 0 0) 0x7fd214018070 con 0x7fd2240d5f40 2024-04-17T22:31:07.007 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.020 7fd2296b4700 1 -- 172.21.15.102:0/894382340 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2240f0790 con 0x7fd2240d5f40 2024-04-17T22:31:07.009 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.023 7fd21b7fe700 1 -- 172.21.15.102:0/894382340 <== mon.2 v2:172.21.15.102:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+110455 (secure 0 0 0) 0x7fd21400d360 con 0x7fd2240d5f40 2024-04-17T22:31:07.114 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.128 7fd2296b4700 1 -- 172.21.15.102:0/894382340 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd2241ae4d0 con 0x7fd2240d5f40 2024-04-17T22:31:07.114 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.128 7fd21b7fe700 1 -- 172.21.15.102:0/894382340 <== mon.2 v2:172.21.15.102:3302/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) v1 ==== 74+0+8764 (secure 0 0 0) 0x7fd214011770 con 0x7fd2240d5f40 2024-04-17T22:31:07.114 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-17T22:31:07.115 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":17,"fsid":"cb63e789-091d-49b1-b120-55f9db526871","created":"2024-04-17 22:28:48.209635","modified":"2024-04-17 22:29:23.257387","last_up_change":"2024-04-17 22:28:54.236215","last_in_change":"2024-04-17 22:28:54.236215","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":4,"require_min_compat_client":"nautilus","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-04-17 22:28:57.478545","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-17 22:29:06.144708","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-17 22:29:06.748924","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3351ef85-ade4-4b90-a8f4-213f4e00ba01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6812","nonce":42955}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6813","nonce":42955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6815","nonce":42955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6814","nonce":42955}]},"public_addr":"172.21.15.102:6812/42955","cluster_addr":"172.21.15.102:6813/42955","heartbeat_back_addr":"172.21.15.102:6815/42955","heartbeat_front_addr":"172.21.15.102:6814/42955","state":["exists","up"]},{"osd":1,"uuid":"04f67eac-f36b-48f5-982d-4cfc776596d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6808","nonce":42958}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6809","nonce":42958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6811","nonce":42958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6810","nonce":42958}]},"public_addr":"172.21.15.102:6808/42958","cluster_addr":"172.21.15.102:6809/42958","heartbeat_back_addr":"172.21.15.102:6811/42958","heartbeat_front_addr":"172.21.15.102:6810/42958","state":["exists","up"]},{"osd":2,"uuid":"d524204b-70b2-40a8-8f73-f9e7d44318af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6804","nonce":42954}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6805","nonce":42954}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6807","nonce":42954}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6806","nonce":42954}]},"public_addr":"172.21.15.102:6804/42954","cluster_addr":"172.21.15.102:6805/42954","heartbeat_back_addr":"172.21.15.102:6807/42954","heartbeat_front_addr":"172.21.15.102:6806/42954","state":["exists","up"]},{"osd":3,"uuid":"4fc9d20c-1ad0-484f-aafa-2c9a7574245b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6800","nonce":42953}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6801","nonce":42953}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6803","nonce":42953}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6802","nonce":42953}]},"public_addr":"172.21.15.102:6800/42953","cluster_addr":"172.21.15.102:6801/42953","heartbeat_back_addr":"172.21.15.102:6803/42953","heartbeat_front_addr":"172.21.15.102:6802/42953","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2024-04-17T22:31:07.116 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.130 7fd2296b4700 1 -- 172.21.15.102:0/894382340 >> v1:172.21.15.102:6816/42360 conn(0x7fd2080145a0 legacy=0x7fd208016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:07.116 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.130 7fd2296b4700 1 -- 172.21.15.102:0/894382340 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd2240d5f40 msgr2=0x7fd2240d6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:07.116 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.130 7fd2296b4700 1 --2- 172.21.15.102:0/894382340 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd2240d5f40 0x7fd2240d6390 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0x7fd2240c5bb0 tx=0x7fd2140068c0).stop 2024-04-17T22:31:07.116 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.130 7fd2296b4700 1 -- 172.21.15.102:0/894382340 shutdown_connections 2024-04-17T22:31:07.116 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.130 7fd2296b4700 1 --2- 172.21.15.102:0/894382340 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd2241b4340 0x7fd2241b4790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:07.116 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.130 7fd2296b4700 1 --2- 172.21.15.102:0/894382340 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd2240d5f40 0x7fd2240d6390 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:07.116 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.130 7fd2296b4700 1 --2- 172.21.15.102:0/894382340 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd2240d69c0 0x7fd2241b3d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:07.116 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.130 7fd2296b4700 1 -- 172.21.15.102:0/894382340 >> 172.21.15.102:0/894382340 conn(0x7fd2240e0f60 msgr2=0x7fd2240ca440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:07.116 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.130 7fd2296b4700 1 -- 172.21.15.102:0/894382340 shutdown_connections 2024-04-17T22:31:07.116 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.130 7fd2296b4700 1 -- 172.21.15.102:0/894382340 wait complete. 2024-04-17T22:31:07.127 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2024-04-17 22:28:57.478545', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '12', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-17 22:29:06.144708', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-17 22:29:06.748924', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-17T22:31:07.127 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2024-04-17T22:31:07.256 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.268 7f6dd037d700 1 Processor -- start 2024-04-17T22:31:07.256 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.268 7f6dd037d700 1 -- start start 2024-04-17T22:31:07.257 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.268 7f6dd037d700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6dc810e240 0x7f6dc810e690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:07.257 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.268 7f6dd037d700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f6dc810ecc0 0x7f6dc80f1730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:07.257 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.268 7f6dd037d700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f6dc80f1d60 0x7f6dc81aa4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:07.257 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.268 7f6dd037d700 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6dc80f9f10 con 0x7f6dc80f1d60 2024-04-17T22:31:07.257 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.268 7f6dd037d700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6dc80f9030 con 0x7f6dc810e240 2024-04-17T22:31:07.257 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.268 7f6dd037d700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6dc80fa1b0 con 0x7f6dc810ecc0 2024-04-17T22:31:07.257 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.268 7f6dcd178700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6dc810e240 0x7f6dc810e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:07.257 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.268 7f6dcd979700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f6dc80f1d60 0x7f6dc81aa4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:07.257 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.268 7f6dcc977700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f6dc810ecc0 0x7f6dc80f1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:07.257 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.269 7f6dcd979700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f6dc80f1d60 0x7f6dc81aa4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:54080/0 (socket says 172.21.15.102:54080) 2024-04-17T22:31:07.257 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.269 7f6dcd178700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6dc810e240 0x7f6dc810e690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:52010/0 (socket says 172.21.15.102:52010) 2024-04-17T22:31:07.257 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.269 7f6dcd979700 1 -- 172.21.15.102:0/2539827273 learned_addr learned my addr 172.21.15.102:0/2539827273 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:31:07.257 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.269 7f6dcd979700 1 -- 172.21.15.102:0/2539827273 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f6dc810ecc0 msgr2=0x7f6dc80f1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:31:07.258 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.269 7f6dcd979700 1 --2- 172.21.15.102:0/2539827273 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f6dc810ecc0 0x7f6dc80f1730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:07.258 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.269 7f6dcd979700 1 -- 172.21.15.102:0/2539827273 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6dc810e240 msgr2=0x7f6dc810e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:31:07.258 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.269 7f6dcd979700 1 --2- 172.21.15.102:0/2539827273 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6dc810e240 0x7f6dc810e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:07.258 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.269 7f6dcd979700 1 -- 172.21.15.102:0/2539827273 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dc80f21b0 con 0x7f6dc80f1d60 2024-04-17T22:31:07.258 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.269 7f6dcd178700 1 --2- 172.21.15.102:0/2539827273 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6dc810e240 0x7f6dc810e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:07.258 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.269 7f6dcc977700 1 --2- 172.21.15.102:0/2539827273 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f6dc810ecc0 0x7f6dc80f1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:07.258 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.269 7f6dcd979700 1 --2- 172.21.15.102:0/2539827273 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6dc80f1d60 0x7f6dc81aa4d0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 rx=0x7f6dc8103580 tx=0x7f6dc0009e40).ready entity=mon.0 client_cookie=db8cd37086eb21b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:07.258 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.269 7f6db7fff700 1 -- 172.21.15.102:0/2539827273 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f6dc80f21b0 con 0x7f6dc80f1d60 2024-04-17T22:31:07.258 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.269 7f6db7fff700 1 -- 172.21.15.102:0/2539827273 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6dc000b830 con 0x7f6dc80f1d60 2024-04-17T22:31:07.258 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.269 7f6db7fff700 1 -- 172.21.15.102:0/2539827273 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f6dc000ba10 con 0x7f6dc80f1d60 2024-04-17T22:31:07.258 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.269 7f6dd037d700 1 -- 172.21.15.102:0/2539827273 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6dc80f1d60 msgr2=0x7f6dc81aa4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:07.258 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.269 7f6dd037d700 1 --2- 172.21.15.102:0/2539827273 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6dc80f1d60 0x7f6dc81aa4d0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 rx=0x7f6dc8103580 tx=0x7f6dc0009e40).stop 2024-04-17T22:31:07.258 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.270 7f6dd037d700 1 -- 172.21.15.102:0/2539827273 shutdown_connections 2024-04-17T22:31:07.259 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.270 7f6dd037d700 1 --2- 172.21.15.102:0/2539827273 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6dc80f1d60 0x7f6dc81aa4d0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:07.259 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.270 7f6dd037d700 1 --2- 172.21.15.102:0/2539827273 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6dc810e240 0x7f6dc810e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:07.259 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.270 7f6dd037d700 1 --2- 172.21.15.102:0/2539827273 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f6dc810ecc0 0x7f6dc80f1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:07.259 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.270 7f6dd037d700 1 -- 172.21.15.102:0/2539827273 >> 172.21.15.102:0/2539827273 conn(0x7f6dc81138a0 msgr2=0x7f6dc80f4f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:07.259 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.270 7f6dd037d700 1 -- 172.21.15.102:0/2539827273 shutdown_connections 2024-04-17T22:31:07.259 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.270 7f6dd037d700 1 -- 172.21.15.102:0/2539827273 wait complete. 2024-04-17T22:31:07.280 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.292 7f6dd037d700 1 Processor -- start 2024-04-17T22:31:07.280 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.292 7f6dd037d700 1 -- start start 2024-04-17T22:31:07.280 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.293 7f6dd037d700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6dc80cd970 0x7f6dc81ad2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:07.280 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.293 7f6dd037d700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f6dc81ad8d0 0x7f6dc81b3c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:07.280 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.293 7f6dd037d700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6dc81add20 0x7f6dc81b4260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:07.280 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.293 7f6dd037d700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dc80e2c80 con 0x7f6dc81add20 2024-04-17T22:31:07.281 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.293 7f6dd037d700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6dc80cd0d0 con 0x7f6dc80cd970 2024-04-17T22:31:07.281 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.293 7f6dd037d700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6dc80c41a0 con 0x7f6dc81ad8d0 2024-04-17T22:31:07.281 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.293 7f6dcc977700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f6dc81ad8d0 0x7f6dc81b3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:07.281 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.293 7f6dcd178700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6dc80cd970 0x7f6dc81ad2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:07.281 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.293 7f6dcd979700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6dc81add20 0x7f6dc81b4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:07.281 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.293 7f6dcc977700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f6dc81ad8d0 0x7f6dc81b3c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:54848/0 (socket says 172.21.15.102:54848) 2024-04-17T22:31:07.281 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.293 7f6dcd178700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6dc80cd970 0x7f6dc81ad2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:52022/0 (socket says 172.21.15.102:52022) 2024-04-17T22:31:07.281 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.293 7f6dcc977700 1 -- 172.21.15.102:0/3596517255 learned_addr learned my addr 172.21.15.102:0/3596517255 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:31:07.281 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.293 7f6dcd979700 1 -- 172.21.15.102:0/3596517255 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f6dc81ad8d0 msgr2=0x7f6dc81b3c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:07.281 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.293 7f6dcd979700 1 --2- 172.21.15.102:0/3596517255 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f6dc81ad8d0 0x7f6dc81b3c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:07.281 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.293 7f6dcd979700 1 -- 172.21.15.102:0/3596517255 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6dc80cd970 msgr2=0x7f6dc81ad2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:07.281 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.293 7f6dcd979700 1 --2- 172.21.15.102:0/3596517255 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6dc80cd970 0x7f6dc81ad2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:07.281 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.293 7f6dcd979700 1 -- 172.21.15.102:0/3596517255 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dc81b4890 con 0x7f6dc81add20 2024-04-17T22:31:07.281 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.293 7f6dcc977700 1 --2- 172.21.15.102:0/3596517255 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f6dc81ad8d0 0x7f6dc81b3c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:07.282 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.293 7f6dcd178700 1 --2- 172.21.15.102:0/3596517255 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6dc80cd970 0x7f6dc81ad2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:07.282 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.293 7f6dcd979700 1 --2- 172.21.15.102:0/3596517255 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6dc81add20 0x7f6dc81b4260 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 rx=0x7f6dc80c59c0 tx=0x7f6dc0006c40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:07.282 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.293 7f6db67fc700 1 -- 172.21.15.102:0/3596517255 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f6dc81b4890 con 0x7f6dc81add20 2024-04-17T22:31:07.282 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.293 7f6db67fc700 1 -- 172.21.15.102:0/3596517255 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6dc00119c0 con 0x7f6dc81add20 2024-04-17T22:31:07.282 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.293 7f6dd037d700 1 -- 172.21.15.102:0/3596517255 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6dc81b4af0 con 0x7f6dc81add20 2024-04-17T22:31:07.282 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.293 7f6db67fc700 1 -- 172.21.15.102:0/3596517255 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f6dc0011d90 con 0x7f6dc81add20 2024-04-17T22:31:07.282 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.294 7f6dd037d700 1 -- 172.21.15.102:0/3596517255 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6dc81bb590 con 0x7f6dc81add20 2024-04-17T22:31:07.282 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.294 7f6db67fc700 1 -- 172.21.15.102:0/3596517255 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7f6dc001c3e0 con 0x7f6dc81add20 2024-04-17T22:31:07.282 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.294 7f6db67fc700 1 -- 172.21.15.102:0/3596517255 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3194+0+0 (secure 0 0 0) 0x7f6dc0020d00 con 0x7f6dc81add20 2024-04-17T22:31:07.282 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.295 7f6dd037d700 1 -- 172.21.15.102:0/3596517255 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6dc80f0800 con 0x7f6dc81add20 2024-04-17T22:31:07.283 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.298 7f6db67fc700 1 -- 172.21.15.102:0/3596517255 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+110455 (secure 0 0 0) 0x7f6dc000b620 con 0x7f6dc81add20 2024-04-17T22:31:07.390 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.404 7f6dd037d700 1 -- 172.21.15.102:0/3596517255 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f6dc80efed0 con 0x7f6dc81add20 2024-04-17T22:31:07.390 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.404 7f6db67fc700 1 -- 172.21.15.102:0/3596517255 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v17) v1 ==== 92+0+10 (secure 0 0 0) 0x7f6dc00117b0 con 0x7f6dc81add20 2024-04-17T22:31:07.390 INFO:teuthology.orchestra.run.smithi102.stdout:pg_num: 8 2024-04-17T22:31:07.392 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.405 7f6dd037d700 1 -- 172.21.15.102:0/3596517255 >> v1:172.21.15.102:6816/42360 conn(0x7f6da80145a0 legacy=0x7f6da8016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:07.392 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.405 7f6dd037d700 1 -- 172.21.15.102:0/3596517255 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6dc81add20 msgr2=0x7f6dc81b4260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:07.392 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.405 7f6dd037d700 1 --2- 172.21.15.102:0/3596517255 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6dc81add20 0x7f6dc81b4260 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 rx=0x7f6dc80c59c0 tx=0x7f6dc0006c40).stop 2024-04-17T22:31:07.392 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.405 7f6dd037d700 1 -- 172.21.15.102:0/3596517255 shutdown_connections 2024-04-17T22:31:07.392 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.405 7f6dd037d700 1 --2- 172.21.15.102:0/3596517255 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6dc81add20 0x7f6dc81b4260 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:07.392 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.405 7f6dd037d700 1 --2- 172.21.15.102:0/3596517255 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6dc80cd970 0x7f6dc81ad2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:07.392 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.405 7f6dd037d700 1 --2- 172.21.15.102:0/3596517255 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f6dc81ad8d0 0x7f6dc81b3c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:07.392 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.405 7f6dd037d700 1 -- 172.21.15.102:0/3596517255 >> 172.21.15.102:0/3596517255 conn(0x7f6dc80e0e00 msgr2=0x7f6dc80dab20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:07.392 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.406 7f6dd037d700 1 -- 172.21.15.102:0/3596517255 shutdown_connections 2024-04-17T22:31:07.393 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.406 7f6dd037d700 1 -- 172.21.15.102:0/3596517255 wait complete. 2024-04-17T22:31:07.402 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-17T22:31:07.531 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.542 7fb2f2deb700 1 Processor -- start 2024-04-17T22:31:07.531 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.542 7fb2f2deb700 1 -- start start 2024-04-17T22:31:07.531 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.543 7fb2f2deb700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb2ec0f5750 0x7fb2ec10c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:07.531 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.543 7fb2f2deb700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fb2ec10c720 0x7fb2ec1a0be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:07.531 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.543 7fb2f2deb700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb2ec10cb70 0x7fb2ec1a1210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:07.531 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.543 7fb2f2deb700 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2ec0f93c0 con 0x7fb2ec10c720 2024-04-17T22:31:07.531 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.543 7fb2f2deb700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2ec1a1840 con 0x7fb2ec10cb70 2024-04-17T22:31:07.531 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.543 7fb2f2deb700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb2ec1a1a70 con 0x7fb2ec0f5750 2024-04-17T22:31:07.532 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.543 7fb2ebfff700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb2ec10cb70 0x7fb2ec1a1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:07.532 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.543 7fb2eaffd700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fb2ec10c720 0x7fb2ec1a0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:07.532 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.543 7fb2eb7fe700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb2ec0f5750 0x7fb2ec10c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:07.532 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.543 7fb2eaffd700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fb2ec10c720 0x7fb2ec1a0be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:54098/0 (socket says 172.21.15.102:54098) 2024-04-17T22:31:07.532 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.543 7fb2ebfff700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb2ec10cb70 0x7fb2ec1a1210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:52024/0 (socket says 172.21.15.102:52024) 2024-04-17T22:31:07.532 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.543 7fb2eaffd700 1 -- 172.21.15.102:0/1075915826 learned_addr learned my addr 172.21.15.102:0/1075915826 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:31:07.532 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.543 7fb2eaffd700 1 -- 172.21.15.102:0/1075915826 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb2ec0f5750 msgr2=0x7fb2ec10c0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:31:07.532 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.543 7fb2eaffd700 1 --2- 172.21.15.102:0/1075915826 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb2ec0f5750 0x7fb2ec10c0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:07.532 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.543 7fb2eaffd700 1 -- 172.21.15.102:0/1075915826 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb2ec10cb70 msgr2=0x7fb2ec1a1210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:31:07.532 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.543 7fb2eaffd700 1 --2- 172.21.15.102:0/1075915826 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb2ec10cb70 0x7fb2ec1a1210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:07.532 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.543 7fb2eaffd700 1 -- 172.21.15.102:0/1075915826 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2ec1a82f0 con 0x7fb2ec10c720 2024-04-17T22:31:07.532 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.543 7fb2eb7fe700 1 --2- 172.21.15.102:0/1075915826 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb2ec0f5750 0x7fb2ec10c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:07.533 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.543 7fb2ebfff700 1 --2- 172.21.15.102:0/1075915826 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb2ec10cb70 0x7fb2ec1a1210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:07.533 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.543 7fb2eaffd700 1 --2- 172.21.15.102:0/1075915826 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb2ec10c720 0x7fb2ec1a0be0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 rx=0x7fb2dc002f70 tx=0x7fb2dc009a50).ready entity=mon.0 client_cookie=5e52bed79b982da server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:07.533 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.544 7fb2ea7fc700 1 -- 172.21.15.102:0/1075915826 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb2ec1a82f0 con 0x7fb2ec10c720 2024-04-17T22:31:07.533 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.544 7fb2ea7fc700 1 -- 172.21.15.102:0/1075915826 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2dc00f510 con 0x7fb2ec10c720 2024-04-17T22:31:07.533 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.544 7fb2ea7fc700 1 -- 172.21.15.102:0/1075915826 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb2dc00f6f0 con 0x7fb2ec10c720 2024-04-17T22:31:07.533 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.544 7fb2f2deb700 1 -- 172.21.15.102:0/1075915826 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb2ec10c720 msgr2=0x7fb2ec1a0be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:07.533 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.544 7fb2f2deb700 1 --2- 172.21.15.102:0/1075915826 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb2ec10c720 0x7fb2ec1a0be0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 rx=0x7fb2dc002f70 tx=0x7fb2dc009a50).stop 2024-04-17T22:31:07.533 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.544 7fb2f2deb700 1 -- 172.21.15.102:0/1075915826 shutdown_connections 2024-04-17T22:31:07.533 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.544 7fb2f2deb700 1 --2- 172.21.15.102:0/1075915826 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb2ec10cb70 0x7fb2ec1a1210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:07.533 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.544 7fb2f2deb700 1 --2- 172.21.15.102:0/1075915826 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb2ec0f5750 0x7fb2ec10c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:07.533 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.544 7fb2f2deb700 1 --2- 172.21.15.102:0/1075915826 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb2ec10c720 0x7fb2ec1a0be0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:07.533 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.544 7fb2f2deb700 1 -- 172.21.15.102:0/1075915826 >> 172.21.15.102:0/1075915826 conn(0x7fb2ec1138a0 msgr2=0x7fb2ec0f4f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:07.533 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.544 7fb2f2deb700 1 -- 172.21.15.102:0/1075915826 shutdown_connections 2024-04-17T22:31:07.533 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.544 7fb2f2deb700 1 -- 172.21.15.102:0/1075915826 wait complete. 2024-04-17T22:31:07.553 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.567 7fb2f2deb700 1 Processor -- start 2024-04-17T22:31:07.553 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.567 7fb2f2deb700 1 -- start start 2024-04-17T22:31:07.555 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.567 7fb2f2deb700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb2ec1a6440 0x7fb2ec1ad370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:07.556 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.567 7fb2f2deb700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb2ec1ad9a0 0x7fb2ec1b3e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:07.556 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.567 7fb2f2deb700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb2ec1addf0 0x7fb2ec1b4460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:07.556 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.567 7fb2f2deb700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2ec0e64a0 con 0x7fb2ec1a6440 2024-04-17T22:31:07.556 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.567 7fb2f2deb700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2ec0f8040 con 0x7fb2ec1ad9a0 2024-04-17T22:31:07.556 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.567 7fb2f2deb700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb2ec0cf8a0 con 0x7fb2ec1addf0 2024-04-17T22:31:07.556 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.568 7fb2eb7fe700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb2ec1a6440 0x7fb2ec1ad370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:07.556 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.568 7fb2eaffd700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb2ec1ad9a0 0x7fb2ec1b3e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:07.556 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.568 7fb2eb7fe700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb2ec1a6440 0x7fb2ec1ad370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:54108/0 (socket says 172.21.15.102:54108) 2024-04-17T22:31:07.556 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.568 7fb2eb7fe700 1 -- 172.21.15.102:0/1097679865 learned_addr learned my addr 172.21.15.102:0/1097679865 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:31:07.557 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.568 7fb2ebfff700 1 --2- 172.21.15.102:0/1097679865 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb2ec1addf0 0x7fb2ec1b4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:07.557 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.568 7fb2eb7fe700 1 -- 172.21.15.102:0/1097679865 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb2ec1addf0 msgr2=0x7fb2ec1b4460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:07.557 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.568 7fb2eb7fe700 1 --2- 172.21.15.102:0/1097679865 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb2ec1addf0 0x7fb2ec1b4460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:07.557 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.568 7fb2eb7fe700 1 -- 172.21.15.102:0/1097679865 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb2ec1ad9a0 msgr2=0x7fb2ec1b3e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:07.557 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.568 7fb2eb7fe700 1 --2- 172.21.15.102:0/1097679865 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb2ec1ad9a0 0x7fb2ec1b3e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:07.557 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.568 7fb2eb7fe700 1 -- 172.21.15.102:0/1097679865 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2ec1b4a90 con 0x7fb2ec1a6440 2024-04-17T22:31:07.557 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.568 7fb2ebfff700 1 --2- 172.21.15.102:0/1097679865 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb2ec1addf0 0x7fb2ec1b4460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:07.557 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.568 7fb2eb7fe700 1 --2- 172.21.15.102:0/1097679865 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb2ec1a6440 0x7fb2ec1ad370 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 rx=0x7fb2ec0c5cd0 tx=0x7fb2e000ed30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:07.557 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.568 7fb2e8ff9700 1 -- 172.21.15.102:0/1097679865 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb2ec1b4a90 con 0x7fb2ec1a6440 2024-04-17T22:31:07.557 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.568 7fb2e8ff9700 1 -- 172.21.15.102:0/1097679865 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2e000c940 con 0x7fb2ec1a6440 2024-04-17T22:31:07.557 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.568 7fb2e8ff9700 1 -- 172.21.15.102:0/1097679865 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb2e000f040 con 0x7fb2ec1a6440 2024-04-17T22:31:07.557 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.568 7fb2f2deb700 1 -- 172.21.15.102:0/1097679865 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2ec1b4db0 con 0x7fb2ec1a6440 2024-04-17T22:31:07.557 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.568 7fb2f2deb700 1 -- 172.21.15.102:0/1097679865 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2ec1bb700 con 0x7fb2ec1a6440 2024-04-17T22:31:07.558 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.569 7fb2e8ff9700 1 -- 172.21.15.102:0/1097679865 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7fb2e0011070 con 0x7fb2ec1a6440 2024-04-17T22:31:07.558 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.569 7fb2e8ff9700 1 -- 172.21.15.102:0/1097679865 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3194+0+0 (secure 0 0 0) 0x7fb2e0010040 con 0x7fb2ec1a6440 2024-04-17T22:31:07.558 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.569 7fb2f2deb700 1 -- 172.21.15.102:0/1097679865 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2ec0f0800 con 0x7fb2ec1a6440 2024-04-17T22:31:07.558 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.572 7fb2e8ff9700 1 -- 172.21.15.102:0/1097679865 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+110455 (secure 0 0 0) 0x7fb2e000c540 con 0x7fb2ec1a6440 2024-04-17T22:31:07.663 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.676 7fb2f2deb700 1 -- 172.21.15.102:0/1097679865 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb2ec1ae550 con 0x7fb2ec1a6440 2024-04-17T22:31:07.663 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.677 7fb2e8ff9700 1 -- 172.21.15.102:0/1097679865 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v17) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb2e0000c30 con 0x7fb2ec1a6440 2024-04-17T22:31:07.663 INFO:teuthology.orchestra.run.smithi102.stdout:pg_num: 64 2024-04-17T22:31:07.664 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.678 7fb2f2deb700 1 -- 172.21.15.102:0/1097679865 >> v1:172.21.15.102:6816/42360 conn(0x7fb2cc0145a0 legacy=0x7fb2cc016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:07.664 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.678 7fb2f2deb700 1 -- 172.21.15.102:0/1097679865 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb2ec1a6440 msgr2=0x7fb2ec1ad370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:07.664 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.678 7fb2f2deb700 1 --2- 172.21.15.102:0/1097679865 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb2ec1a6440 0x7fb2ec1ad370 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 rx=0x7fb2ec0c5cd0 tx=0x7fb2e000ed30).stop 2024-04-17T22:31:07.664 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.678 7fb2f2deb700 1 -- 172.21.15.102:0/1097679865 shutdown_connections 2024-04-17T22:31:07.664 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.678 7fb2f2deb700 1 --2- 172.21.15.102:0/1097679865 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb2ec1addf0 0x7fb2ec1b4460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:07.665 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.678 7fb2f2deb700 1 --2- 172.21.15.102:0/1097679865 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb2ec1a6440 0x7fb2ec1ad370 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:07.665 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.678 7fb2f2deb700 1 --2- 172.21.15.102:0/1097679865 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb2ec1ad9a0 0x7fb2ec1b3e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:07.665 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.678 7fb2f2deb700 1 -- 172.21.15.102:0/1097679865 >> 172.21.15.102:0/1097679865 conn(0x7fb2ec0c3880 msgr2=0x7fb2ec0dae50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:07.665 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.678 7fb2f2deb700 1 -- 172.21.15.102:0/1097679865 shutdown_connections 2024-04-17T22:31:07.665 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.678 7fb2f2deb700 1 -- 172.21.15.102:0/1097679865 wait complete. 2024-04-17T22:31:07.675 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-17T22:31:07.804 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.816 7fdc88d24700 1 Processor -- start 2024-04-17T22:31:07.804 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.816 7fdc88d24700 1 -- start start 2024-04-17T22:31:07.804 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.816 7fdc88d24700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fdc8410b810 0x7fdc841a52c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:07.804 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.816 7fdc88d24700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fdc8410bc60 0x7fdc841a7900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:07.804 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.816 7fdc88d24700 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc84103b40 con 0x7fdc840f5840 2024-04-17T22:31:07.804 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.816 7fdc88d24700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc840fa370 con 0x7fdc8410bc60 2024-04-17T22:31:07.804 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.816 7fdc88d24700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdc841a0b00 con 0x7fdc8410b810 2024-04-17T22:31:07.804 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.816 7fdc8130f700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fdc8410b810 0x7fdc841a52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:07.805 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.816 7fdc8130f700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fdc8410b810 0x7fdc841a52c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:54882/0 (socket says 172.21.15.102:54882) 2024-04-17T22:31:07.805 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.816 7fdc82311700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fdc8410bc60 0x7fdc841a7900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:07.805 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.816 7fdc8130f700 1 -- 172.21.15.102:0/347054222 learned_addr learned my addr 172.21.15.102:0/347054222 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:31:07.805 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.817 7fdc82311700 1 -- 172.21.15.102:0/347054222 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fdc8410b810 msgr2=0x7fdc841a52c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:31:07.805 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.817 7fdc82311700 1 --2- 172.21.15.102:0/347054222 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fdc8410b810 0x7fdc841a52c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:07.805 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.817 7fdc82311700 1 -- 172.21.15.102:0/347054222 >> v1:172.21.15.102:6789/0 conn(0x7fdc840f5840 legacy=0x7fdc8410b010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:07.805 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.817 7fdc82311700 1 -- 172.21.15.102:0/347054222 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc841a8f40 con 0x7fdc8410bc60 2024-04-17T22:31:07.805 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.817 7fdc8130f700 1 --2- 172.21.15.102:0/347054222 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fdc8410b810 0x7fdc841a52c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:07.805 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.817 7fdc82311700 1 --2- 172.21.15.102:0/347054222 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fdc8410bc60 0x7fdc841a7900 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0x7fdc7c0032f0 tx=0x7fdc7c009d40).ready entity=mon.1 client_cookie=f222d77a5259c36b server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:07.805 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.817 7fdc80b0e700 1 -- 172.21.15.102:0/347054222 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fdc841a8f40 con 0x7fdc8410bc60 2024-04-17T22:31:07.805 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.817 7fdc80b0e700 1 -- 172.21.15.102:0/347054222 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc7c00b800 con 0x7fdc8410bc60 2024-04-17T22:31:07.805 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.817 7fdc80b0e700 1 -- 172.21.15.102:0/347054222 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fdc7c00b9e0 con 0x7fdc8410bc60 2024-04-17T22:31:07.805 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.817 7fdc88d24700 1 -- 172.21.15.102:0/347054222 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fdc8410bc60 msgr2=0x7fdc841a7900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:07.805 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.817 7fdc88d24700 1 --2- 172.21.15.102:0/347054222 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fdc8410bc60 0x7fdc841a7900 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0x7fdc7c0032f0 tx=0x7fdc7c009d40).stop 2024-04-17T22:31:07.806 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.817 7fdc88d24700 1 -- 172.21.15.102:0/347054222 shutdown_connections 2024-04-17T22:31:07.806 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.817 7fdc88d24700 1 --2- 172.21.15.102:0/347054222 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fdc8410bc60 0x7fdc841a7900 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:07.806 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.817 7fdc88d24700 1 --2- 172.21.15.102:0/347054222 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fdc8410b810 0x7fdc841a52c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:07.806 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.817 7fdc88d24700 1 -- 172.21.15.102:0/347054222 >> 172.21.15.102:0/347054222 conn(0x7fdc841138a0 msgr2=0x7fdc840f4f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:07.806 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.818 7fdc88d24700 1 -- 172.21.15.102:0/347054222 shutdown_connections 2024-04-17T22:31:07.806 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.818 7fdc88d24700 1 -- 172.21.15.102:0/347054222 wait complete. 2024-04-17T22:31:07.828 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.840 7fdc88d24700 1 Processor -- start 2024-04-17T22:31:07.828 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.840 7fdc88d24700 1 -- start start 2024-04-17T22:31:07.829 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.840 7fdc88d24700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdc840f17b0 0x7fdc840f1c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:07.829 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.840 7fdc88d24700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fdc840f2230 0x7fdc841ad930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:07.829 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.840 7fdc88d24700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fdc841adf60 0x7fdc841bba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:07.829 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.840 7fdc88d24700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc840d4670 con 0x7fdc840f17b0 2024-04-17T22:31:07.829 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.840 7fdc88d24700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc840d4a50 con 0x7fdc841adf60 2024-04-17T22:31:07.829 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.840 7fdc88d24700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdc840e0600 con 0x7fdc840f2230 2024-04-17T22:31:07.829 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.841 7fdc81b10700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdc840f17b0 0x7fdc840f1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:07.829 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.841 7fdc81b10700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdc840f17b0 0x7fdc840f1c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:54122/0 (socket says 172.21.15.102:54122) 2024-04-17T22:31:07.829 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.841 7fdc82311700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fdc841adf60 0x7fdc841bba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:07.829 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.841 7fdc81b10700 1 -- 172.21.15.102:0/3602196316 learned_addr learned my addr 172.21.15.102:0/3602196316 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:31:07.829 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.841 7fdc8130f700 1 --2- 172.21.15.102:0/3602196316 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fdc840f2230 0x7fdc841ad930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:07.829 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.841 7fdc8130f700 1 -- 172.21.15.102:0/3602196316 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fdc841adf60 msgr2=0x7fdc841bba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:07.829 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.841 7fdc8130f700 1 --2- 172.21.15.102:0/3602196316 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fdc841adf60 0x7fdc841bba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:07.830 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.841 7fdc8130f700 1 -- 172.21.15.102:0/3602196316 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdc840f17b0 msgr2=0x7fdc840f1c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:07.830 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.841 7fdc8130f700 1 --2- 172.21.15.102:0/3602196316 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdc840f17b0 0x7fdc840f1c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:07.830 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.841 7fdc8130f700 1 -- 172.21.15.102:0/3602196316 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc841ae3b0 con 0x7fdc840f2230 2024-04-17T22:31:07.830 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.841 7fdc81b10700 1 --2- 172.21.15.102:0/3602196316 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdc840f17b0 0x7fdc840f1c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:07.830 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.841 7fdc82311700 1 --2- 172.21.15.102:0/3602196316 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fdc841adf60 0x7fdc841bba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:07.830 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.841 7fdc8130f700 1 --2- 172.21.15.102:0/3602196316 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fdc840f2230 0x7fdc841ad930 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 rx=0x7fdc841a5150 tx=0x7fdc7400ea50).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:07.830 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.841 7fdc72ffd700 1 -- 172.21.15.102:0/3602196316 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fdc841ae3b0 con 0x7fdc840f2230 2024-04-17T22:31:07.830 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.841 7fdc72ffd700 1 -- 172.21.15.102:0/3602196316 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc74012600 con 0x7fdc840f2230 2024-04-17T22:31:07.830 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.841 7fdc88d24700 1 -- 172.21.15.102:0/3602196316 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc841ae6d0 con 0x7fdc840f2230 2024-04-17T22:31:07.830 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.841 7fdc72ffd700 1 -- 172.21.15.102:0/3602196316 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fdc7400f040 con 0x7fdc840f2230 2024-04-17T22:31:07.830 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.841 7fdc88d24700 1 -- 172.21.15.102:0/3602196316 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc841bc290 con 0x7fdc840f2230 2024-04-17T22:31:07.830 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.842 7fdc72ffd700 1 -- 172.21.15.102:0/3602196316 <== mon.2 v2:172.21.15.102:3302/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7fdc74011070 con 0x7fdc840f2230 2024-04-17T22:31:07.830 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.842 7fdc72ffd700 1 -- 172.21.15.102:0/3602196316 <== mon.2 v2:172.21.15.102:3302/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3194+0+0 (secure 0 0 0) 0x7fdc74010040 con 0x7fdc840f2230 2024-04-17T22:31:07.831 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.842 7fdc88d24700 1 -- 172.21.15.102:0/3602196316 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc840f0800 con 0x7fdc840f2230 2024-04-17T22:31:07.831 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.845 7fdc72ffd700 1 -- 172.21.15.102:0/3602196316 <== mon.2 v2:172.21.15.102:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+110455 (secure 0 0 0) 0x7fdc740123f0 con 0x7fdc840f2230 2024-04-17T22:31:07.936 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.950 7fdc88d24700 1 -- 172.21.15.102:0/3602196316 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fdc840f2680 con 0x7fdc840f2230 2024-04-17T22:31:07.936 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.950 7fdc72ffd700 1 -- 172.21.15.102:0/3602196316 <== mon.2 v2:172.21.15.102:3302/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v17) v1 ==== 100+0+11 (secure 0 0 0) 0x7fdc74000c30 con 0x7fdc840f2230 2024-04-17T22:31:07.936 INFO:teuthology.orchestra.run.smithi102.stdout:pg_num: 64 2024-04-17T22:31:07.938 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.951 7fdc88d24700 1 -- 172.21.15.102:0/3602196316 >> v1:172.21.15.102:6816/42360 conn(0x7fdc600145a0 legacy=0x7fdc60016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:07.938 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.951 7fdc88d24700 1 -- 172.21.15.102:0/3602196316 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fdc840f2230 msgr2=0x7fdc841ad930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:07.938 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.951 7fdc88d24700 1 --2- 172.21.15.102:0/3602196316 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fdc840f2230 0x7fdc841ad930 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 rx=0x7fdc841a5150 tx=0x7fdc7400ea50).stop 2024-04-17T22:31:07.938 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.951 7fdc88d24700 1 -- 172.21.15.102:0/3602196316 shutdown_connections 2024-04-17T22:31:07.938 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.952 7fdc88d24700 1 --2- 172.21.15.102:0/3602196316 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fdc841adf60 0x7fdc841bba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:07.938 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.952 7fdc88d24700 1 --2- 172.21.15.102:0/3602196316 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdc840f17b0 0x7fdc840f1c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:07.938 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.952 7fdc88d24700 1 --2- 172.21.15.102:0/3602196316 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fdc840f2230 0x7fdc841ad930 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:07.938 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.952 7fdc88d24700 1 -- 172.21.15.102:0/3602196316 >> 172.21.15.102:0/3602196316 conn(0x7fdc840c6af0 msgr2=0x7fdc840d65c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:07.938 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.952 7fdc88d24700 1 -- 172.21.15.102:0/3602196316 shutdown_connections 2024-04-17T22:31:07.938 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:07.952 7fdc88d24700 1 -- 172.21.15.102:0/3602196316 wait complete. 2024-04-17T22:31:07.953 INFO:tasks.cephfs_test_runner:Starting test: test_newops_getvxattr (tasks.cephfs.test_newops.TestNewOps) For nautilus it will crash the MDSs when receive unknown OPs, as a workaround 2024-04-17T22:31:07.953 INFO:tasks.cephfs_test_runner:test_newops_getvxattr (tasks.cephfs.test_newops.TestNewOps) 2024-04-17T22:31:07.954 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T22:31:08.081 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.092 7f7400ac2700 1 Processor -- start 2024-04-17T22:31:08.081 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.093 7f7400ac2700 1 -- start start 2024-04-17T22:31:08.081 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.093 7f7400ac2700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f73fc0f57d0 0x7f73fc10afa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:08.081 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.093 7f7400ac2700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f73fc10b5d0 0x7f73fc19fa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:08.081 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.093 7f7400ac2700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f73fc10ba20 0x7f73fc1a00b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:08.081 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.093 7f7400ac2700 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73fc10be70 con 0x7f73fc10ba20 2024-04-17T22:31:08.081 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.093 7f7400ac2700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73fc0fa300 con 0x7f73fc10b5d0 2024-04-17T22:31:08.081 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.093 7f7400ac2700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f73fc103ad0 con 0x7f73fc0f57d0 2024-04-17T22:31:08.082 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.093 7f73f904f700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f73fc10b5d0 0x7f73fc19fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:08.082 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.093 7f73fa051700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f73fc10ba20 0x7f73fc1a00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:08.082 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.093 7f73f904f700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f73fc10b5d0 0x7f73fc19fa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:52066/0 (socket says 172.21.15.102:52066) 2024-04-17T22:31:08.082 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.093 7f73f904f700 1 -- 172.21.15.102:0/3060683993 learned_addr learned my addr 172.21.15.102:0/3060683993 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:31:08.082 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.093 7f73f9850700 1 --2- 172.21.15.102:0/3060683993 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f73fc0f57d0 0x7f73fc10afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:08.082 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.093 7f73f904f700 1 -- 172.21.15.102:0/3060683993 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f73fc0f57d0 msgr2=0x7f73fc10afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:31:08.082 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.093 7f73f904f700 1 --2- 172.21.15.102:0/3060683993 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f73fc0f57d0 0x7f73fc10afa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:08.082 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.093 7f73f904f700 1 -- 172.21.15.102:0/3060683993 >> v2:172.21.15.102:3300/0 conn(0x7f73fc10ba20 msgr2=0x7f73fc1a00b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:31:08.082 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.093 7f73f904f700 1 --2- 172.21.15.102:0/3060683993 >> v2:172.21.15.102:3300/0 conn(0x7f73fc10ba20 0x7f73fc1a00b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:08.082 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.093 7f73f904f700 1 -- 172.21.15.102:0/3060683993 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73fc1a06e0 con 0x7f73fc10b5d0 2024-04-17T22:31:08.082 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.093 7f73fa051700 1 --2- 172.21.15.102:0/3060683993 >> v2:172.21.15.102:3300/0 conn(0x7f73fc10ba20 0x7f73fc1a00b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:08.082 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.094 7f73f904f700 1 --2- 172.21.15.102:0/3060683993 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f73fc10b5d0 0x7f73fc19fa80 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 rx=0x7f73ec002f70 tx=0x7f73ec00bc80).ready entity=mon.1 client_cookie=15d207d1c8b710bd server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:08.082 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.094 7f73f884e700 1 -- 172.21.15.102:0/3060683993 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f73fc1a06e0 con 0x7f73fc10b5d0 2024-04-17T22:31:08.082 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.094 7f73f884e700 1 -- 172.21.15.102:0/3060683993 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73ec00dc00 con 0x7f73fc10b5d0 2024-04-17T22:31:08.082 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.094 7f73f884e700 1 -- 172.21.15.102:0/3060683993 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f73ec010470 con 0x7f73fc10b5d0 2024-04-17T22:31:08.083 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.094 7f7400ac2700 1 -- 172.21.15.102:0/3060683993 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f73fc10b5d0 msgr2=0x7f73fc19fa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:08.083 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.094 7f7400ac2700 1 --2- 172.21.15.102:0/3060683993 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f73fc10b5d0 0x7f73fc19fa80 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 rx=0x7f73ec002f70 tx=0x7f73ec00bc80).stop 2024-04-17T22:31:08.083 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.094 7f7400ac2700 1 -- 172.21.15.102:0/3060683993 shutdown_connections 2024-04-17T22:31:08.083 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.094 7f7400ac2700 1 --2- 172.21.15.102:0/3060683993 >> v2:172.21.15.102:3300/0 conn(0x7f73fc10ba20 0x7f73fc1a00b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:08.083 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.094 7f7400ac2700 1 --2- 172.21.15.102:0/3060683993 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f73fc0f57d0 0x7f73fc10afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:08.083 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.094 7f7400ac2700 1 --2- 172.21.15.102:0/3060683993 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f73fc10b5d0 0x7f73fc19fa80 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:08.083 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.094 7f7400ac2700 1 -- 172.21.15.102:0/3060683993 >> 172.21.15.102:0/3060683993 conn(0x7f73fc113830 msgr2=0x7f73fc0f4ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:08.083 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.094 7f7400ac2700 1 -- 172.21.15.102:0/3060683993 shutdown_connections 2024-04-17T22:31:08.083 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.094 7f7400ac2700 1 -- 172.21.15.102:0/3060683993 wait complete. 2024-04-17T22:31:08.103 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.117 7f7400ac2700 1 Processor -- start 2024-04-17T22:31:08.103 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.117 7f7400ac2700 1 -- start start 2024-04-17T22:31:08.105 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.117 7f7400ac2700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f73fc1acfd0 0x7f73fc1ad420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:08.105 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.117 7f7400ac2700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f73fc1ada50 0x7f73fc0d6220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:08.105 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.117 7f7400ac2700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f73fc0d6850 0x7f73fc0d6ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:08.105 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.117 7f7400ac2700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73fc0d5790 con 0x7f73fc1ada50 2024-04-17T22:31:08.105 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.117 7f7400ac2700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73fc0d4bd0 con 0x7f73fc1acfd0 2024-04-17T22:31:08.105 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.117 7f7400ac2700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f73fc0d53e0 con 0x7f73fc0d6850 2024-04-17T22:31:08.105 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.117 7f73f9850700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f73fc1acfd0 0x7f73fc1ad420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:08.106 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.117 7f73fa051700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f73fc0d6850 0x7f73fc0d6ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:08.106 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.117 7f73f904f700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f73fc1ada50 0x7f73fc0d6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:08.106 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.117 7f73f9850700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f73fc1acfd0 0x7f73fc1ad420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:52074/0 (socket says 172.21.15.102:52074) 2024-04-17T22:31:08.106 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.117 7f73f9850700 1 -- 172.21.15.102:0/2387771192 learned_addr learned my addr 172.21.15.102:0/2387771192 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:31:08.106 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.117 7f73f904f700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f73fc1ada50 0x7f73fc0d6220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:54138/0 (socket says 172.21.15.102:54138) 2024-04-17T22:31:08.106 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.117 7f73fa051700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f73fc0d6850 0x7f73fc0d6ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:54906/0 (socket says 172.21.15.102:54906) 2024-04-17T22:31:08.106 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.118 7f73f904f700 1 -- 172.21.15.102:0/2387771192 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f73fc0d6850 msgr2=0x7f73fc0d6ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:08.106 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.118 7f73f904f700 1 --2- 172.21.15.102:0/2387771192 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f73fc0d6850 0x7f73fc0d6ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:08.106 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.118 7f73f904f700 1 -- 172.21.15.102:0/2387771192 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f73fc1acfd0 msgr2=0x7f73fc1ad420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:08.106 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.118 7f73f904f700 1 --2- 172.21.15.102:0/2387771192 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f73fc1acfd0 0x7f73fc1ad420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:08.106 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.118 7f73f904f700 1 -- 172.21.15.102:0/2387771192 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73fc1bb260 con 0x7f73fc1ada50 2024-04-17T22:31:08.106 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.118 7f73f9850700 1 --2- 172.21.15.102:0/2387771192 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f73fc1acfd0 0x7f73fc1ad420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:08.106 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.118 7f73f904f700 1 --2- 172.21.15.102:0/2387771192 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f73fc1ada50 0x7f73fc0d6220 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 rx=0x7f73fc0d5b60 tx=0x7f73ec0068c0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:08.107 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.118 7f73eaffd700 1 -- 172.21.15.102:0/2387771192 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f73fc1bb260 con 0x7f73fc1ada50 2024-04-17T22:31:08.107 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.118 7f73eaffd700 1 -- 172.21.15.102:0/2387771192 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73ec022410 con 0x7f73fc1ada50 2024-04-17T22:31:08.107 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.118 7f73eaffd700 1 -- 172.21.15.102:0/2387771192 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f73ec01e070 con 0x7f73fc1ada50 2024-04-17T22:31:08.107 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.118 7f7400ac2700 1 -- 172.21.15.102:0/2387771192 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73fc1bb460 con 0x7f73fc1ada50 2024-04-17T22:31:08.107 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.118 7f7400ac2700 1 -- 172.21.15.102:0/2387771192 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73fc1bb840 con 0x7f73fc1ada50 2024-04-17T22:31:08.107 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.118 7f73eaffd700 1 -- 172.21.15.102:0/2387771192 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7f73ec02a070 con 0x7f73fc1ada50 2024-04-17T22:31:08.107 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.119 7f73eaffd700 1 -- 172.21.15.102:0/2387771192 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3194+0+0 (secure 0 0 0) 0x7f73ec001450 con 0x7f73fc1ada50 2024-04-17T22:31:08.107 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.119 7f7400ac2700 1 -- 172.21.15.102:0/2387771192 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73fc0f0790 con 0x7f73fc1ada50 2024-04-17T22:31:08.108 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.122 7f73eaffd700 1 -- 172.21.15.102:0/2387771192 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+110455 (secure 0 0 0) 0x7f73ec010bd0 con 0x7f73fc1ada50 2024-04-17T22:31:08.213 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.227 7f7400ac2700 1 -- 172.21.15.102:0/2387771192 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f73fc0abea0 con 0x7f73fc1ada50 2024-04-17T22:31:08.213 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.227 7f73eaffd700 1 -- 172.21.15.102:0/2387771192 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) v1 ==== 74+0+8764 (secure 0 0 0) 0x7f73ec00d800 con 0x7f73fc1ada50 2024-04-17T22:31:08.214 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-17T22:31:08.214 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":17,"fsid":"cb63e789-091d-49b1-b120-55f9db526871","created":"2024-04-17 22:28:48.209635","modified":"2024-04-17 22:29:23.257387","last_up_change":"2024-04-17 22:28:54.236215","last_in_change":"2024-04-17 22:28:54.236215","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":4,"require_min_compat_client":"nautilus","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-04-17 22:28:57.478545","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-17 22:29:06.144708","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-17 22:29:06.748924","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3351ef85-ade4-4b90-a8f4-213f4e00ba01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6812","nonce":42955}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6813","nonce":42955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6815","nonce":42955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6814","nonce":42955}]},"public_addr":"172.21.15.102:6812/42955","cluster_addr":"172.21.15.102:6813/42955","heartbeat_back_addr":"172.21.15.102:6815/42955","heartbeat_front_addr":"172.21.15.102:6814/42955","state":["exists","up"]},{"osd":1,"uuid":"04f67eac-f36b-48f5-982d-4cfc776596d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6808","nonce":42958}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6809","nonce":42958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6811","nonce":42958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6810","nonce":42958}]},"public_addr":"172.21.15.102:6808/42958","cluster_addr":"172.21.15.102:6809/42958","heartbeat_back_addr":"172.21.15.102:6811/42958","heartbeat_front_addr":"172.21.15.102:6810/42958","state":["exists","up"]},{"osd":2,"uuid":"d524204b-70b2-40a8-8f73-f9e7d44318af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6804","nonce":42954}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6805","nonce":42954}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6807","nonce":42954}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6806","nonce":42954}]},"public_addr":"172.21.15.102:6804/42954","cluster_addr":"172.21.15.102:6805/42954","heartbeat_back_addr":"172.21.15.102:6807/42954","heartbeat_front_addr":"172.21.15.102:6806/42954","state":["exists","up"]},{"osd":3,"uuid":"4fc9d20c-1ad0-484f-aafa-2c9a7574245b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6800","nonce":42953}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6801","nonce":42953}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6803","nonce":42953}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6802","nonce":42953}]},"public_addr":"172.21.15.102:6800/42953","cluster_addr":"172.21.15.102:6801/42953","heartbeat_back_addr":"172.21.15.102:6803/42953","heartbeat_front_addr":"172.21.15.102:6802/42953","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2024-04-17T22:31:08.215 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.228 7f7400ac2700 1 -- 172.21.15.102:0/2387771192 >> v1:172.21.15.102:6816/42360 conn(0x7f73d80145a0 legacy=0x7f73d8016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:08.215 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.228 7f7400ac2700 1 -- 172.21.15.102:0/2387771192 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f73fc1ada50 msgr2=0x7f73fc0d6220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:08.215 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.228 7f7400ac2700 1 --2- 172.21.15.102:0/2387771192 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f73fc1ada50 0x7f73fc0d6220 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 rx=0x7f73fc0d5b60 tx=0x7f73ec0068c0).stop 2024-04-17T22:31:08.215 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.229 7f7400ac2700 1 -- 172.21.15.102:0/2387771192 shutdown_connections 2024-04-17T22:31:08.215 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.229 7f7400ac2700 1 --2- 172.21.15.102:0/2387771192 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f73fc0d6850 0x7f73fc0d6ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:08.215 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.229 7f7400ac2700 1 --2- 172.21.15.102:0/2387771192 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f73fc1acfd0 0x7f73fc1ad420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:08.215 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.229 7f7400ac2700 1 --2- 172.21.15.102:0/2387771192 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f73fc1ada50 0x7f73fc0d6220 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:08.215 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.229 7f7400ac2700 1 -- 172.21.15.102:0/2387771192 >> 172.21.15.102:0/2387771192 conn(0x7f73fc0c3ea0 msgr2=0x7f73fc0c4650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:08.215 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.229 7f7400ac2700 1 -- 172.21.15.102:0/2387771192 shutdown_connections 2024-04-17T22:31:08.216 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.229 7f7400ac2700 1 -- 172.21.15.102:0/2387771192 wait complete. 2024-04-17T22:31:08.225 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2024-04-17 22:28:57.478545', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '12', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-17 22:29:06.144708', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-17 22:29:06.748924', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-17T22:31:08.226 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2024-04-17T22:31:08.355 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.367 7f33f73f2700 1 Processor -- start 2024-04-17T22:31:08.356 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.367 7f33f73f2700 1 -- start start 2024-04-17T22:31:08.356 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.368 7f33f73f2700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f33f00f5840 0x7f33f010b010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:08.356 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.368 7f33f73f2700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f33f010b640 0x7f33f019faf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:08.356 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.368 7f33f73f2700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f33f010ba90 0x7f33f01a0120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:08.356 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.368 7f33f73f2700 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33f010bee0 con 0x7f33f010ba90 2024-04-17T22:31:08.356 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.368 7f33f73f2700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33f00fa370 con 0x7f33f010b640 2024-04-17T22:31:08.356 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.368 7f33f73f2700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f33f0103b40 con 0x7f33f00f5840 2024-04-17T22:31:08.356 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.368 7f33effff700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f33f00f5840 0x7f33f010b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:08.356 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.368 7f33f49ee700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f33f010ba90 0x7f33f01a0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:08.356 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.368 7f33e7fff700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f33f010b640 0x7f33f019faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:08.356 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.368 7f33effff700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f33f00f5840 0x7f33f010b010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:54918/0 (socket says 172.21.15.102:54918) 2024-04-17T22:31:08.356 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.368 7f33f49ee700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f33f010ba90 0x7f33f01a0120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:54140/0 (socket says 172.21.15.102:54140) 2024-04-17T22:31:08.356 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.368 7f33e7fff700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f33f010b640 0x7f33f019faf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:52088/0 (socket says 172.21.15.102:52088) 2024-04-17T22:31:08.356 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.368 7f33effff700 1 -- 172.21.15.102:0/3530749195 learned_addr learned my addr 172.21.15.102:0/3530749195 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:31:08.357 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.368 7f33effff700 1 -- 172.21.15.102:0/3530749195 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f33f010b640 msgr2=0x7f33f019faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:31:08.357 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.368 7f33effff700 1 --2- 172.21.15.102:0/3530749195 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f33f010b640 0x7f33f019faf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:08.357 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.368 7f33effff700 1 -- 172.21.15.102:0/3530749195 >> v2:172.21.15.102:3300/0 conn(0x7f33f010ba90 msgr2=0x7f33f01a0120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:31:08.357 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.368 7f33effff700 1 --2- 172.21.15.102:0/3530749195 >> v2:172.21.15.102:3300/0 conn(0x7f33f010ba90 0x7f33f01a0120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:08.357 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.368 7f33effff700 1 -- 172.21.15.102:0/3530749195 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33f01a0750 con 0x7f33f00f5840 2024-04-17T22:31:08.357 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.368 7f33e7fff700 1 --2- 172.21.15.102:0/3530749195 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f33f010b640 0x7f33f019faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:08.357 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.368 7f33f49ee700 1 --2- 172.21.15.102:0/3530749195 >> v2:172.21.15.102:3300/0 conn(0x7f33f010ba90 0x7f33f01a0120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:08.357 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.368 7f33effff700 1 --2- 172.21.15.102:0/3530749195 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f33f00f5840 0x7f33f010b010 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 rx=0x7f33f00fa520 tx=0x7f33e0009a00).ready entity=mon.2 client_cookie=bb2120968b433c24 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:08.357 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.368 7f33ef7fe700 1 -- 172.21.15.102:0/3530749195 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f33f01a0750 con 0x7f33f00f5840 2024-04-17T22:31:08.357 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.368 7f33ef7fe700 1 -- 172.21.15.102:0/3530749195 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33e000f510 con 0x7f33f00f5840 2024-04-17T22:31:08.357 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.368 7f33ef7fe700 1 -- 172.21.15.102:0/3530749195 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f33e000f6f0 con 0x7f33f00f5840 2024-04-17T22:31:08.357 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.368 7f33f73f2700 1 -- 172.21.15.102:0/3530749195 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f33f00f5840 msgr2=0x7f33f010b010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:08.357 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.368 7f33f73f2700 1 --2- 172.21.15.102:0/3530749195 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f33f00f5840 0x7f33f010b010 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 rx=0x7f33f00fa520 tx=0x7f33e0009a00).stop 2024-04-17T22:31:08.357 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.369 7f33f73f2700 1 -- 172.21.15.102:0/3530749195 shutdown_connections 2024-04-17T22:31:08.357 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.369 7f33f73f2700 1 --2- 172.21.15.102:0/3530749195 >> v2:172.21.15.102:3300/0 conn(0x7f33f010ba90 0x7f33f01a0120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:08.358 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.369 7f33f73f2700 1 --2- 172.21.15.102:0/3530749195 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f33f00f5840 0x7f33f010b010 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:08.358 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.369 7f33f73f2700 1 --2- 172.21.15.102:0/3530749195 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f33f010b640 0x7f33f019faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:08.358 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.369 7f33f73f2700 1 -- 172.21.15.102:0/3530749195 >> 172.21.15.102:0/3530749195 conn(0x7f33f01138a0 msgr2=0x7f33f00f4f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:08.358 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.369 7f33f73f2700 1 -- 172.21.15.102:0/3530749195 shutdown_connections 2024-04-17T22:31:08.358 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.369 7f33f73f2700 1 -- 172.21.15.102:0/3530749195 wait complete. 2024-04-17T22:31:08.379 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.391 7f33f73f2700 1 Processor -- start 2024-04-17T22:31:08.380 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.391 7f33f73f2700 1 -- start start 2024-04-17T22:31:08.380 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.392 7f33f73f2700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f33f00cc570 0x7f33f00cc9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:08.380 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.392 7f33f73f2700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f33f00ccff0 0x7f33f01afeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:08.380 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.392 7f33f73f2700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f33f01b04e0 0x7f33f01b0930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:08.380 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.392 7f33f73f2700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33f00dd320 con 0x7f33f00ccff0 2024-04-17T22:31:08.380 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.392 7f33f73f2700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33f00d71c0 con 0x7f33f01b04e0 2024-04-17T22:31:08.380 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.392 7f33f73f2700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f33f00dcf40 con 0x7f33f00cc570 2024-04-17T22:31:08.380 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.392 7f33e7fff700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f33f00ccff0 0x7f33f01afeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:08.380 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.392 7f33effff700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f33f00cc570 0x7f33f00cc9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:08.380 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.392 7f33f49ee700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f33f01b04e0 0x7f33f01b0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:08.380 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.392 7f33effff700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f33f00cc570 0x7f33f00cc9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:54922/0 (socket says 172.21.15.102:54922) 2024-04-17T22:31:08.380 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.392 7f33e7fff700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f33f00ccff0 0x7f33f01afeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:54142/0 (socket says 172.21.15.102:54142) 2024-04-17T22:31:08.380 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.392 7f33effff700 1 -- 172.21.15.102:0/1741637663 learned_addr learned my addr 172.21.15.102:0/1741637663 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:31:08.380 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.392 7f33e7fff700 1 -- 172.21.15.102:0/1741637663 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f33f00cc570 msgr2=0x7f33f00cc9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:08.381 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.392 7f33e7fff700 1 --2- 172.21.15.102:0/1741637663 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f33f00cc570 0x7f33f00cc9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:08.381 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.392 7f33e7fff700 1 -- 172.21.15.102:0/1741637663 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f33f01b04e0 msgr2=0x7f33f01b0930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:08.381 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.392 7f33e7fff700 1 --2- 172.21.15.102:0/1741637663 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f33f01b04e0 0x7f33f01b0930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:08.381 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.392 7f33e7fff700 1 -- 172.21.15.102:0/1741637663 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33f01b7620 con 0x7f33f00ccff0 2024-04-17T22:31:08.381 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.392 7f33f49ee700 1 --2- 172.21.15.102:0/1741637663 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f33f01b04e0 0x7f33f01b0930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:08.381 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.392 7f33e7fff700 1 --2- 172.21.15.102:0/1741637663 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f33f00ccff0 0x7f33f01afeb0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 rx=0x7f33d800e000 tx=0x7f33d800df30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:08.381 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.392 7f33edffb700 1 -- 172.21.15.102:0/1741637663 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f33f01b7620 con 0x7f33f00ccff0 2024-04-17T22:31:08.381 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.392 7f33edffb700 1 -- 172.21.15.102:0/1741637663 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33d800ed90 con 0x7f33f00ccff0 2024-04-17T22:31:08.381 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.393 7f33f73f2700 1 -- 172.21.15.102:0/1741637663 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33f01b7820 con 0x7f33f00ccff0 2024-04-17T22:31:08.381 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.393 7f33edffb700 1 -- 172.21.15.102:0/1741637663 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f33d801a660 con 0x7f33f00ccff0 2024-04-17T22:31:08.381 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.393 7f33f73f2700 1 -- 172.21.15.102:0/1741637663 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33f01b7c00 con 0x7f33f00ccff0 2024-04-17T22:31:08.381 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.393 7f33f73f2700 1 -- 172.21.15.102:0/1741637663 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33f00f0800 con 0x7f33f00ccff0 2024-04-17T22:31:08.381 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.394 7f33edffb700 1 -- 172.21.15.102:0/1741637663 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7f33d8022070 con 0x7f33f00ccff0 2024-04-17T22:31:08.382 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.394 7f33edffb700 1 -- 172.21.15.102:0/1741637663 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3194+0+0 (secure 0 0 0) 0x7f33d8012d30 con 0x7f33f00ccff0 2024-04-17T22:31:08.383 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.397 7f33edffb700 1 -- 172.21.15.102:0/1741637663 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+110455 (secure 0 0 0) 0x7f33d800e990 con 0x7f33f00ccff0 2024-04-17T22:31:08.488 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.502 7f33f73f2700 1 -- 172.21.15.102:0/1741637663 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f33f00c5f60 con 0x7f33f00ccff0 2024-04-17T22:31:08.488 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.502 7f33edffb700 1 -- 172.21.15.102:0/1741637663 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v17) v1 ==== 92+0+10 (secure 0 0 0) 0x7f33d8000c30 con 0x7f33f00ccff0 2024-04-17T22:31:08.488 INFO:teuthology.orchestra.run.smithi102.stdout:pg_num: 8 2024-04-17T22:31:08.490 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.503 7f33f73f2700 1 -- 172.21.15.102:0/1741637663 >> v1:172.21.15.102:6816/42360 conn(0x7f33d40145a0 legacy=0x7f33d4016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:08.490 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.503 7f33f73f2700 1 -- 172.21.15.102:0/1741637663 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f33f00ccff0 msgr2=0x7f33f01afeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:08.490 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.503 7f33f73f2700 1 --2- 172.21.15.102:0/1741637663 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f33f00ccff0 0x7f33f01afeb0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 rx=0x7f33d800e000 tx=0x7f33d800df30).stop 2024-04-17T22:31:08.490 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.503 7f33f73f2700 1 -- 172.21.15.102:0/1741637663 shutdown_connections 2024-04-17T22:31:08.490 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.503 7f33f73f2700 1 --2- 172.21.15.102:0/1741637663 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f33f01b04e0 0x7f33f01b0930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:08.490 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.503 7f33f73f2700 1 --2- 172.21.15.102:0/1741637663 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f33f00cc570 0x7f33f00cc9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:08.490 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.503 7f33f73f2700 1 --2- 172.21.15.102:0/1741637663 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f33f00ccff0 0x7f33f01afeb0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:08.490 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.503 7f33f73f2700 1 -- 172.21.15.102:0/1741637663 >> 172.21.15.102:0/1741637663 conn(0x7f33f00c5b10 msgr2=0x7f33f01a42d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:08.490 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.503 7f33f73f2700 1 -- 172.21.15.102:0/1741637663 shutdown_connections 2024-04-17T22:31:08.491 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.503 7f33f73f2700 1 -- 172.21.15.102:0/1741637663 wait complete. 2024-04-17T22:31:08.500 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-17T22:31:08.628 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.640 7f2f15f68700 1 Processor -- start 2024-04-17T22:31:08.632 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.640 7f2f15f68700 1 -- start start 2024-04-17T22:31:08.632 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.641 7f2f15f68700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f2f1010b810 0x7f2f101a52c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:08.632 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.641 7f2f15f68700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f2f1010bc60 0x7f2f101a7900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:08.632 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.641 7f2f15f68700 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f10103b40 con 0x7f2f100f5840 2024-04-17T22:31:08.632 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.641 7f2f15f68700 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f100fa370 con 0x7f2f1010b810 2024-04-17T22:31:08.632 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.641 7f2f15f68700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2f101a0b00 con 0x7f2f1010bc60 2024-04-17T22:31:08.632 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.641 7f2f0f4ee700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f2f1010bc60 0x7f2f101a7900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:08.632 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.641 7f2f0e4ec700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f2f1010b810 0x7f2f101a52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:08.632 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.641 7f2f0f4ee700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f2f1010bc60 0x7f2f101a7900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:54938/0 (socket says 172.21.15.102:54938) 2024-04-17T22:31:08.632 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.641 7f2f0eced700 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f2f100f5840 0x7f2f1010b010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:38740/0 (socket says 172.21.15.102:38740) 2024-04-17T22:31:08.632 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.641 7f2f0e4ec700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f2f1010b810 0x7f2f101a52c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:54156/0 (socket says 172.21.15.102:54156) 2024-04-17T22:31:08.632 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.641 7f2f0f4ee700 1 -- 172.21.15.102:0/2555572737 learned_addr learned my addr 172.21.15.102:0/2555572737 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:31:08.632 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.641 7f2f0e4ec700 1 -- 172.21.15.102:0/2555572737 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f2f1010bc60 msgr2=0x7f2f101a7900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:31:08.633 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.641 7f2f0e4ec700 1 --2- 172.21.15.102:0/2555572737 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f2f1010bc60 0x7f2f101a7900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:08.633 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.641 7f2f0e4ec700 1 -- 172.21.15.102:0/2555572737 >> v1:172.21.15.102:6789/0 conn(0x7f2f100f5840 legacy=0x7f2f1010b010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:08.633 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.641 7f2f0e4ec700 1 -- 172.21.15.102:0/2555572737 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f101a8f40 con 0x7f2f1010b810 2024-04-17T22:31:08.633 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.641 7f2f0f4ee700 1 --2- 172.21.15.102:0/2555572737 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f2f1010bc60 0x7f2f101a7900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:08.633 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.641 7f2f0e4ec700 1 --2- 172.21.15.102:0/2555572737 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2f1010b810 0x7f2f101a52c0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 rx=0x7f2f00002f70 tx=0x7f2f0000dc50).ready entity=mon.0 client_cookie=9ffc219244121b7f server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:08.633 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.642 7f2f0dceb700 1 -- 172.21.15.102:0/2555572737 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f2f101a8f40 con 0x7f2f1010b810 2024-04-17T22:31:08.633 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.642 7f2f0dceb700 1 -- 172.21.15.102:0/2555572737 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f0000bbe0 con 0x7f2f1010b810 2024-04-17T22:31:08.633 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.642 7f2f0dceb700 1 -- 172.21.15.102:0/2555572737 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f2f00015070 con 0x7f2f1010b810 2024-04-17T22:31:08.633 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.642 7f2f15f68700 1 -- 172.21.15.102:0/2555572737 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2f1010b810 msgr2=0x7f2f101a52c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:08.633 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.642 7f2f15f68700 1 --2- 172.21.15.102:0/2555572737 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2f1010b810 0x7f2f101a52c0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 rx=0x7f2f00002f70 tx=0x7f2f0000dc50).stop 2024-04-17T22:31:08.634 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.642 7f2f15f68700 1 -- 172.21.15.102:0/2555572737 shutdown_connections 2024-04-17T22:31:08.634 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.642 7f2f15f68700 1 --2- 172.21.15.102:0/2555572737 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f2f1010bc60 0x7f2f101a7900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:08.634 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.642 7f2f15f68700 1 --2- 172.21.15.102:0/2555572737 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2f1010b810 0x7f2f101a52c0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:08.634 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.642 7f2f15f68700 1 -- 172.21.15.102:0/2555572737 >> 172.21.15.102:0/2555572737 conn(0x7f2f101138a0 msgr2=0x7f2f100f4f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:08.634 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.642 7f2f15f68700 1 -- 172.21.15.102:0/2555572737 shutdown_connections 2024-04-17T22:31:08.634 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.642 7f2f15f68700 1 -- 172.21.15.102:0/2555572737 wait complete. 2024-04-17T22:31:08.653 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.665 7f2f15f68700 1 Processor -- start 2024-04-17T22:31:08.653 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.665 7f2f15f68700 1 -- start start 2024-04-17T22:31:08.653 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.665 7f2f15f68700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2f100d5660 0x7f2f100d5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:08.653 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.665 7f2f15f68700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f2f100d60e0 0x7f2f101b0330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:08.653 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.665 7f2f15f68700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2f101b0960 0x7f2f101b0db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:08.653 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.665 7f2f15f68700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f100dcab0 con 0x7f2f101b0960 2024-04-17T22:31:08.653 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.665 7f2f15f68700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f100c3b00 con 0x7f2f100d5660 2024-04-17T22:31:08.653 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.665 7f2f15f68700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2f100dccf0 con 0x7f2f100d60e0 2024-04-17T22:31:08.653 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.665 7f2f0eced700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2f100d5660 0x7f2f100d5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:08.654 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.665 7f2f0f4ee700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2f101b0960 0x7f2f101b0db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:08.654 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.665 7f2f0eced700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2f100d5660 0x7f2f100d5ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:52098/0 (socket says 172.21.15.102:52098) 2024-04-17T22:31:08.654 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.665 7f2f0f4ee700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2f101b0960 0x7f2f101b0db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:54158/0 (socket says 172.21.15.102:54158) 2024-04-17T22:31:08.654 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.665 7f2f0eced700 1 -- 172.21.15.102:0/2243026620 learned_addr learned my addr 172.21.15.102:0/2243026620 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:31:08.654 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.665 7f2f0e4ec700 1 --2- 172.21.15.102:0/2243026620 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f2f100d60e0 0x7f2f101b0330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:08.654 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.666 7f2f0eced700 1 -- 172.21.15.102:0/2243026620 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f2f100d60e0 msgr2=0x7f2f101b0330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:08.654 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.666 7f2f0eced700 1 --2- 172.21.15.102:0/2243026620 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f2f100d60e0 0x7f2f101b0330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:08.654 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.666 7f2f0eced700 1 -- 172.21.15.102:0/2243026620 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2f101b0960 msgr2=0x7f2f101b0db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:08.654 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.666 7f2f0eced700 1 --2- 172.21.15.102:0/2243026620 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2f101b0960 0x7f2f101b0db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:08.654 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.666 7f2f0eced700 1 -- 172.21.15.102:0/2243026620 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f100dd350 con 0x7f2f100d5660 2024-04-17T22:31:08.654 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.666 7f2f0f4ee700 1 --2- 172.21.15.102:0/2243026620 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2f101b0960 0x7f2f101b0db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:08.654 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.666 7f2f0eced700 1 --2- 172.21.15.102:0/2243026620 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2f100d5660 0x7f2f100d5ab0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 rx=0x7f2f04000bc0 tx=0x7f2f0400c7e0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:08.654 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.666 7f2efffff700 1 -- 172.21.15.102:0/2243026620 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f2f100dd350 con 0x7f2f100d5660 2024-04-17T22:31:08.654 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.666 7f2efffff700 1 -- 172.21.15.102:0/2243026620 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f040072e0 con 0x7f2f100d5660 2024-04-17T22:31:08.654 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.666 7f2f15f68700 1 -- 172.21.15.102:0/2243026620 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f101b7a20 con 0x7f2f100d5660 2024-04-17T22:31:08.655 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.666 7f2efffff700 1 -- 172.21.15.102:0/2243026620 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f2f04005750 con 0x7f2f100d5660 2024-04-17T22:31:08.655 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.666 7f2f15f68700 1 -- 172.21.15.102:0/2243026620 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f101b7e00 con 0x7f2f100d5660 2024-04-17T22:31:08.655 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.666 7f2efffff700 1 -- 172.21.15.102:0/2243026620 <== mon.1 v2:172.21.15.102:3301/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7f2f04005950 con 0x7f2f100d5660 2024-04-17T22:31:08.655 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.667 7f2efffff700 1 -- 172.21.15.102:0/2243026620 <== mon.1 v2:172.21.15.102:3301/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3194+0+0 (secure 0 0 0) 0x7f2f04014a10 con 0x7f2f100d5660 2024-04-17T22:31:08.655 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.667 7f2f15f68700 1 -- 172.21.15.102:0/2243026620 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f100dd550 con 0x7f2f100d5660 2024-04-17T22:31:08.656 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.670 7f2efffff700 1 -- 172.21.15.102:0/2243026620 <== mon.1 v2:172.21.15.102:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+110455 (secure 0 0 0) 0x7f2f040070d0 con 0x7f2f100d5660 2024-04-17T22:31:08.760 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.774 7f2f15f68700 1 -- 172.21.15.102:0/2243026620 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f2f100f0800 con 0x7f2f100d5660 2024-04-17T22:31:08.760 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.774 7f2efffff700 1 -- 172.21.15.102:0/2243026620 <== mon.1 v2:172.21.15.102:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v17) v1 ==== 104+0+11 (secure 0 0 0) 0x7f2f04000c30 con 0x7f2f100d5660 2024-04-17T22:31:08.760 INFO:teuthology.orchestra.run.smithi102.stdout:pg_num: 64 2024-04-17T22:31:08.762 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.775 7f2f15f68700 1 -- 172.21.15.102:0/2243026620 >> v1:172.21.15.102:6816/42360 conn(0x7f2eec0145a0 legacy=0x7f2eec016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:08.762 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.775 7f2f15f68700 1 -- 172.21.15.102:0/2243026620 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2f100d5660 msgr2=0x7f2f100d5ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:08.762 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.775 7f2f15f68700 1 --2- 172.21.15.102:0/2243026620 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2f100d5660 0x7f2f100d5ab0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 rx=0x7f2f04000bc0 tx=0x7f2f0400c7e0).stop 2024-04-17T22:31:08.762 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.775 7f2f15f68700 1 -- 172.21.15.102:0/2243026620 shutdown_connections 2024-04-17T22:31:08.762 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.775 7f2f15f68700 1 --2- 172.21.15.102:0/2243026620 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2f101b0960 0x7f2f101b0db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:08.762 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.775 7f2f15f68700 1 --2- 172.21.15.102:0/2243026620 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2f100d5660 0x7f2f100d5ab0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:08.762 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.775 7f2f15f68700 1 --2- 172.21.15.102:0/2243026620 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f2f100d60e0 0x7f2f101b0330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:08.762 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.775 7f2f15f68700 1 -- 172.21.15.102:0/2243026620 >> 172.21.15.102:0/2243026620 conn(0x7f2f100e0b00 msgr2=0x7f2f100ce980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:08.762 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.775 7f2f15f68700 1 -- 172.21.15.102:0/2243026620 shutdown_connections 2024-04-17T22:31:08.762 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.775 7f2f15f68700 1 -- 172.21.15.102:0/2243026620 wait complete. 2024-04-17T22:31:08.772 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-17T22:31:08.901 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.913 7fd59ee88700 1 Processor -- start 2024-04-17T22:31:08.901 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.913 7fd59ee88700 1 -- start start 2024-04-17T22:31:08.901 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.914 7fd59ee88700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd59810e240 0x7fd59810e690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:08.901 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.914 7fd59ee88700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd5980f1f30 0x7fd5981ab4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:08.901 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.914 7fd59ee88700 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5980f2380 con 0x7fd59810ecc0 2024-04-17T22:31:08.902 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.914 7fd59ee88700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5980f9f10 con 0x7fd59810e240 2024-04-17T22:31:08.902 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.914 7fd59ee88700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd5980f9030 con 0x7fd5980f1f30 2024-04-17T22:31:08.902 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.914 7fd5977fe700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd59810e240 0x7fd59810e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:08.902 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.914 7fd596ffd700 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7fd59810ecc0 0x7fd5980f1730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:38744/0 (socket says 172.21.15.102:38744) 2024-04-17T22:31:08.902 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.914 7fd596ffd700 1 -- 172.21.15.102:0/378210256 learned_addr learned my addr 172.21.15.102:0/378210256 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-17T22:31:08.902 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.914 7fd597fff700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd5980f1f30 0x7fd5981ab4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:08.902 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.914 7fd5977fe700 1 --2- 172.21.15.102:0/378210256 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd59810e240 0x7fd59810e690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:52102/0 (socket says 172.21.15.102:52102) 2024-04-17T22:31:08.902 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.914 7fd5977fe700 1 -- 172.21.15.102:0/378210256 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd5980f1f30 msgr2=0x7fd5981ab4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:31:08.902 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.914 7fd5977fe700 1 --2- 172.21.15.102:0/378210256 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd5980f1f30 0x7fd5981ab4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:08.902 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.914 7fd5967fc700 1 -- 172.21.15.102:0/378210256 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1339943501 0 0) 0x7fd580001110 con 0x7fd59810ecc0 2024-04-17T22:31:08.902 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.914 7fd5977fe700 1 -- 172.21.15.102:0/378210256 >> v1:172.21.15.102:6789/0 conn(0x7fd59810ecc0 legacy=0x7fd5980f1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:08.902 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.914 7fd5977fe700 1 -- 172.21.15.102:0/378210256 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5980f25a0 con 0x7fd59810e240 2024-04-17T22:31:08.902 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.914 7fd597fff700 1 --2- 172.21.15.102:0/378210256 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd5980f1f30 0x7fd5981ab4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:08.902 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.914 7fd5977fe700 1 --2- 172.21.15.102:0/378210256 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd59810e240 0x7fd59810e690 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0x7fd5980f46a0 tx=0x7fd590009d70).ready entity=mon.1 client_cookie=505c4154f01aba99 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:08.903 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.914 7fd5967fc700 1 -- 172.21.15.102:0/378210256 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd5980f25a0 con 0x7fd59810e240 2024-04-17T22:31:08.903 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.914 7fd5967fc700 1 -- 172.21.15.102:0/378210256 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd59000b7c0 con 0x7fd59810e240 2024-04-17T22:31:08.903 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.914 7fd5967fc700 1 -- 172.21.15.102:0/378210256 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd59000b9a0 con 0x7fd59810e240 2024-04-17T22:31:08.903 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.914 7fd59ee88700 1 -- 172.21.15.102:0/378210256 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd59810e240 msgr2=0x7fd59810e690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:08.903 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.914 7fd59ee88700 1 --2- 172.21.15.102:0/378210256 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd59810e240 0x7fd59810e690 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0x7fd5980f46a0 tx=0x7fd590009d70).stop 2024-04-17T22:31:08.903 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.915 7fd59ee88700 1 -- 172.21.15.102:0/378210256 shutdown_connections 2024-04-17T22:31:08.903 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.915 7fd59ee88700 1 --2- 172.21.15.102:0/378210256 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd5980f1f30 0x7fd5981ab4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:08.903 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.915 7fd59ee88700 1 --2- 172.21.15.102:0/378210256 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd59810e240 0x7fd59810e690 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:08.903 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.915 7fd59ee88700 1 -- 172.21.15.102:0/378210256 >> 172.21.15.102:0/378210256 conn(0x7fd5981138a0 msgr2=0x7fd5980f4f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:08.903 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.915 7fd59ee88700 1 -- 172.21.15.102:0/378210256 shutdown_connections 2024-04-17T22:31:08.903 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.915 7fd59ee88700 1 -- 172.21.15.102:0/378210256 wait complete. 2024-04-17T22:31:08.924 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.938 7fd59ee88700 1 Processor -- start 2024-04-17T22:31:08.924 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.938 7fd59ee88700 1 -- start start 2024-04-17T22:31:08.926 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.938 7fd59ee88700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd5981ad9e0 0x7fd5981ade30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:08.926 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.938 7fd59ee88700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd5981ae460 0x7fd5981b4390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:08.927 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.938 7fd59ee88700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd5981b49c0 0x7fd5981b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:08.927 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.938 7fd59ee88700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5980f0520 con 0x7fd5981ae460 2024-04-17T22:31:08.927 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.938 7fd59ee88700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5980fa1b0 con 0x7fd5981b49c0 2024-04-17T22:31:08.927 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.938 7fd59ee88700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd5980d7aa0 con 0x7fd5981ad9e0 2024-04-17T22:31:08.927 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.938 7fd597fff700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd5981b49c0 0x7fd5981b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:08.927 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.938 7fd596ffd700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd5981ae460 0x7fd5981b4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:08.927 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.938 7fd5977fe700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd5981ad9e0 0x7fd5981ade30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:08.927 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.938 7fd597fff700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd5981b49c0 0x7fd5981b4e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:52116/0 (socket says 172.21.15.102:52116) 2024-04-17T22:31:08.927 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.938 7fd597fff700 1 -- 172.21.15.102:0/1260016018 learned_addr learned my addr 172.21.15.102:0/1260016018 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:31:08.927 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.939 7fd597fff700 1 -- 172.21.15.102:0/1260016018 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd5981ad9e0 msgr2=0x7fd5981ade30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:08.927 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.939 7fd597fff700 1 --2- 172.21.15.102:0/1260016018 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd5981ad9e0 0x7fd5981ade30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:08.927 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.939 7fd597fff700 1 -- 172.21.15.102:0/1260016018 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd5981ae460 msgr2=0x7fd5981b4390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:08.927 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.939 7fd597fff700 1 --2- 172.21.15.102:0/1260016018 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd5981ae460 0x7fd5981b4390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:08.927 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.939 7fd597fff700 1 -- 172.21.15.102:0/1260016018 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5980d7d60 con 0x7fd5981b49c0 2024-04-17T22:31:08.928 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.939 7fd596ffd700 1 --2- 172.21.15.102:0/1260016018 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd5981ae460 0x7fd5981b4390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:08.928 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.939 7fd5977fe700 1 --2- 172.21.15.102:0/1260016018 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd5981ad9e0 0x7fd5981ade30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:08.928 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.939 7fd597fff700 1 --2- 172.21.15.102:0/1260016018 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd5981b49c0 0x7fd5981b4e10 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 rx=0x7fd5980d7910 tx=0x7fd58800eca0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:08.928 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.939 7fd594ff9700 1 -- 172.21.15.102:0/1260016018 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd5980d7d60 con 0x7fd5981b49c0 2024-04-17T22:31:08.928 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.939 7fd594ff9700 1 -- 172.21.15.102:0/1260016018 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd58800c790 con 0x7fd5981b49c0 2024-04-17T22:31:08.928 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.939 7fd59ee88700 1 -- 172.21.15.102:0/1260016018 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5980d8020 con 0x7fd5981b49c0 2024-04-17T22:31:08.928 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.939 7fd594ff9700 1 -- 172.21.15.102:0/1260016018 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd58800f040 con 0x7fd5981b49c0 2024-04-17T22:31:08.928 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.939 7fd59ee88700 1 -- 172.21.15.102:0/1260016018 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5981bbd70 con 0x7fd5981b49c0 2024-04-17T22:31:08.928 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.939 7fd594ff9700 1 -- 172.21.15.102:0/1260016018 <== mon.1 v2:172.21.15.102:3301/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7fd588011070 con 0x7fd5981b49c0 2024-04-17T22:31:08.928 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.940 7fd594ff9700 1 -- 172.21.15.102:0/1260016018 <== mon.1 v2:172.21.15.102:3301/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3194+0+0 (secure 0 0 0) 0x7fd588010040 con 0x7fd5981b49c0 2024-04-17T22:31:08.928 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.940 7fd59ee88700 1 -- 172.21.15.102:0/1260016018 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5981aeab0 con 0x7fd5981b49c0 2024-04-17T22:31:08.929 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:08.943 7fd594ff9700 1 -- 172.21.15.102:0/1260016018 <== mon.1 v2:172.21.15.102:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+110455 (secure 0 0 0) 0x7fd58800ede0 con 0x7fd5981b49c0 2024-04-17T22:31:09.035 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:09.049 7fd59ee88700 1 -- 172.21.15.102:0/1260016018 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd5980a64e0 con 0x7fd5981b49c0 2024-04-17T22:31:09.035 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:09.049 7fd594ff9700 1 -- 172.21.15.102:0/1260016018 <== mon.1 v2:172.21.15.102:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v17) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd598114420 con 0x7fd5981b49c0 2024-04-17T22:31:09.035 INFO:teuthology.orchestra.run.smithi102.stdout:pg_num: 64 2024-04-17T22:31:09.037 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:09.050 7fd59ee88700 1 -- 172.21.15.102:0/1260016018 >> v1:172.21.15.102:6816/42360 conn(0x7fd57c0145a0 legacy=0x7fd57c016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:09.037 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:09.050 7fd59ee88700 1 -- 172.21.15.102:0/1260016018 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd5981b49c0 msgr2=0x7fd5981b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:09.037 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:09.051 7fd59ee88700 1 --2- 172.21.15.102:0/1260016018 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd5981b49c0 0x7fd5981b4e10 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 rx=0x7fd5980d7910 tx=0x7fd58800eca0).stop 2024-04-17T22:31:09.037 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:09.051 7fd59ee88700 1 -- 172.21.15.102:0/1260016018 shutdown_connections 2024-04-17T22:31:09.037 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:09.051 7fd59ee88700 1 --2- 172.21.15.102:0/1260016018 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd5981b49c0 0x7fd5981b4e10 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:09.037 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:09.051 7fd59ee88700 1 --2- 172.21.15.102:0/1260016018 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd5981ad9e0 0x7fd5981ade30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:09.037 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:09.051 7fd59ee88700 1 --2- 172.21.15.102:0/1260016018 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd5981ae460 0x7fd5981b4390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:09.037 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:09.051 7fd59ee88700 1 -- 172.21.15.102:0/1260016018 >> 172.21.15.102:0/1260016018 conn(0x7fd5980c6af0 msgr2=0x7fd5981a84e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:09.038 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:09.051 7fd59ee88700 1 -- 172.21.15.102:0/1260016018 shutdown_connections 2024-04-17T22:31:09.038 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:09.051 7fd59ee88700 1 -- 172.21.15.102:0/1260016018 wait complete. 2024-04-17T22:31:09.048 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_newops.TestNewOps.test_newops_getvxattr' 2024-04-17T22:31:09.175 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:09.187 7f57a2c09700 1 Processor -- start 2024-04-17T22:31:09.175 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:09.187 7f57a2c09700 1 -- start start 2024-04-17T22:31:09.175 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:09.187 7f57a2c09700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f579c10e1d0 0x7f579c10e620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:09.175 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:09.187 7f57a2c09700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f579c10ec50 0x7f579c0f16c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:09.175 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:09.187 7f57a2c09700 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f579c0f2140 con 0x7f579c0f1cf0 2024-04-17T22:31:09.175 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:09.187 7f57a2c09700 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f579c0f9ea0 con 0x7f579c10ec50 2024-04-17T22:31:09.175 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:09.187 7f57a2c09700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f579c0f8fc0 con 0x7f579c10e1d0 2024-04-17T22:31:09.175 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:09.187 7f579affd700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f579c10ec50 0x7f579c0f16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:09.175 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:09.187 7f579b7fe700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f579c10e1d0 0x7f579c10e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:09.176 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:09.187 7f579affd700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f579c10ec50 0x7f579c0f16c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:54172/0 (socket says 172.21.15.102:54172) 2024-04-17T22:31:09.176 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:09.187 7f579affd700 1 -- 172.21.15.102:0/12228076 learned_addr learned my addr 172.21.15.102:0/12228076 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:31:09.176 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:09.187 7f579b7fe700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f579c10e1d0 0x7f579c10e620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:52118/0 (socket says 172.21.15.102:52118) 2024-04-17T22:31:09.176 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:09.188 7f579affd700 1 -- 172.21.15.102:0/12228076 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f579c10e1d0 msgr2=0x7f579c10e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:31:09.176 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:09.188 7f579affd700 1 --2- 172.21.15.102:0/12228076 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f579c10e1d0 0x7f579c10e620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:09.176 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:09.188 7f579affd700 1 -- 172.21.15.102:0/12228076 >> v1:172.21.15.102:6789/0 conn(0x7f579c0f1cf0 legacy=0x7f579c1aa460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:09.176 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:09.188 7f579affd700 1 -- 172.21.15.102:0/12228076 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f579c0f2360 con 0x7f579c10ec50 2024-04-17T22:31:09.176 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:09.188 7f579a7fc700 1 -- 172.21.15.102:0/12228076 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1000245414 0 0) 0x7f5794001490 con 0x7f579c0f1cf0 2024-04-17T22:31:09.176 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:09.188 7f579b7fe700 1 --2- 172.21.15.102:0/12228076 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f579c10e1d0 0x7f579c10e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:09.176 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:09.188 7f579affd700 1 --2- 172.21.15.102:0/12228076 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f579c10ec50 0x7f579c0f16c0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 rx=0x7f579c103510 tx=0x7f578c009a80).ready entity=mon.0 client_cookie=7d9827742a2dce40 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:09.176 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:09.188 7f579a7fc700 1 -- 172.21.15.102:0/12228076 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f579c0f2360 con 0x7f579c10ec50 2024-04-17T22:31:09.176 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:09.188 7f579a7fc700 1 -- 172.21.15.102:0/12228076 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f578c00f510 con 0x7f579c10ec50 2024-04-17T22:31:09.176 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:09.188 7f579a7fc700 1 -- 172.21.15.102:0/12228076 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f578c00f740 con 0x7f579c10ec50 2024-04-17T22:31:09.176 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:09.188 7f57a2c09700 1 -- 172.21.15.102:0/12228076 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f579c10ec50 msgr2=0x7f579c0f16c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:09.176 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:09.188 7f57a2c09700 1 --2- 172.21.15.102:0/12228076 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f579c10ec50 0x7f579c0f16c0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 rx=0x7f579c103510 tx=0x7f578c009a80).stop 2024-04-17T22:31:09.177 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:09.189 7f57a2c09700 1 -- 172.21.15.102:0/12228076 shutdown_connections 2024-04-17T22:31:09.177 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:09.189 7f57a2c09700 1 --2- 172.21.15.102:0/12228076 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f579c10e1d0 0x7f579c10e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:09.177 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:09.189 7f57a2c09700 1 --2- 172.21.15.102:0/12228076 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f579c10ec50 0x7f579c0f16c0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:09.177 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:09.189 7f57a2c09700 1 -- 172.21.15.102:0/12228076 >> 172.21.15.102:0/12228076 conn(0x7f579c113830 msgr2=0x7f579c0f4ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:09.177 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:09.189 7f57a2c09700 1 -- 172.21.15.102:0/12228076 shutdown_connections 2024-04-17T22:31:09.177 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:09.189 7f57a2c09700 1 -- 172.21.15.102:0/12228076 wait complete. 2024-04-17T22:31:09.199 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:09.211 7f57a2c09700 1 Processor -- start 2024-04-17T22:31:09.199 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:09.211 7f57a2c09700 1 -- start start 2024-04-17T22:31:09.200 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:09.211 7f57a2c09700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f579c0dd8f0 0x7f579c0d6150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:09.200 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:09.211 7f57a2c09700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f579c0d6780 0x7f579c1b3fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:09.200 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:09.212 7f57a2c09700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f579c0d6bd0 0x7f579c1b45d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:09.200 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:09.212 7f57a2c09700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f579c113a40 con 0x7f579c0dd8f0 2024-04-17T22:31:09.200 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:09.212 7f57a2c09700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f579c0fa140 con 0x7f579c0d6bd0 2024-04-17T22:31:09.200 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:09.212 7f57a2c09700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f579c0dcf70 con 0x7f579c0d6780 2024-04-17T22:31:09.200 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:09.212 7f579b7fe700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f579c0dd8f0 0x7f579c0d6150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:09.200 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:09.212 7f579affd700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f579c0d6780 0x7f579c1b3fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:09.200 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:09.212 7f579bfff700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f579c0d6bd0 0x7f579c1b45d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:09.200 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:09.212 7f579affd700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f579c0d6780 0x7f579c1b3fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:54960/0 (socket says 172.21.15.102:54960) 2024-04-17T22:31:09.200 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:09.212 7f579b7fe700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f579c0dd8f0 0x7f579c0d6150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:54186/0 (socket says 172.21.15.102:54186) 2024-04-17T22:31:09.200 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:09.212 7f579affd700 1 -- 172.21.15.102:0/1039031922 learned_addr learned my addr 172.21.15.102:0/1039031922 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:31:09.200 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:09.212 7f579affd700 1 -- 172.21.15.102:0/1039031922 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f579c0d6bd0 msgr2=0x7f579c1b45d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:09.200 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:09.212 7f579affd700 1 --2- 172.21.15.102:0/1039031922 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f579c0d6bd0 0x7f579c1b45d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:09.201 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:09.212 7f579affd700 1 -- 172.21.15.102:0/1039031922 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f579c0dd8f0 msgr2=0x7f579c0d6150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:09.201 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:09.212 7f579affd700 1 --2- 172.21.15.102:0/1039031922 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f579c0dd8f0 0x7f579c0d6150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:09.201 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:09.212 7f579affd700 1 -- 172.21.15.102:0/1039031922 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f579c1b4c00 con 0x7f579c0d6780 2024-04-17T22:31:09.201 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:09.212 7f579affd700 1 --2- 172.21.15.102:0/1039031922 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f579c0d6780 0x7f579c1b3fa0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 rx=0x7f578c0030b0 tx=0x7f578c0068c0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:09.201 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:09.212 7f5798ff9700 1 -- 172.21.15.102:0/1039031922 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f579c1b4c00 con 0x7f579c0d6780 2024-04-17T22:31:09.201 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:09.212 7f5798ff9700 1 -- 172.21.15.102:0/1039031922 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f578c011710 con 0x7f579c0d6780 2024-04-17T22:31:09.201 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:09.212 7f57a2c09700 1 -- 172.21.15.102:0/1039031922 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f579c1b4ec0 con 0x7f579c0d6780 2024-04-17T22:31:09.201 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:09.212 7f5798ff9700 1 -- 172.21.15.102:0/1039031922 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f578c011ae0 con 0x7f579c0d6780 2024-04-17T22:31:09.201 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:09.212 7f57a2c09700 1 -- 172.21.15.102:0/1039031922 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f579c1bb710 con 0x7f579c0d6780 2024-04-17T22:31:09.201 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:09.213 7f5798ff9700 1 -- 172.21.15.102:0/1039031922 <== mon.2 v2:172.21.15.102:3302/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7f578c019030 con 0x7f579c0d6780 2024-04-17T22:31:09.201 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:09.213 7f5798ff9700 1 -- 172.21.15.102:0/1039031922 <== mon.2 v2:172.21.15.102:3302/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3194+0+0 (secure 0 0 0) 0x7f578c018040 con 0x7f579c0d6780 2024-04-17T22:31:09.201 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:09.213 7f57a2c09700 1 -- 172.21.15.102:0/1039031922 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f579c0f0790 con 0x7f579c0d6780 2024-04-17T22:31:09.202 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:09.216 7f5798ff9700 1 -- 172.21.15.102:0/1039031922 <== mon.2 v2:172.21.15.102:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+110455 (secure 0 0 0) 0x7f578c011500 con 0x7f579c0d6780 2024-04-17T22:31:09.341 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:09.355 7f57a2c09700 1 -- 172.21.15.102:0/1039031922 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_newops.TestNewOps.test_newops_getvxattr"]} v 0) v1 -- 0x7f579c1ae700 con 0x7f579c0d6780 2024-04-17T22:31:09.392 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:09.406 7f5798ff9700 1 -- 172.21.15.102:0/1039031922 <== mon.2 v2:172.21.15.102:3302/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_newops.TestNewOps.test_newops_getvxattr"]}]=0 v62) v1 ==== 139+0+0 (secure 0 0 0) 0x7f578c00f300 con 0x7f579c0d6780 2024-04-17T22:31:09.394 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:09.407 7f57a2c09700 1 -- 172.21.15.102:0/1039031922 >> v1:172.21.15.102:6816/42360 conn(0x7f577c0145a0 legacy=0x7f577c016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:09.394 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:09.407 7f57a2c09700 1 -- 172.21.15.102:0/1039031922 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f579c0d6780 msgr2=0x7f579c1b3fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:09.394 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:09.407 7f57a2c09700 1 --2- 172.21.15.102:0/1039031922 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f579c0d6780 0x7f579c1b3fa0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 rx=0x7f578c0030b0 tx=0x7f578c0068c0).stop 2024-04-17T22:31:09.394 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:09.408 7f57a2c09700 1 -- 172.21.15.102:0/1039031922 shutdown_connections 2024-04-17T22:31:09.394 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:09.408 7f57a2c09700 1 --2- 172.21.15.102:0/1039031922 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f579c0d6bd0 0x7f579c1b45d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:09.394 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:09.408 7f57a2c09700 1 --2- 172.21.15.102:0/1039031922 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f579c0dd8f0 0x7f579c0d6150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:09.394 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:09.408 7f57a2c09700 1 --2- 172.21.15.102:0/1039031922 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f579c0d6780 0x7f579c1b3fa0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:09.395 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:09.408 7f57a2c09700 1 -- 172.21.15.102:0/1039031922 >> 172.21.15.102:0/1039031922 conn(0x7f579c0c6a60 msgr2=0x7f579c0c9570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:09.395 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:09.408 7f57a2c09700 1 -- 172.21.15.102:0/1039031922 shutdown_connections 2024-04-17T22:31:09.395 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:09.408 7f57a2c09700 1 -- 172.21.15.102:0/1039031922 wait complete. 2024-04-17T22:31:09.405 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-17T22:31:09.533 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:09.545 7facffce8700 1 Processor -- start 2024-04-17T22:31:09.533 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:09.545 7facffce8700 1 -- start start 2024-04-17T22:31:09.533 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:09.545 7facffce8700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7facf80f5750 0x7facf810c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:09.534 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:09.545 7facffce8700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7facf810c720 0x7facf81a0be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:09.534 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:09.546 7facffce8700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7facf810cb70 0x7facf81a1210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:09.534 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:09.546 7facffce8700 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7facf80f93c0 con 0x7facf810cb70 2024-04-17T22:31:09.534 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:09.546 7facffce8700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7facf81a1840 con 0x7facf80f5750 2024-04-17T22:31:09.534 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:09.546 7facffce8700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7facf81a1a70 con 0x7facf810c720 2024-04-17T22:31:09.534 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:09.546 7facf7fff700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7facf810c720 0x7facf81a0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:09.534 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:09.546 7facfd2e4700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7facf810cb70 0x7facf81a1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:09.534 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:09.546 7facf7fff700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7facf810c720 0x7facf81a0be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:54974/0 (socket says 172.21.15.102:54974) 2024-04-17T22:31:09.534 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:09.546 7facfd2e4700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7facf810cb70 0x7facf81a1210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:54202/0 (socket says 172.21.15.102:54202) 2024-04-17T22:31:09.534 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:09.546 7facf7fff700 1 -- 172.21.15.102:0/3773915685 learned_addr learned my addr 172.21.15.102:0/3773915685 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:31:09.534 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:09.546 7facfcae3700 1 --2- 172.21.15.102:0/3773915685 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7facf80f5750 0x7facf810c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:09.534 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:09.546 7facfcae3700 1 -- 172.21.15.102:0/3773915685 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7facf810c720 msgr2=0x7facf81a0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:31:09.535 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:09.546 7facfcae3700 1 --2- 172.21.15.102:0/3773915685 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7facf810c720 0x7facf81a0be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:09.535 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:09.546 7facfcae3700 1 -- 172.21.15.102:0/3773915685 >> v2:172.21.15.102:3300/0 conn(0x7facf810cb70 msgr2=0x7facf81a1210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:31:09.535 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:09.546 7facfcae3700 1 --2- 172.21.15.102:0/3773915685 >> v2:172.21.15.102:3300/0 conn(0x7facf810cb70 0x7facf81a1210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:09.535 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:09.546 7facfcae3700 1 -- 172.21.15.102:0/3773915685 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facf81a82f0 con 0x7facf80f5750 2024-04-17T22:31:09.535 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:09.546 7facfd2e4700 1 --2- 172.21.15.102:0/3773915685 >> v2:172.21.15.102:3300/0 conn(0x7facf810cb70 0x7facf81a1210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:09.535 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:09.546 7facf7fff700 1 --2- 172.21.15.102:0/3773915685 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7facf810c720 0x7facf81a0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:09.535 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:09.546 7facfcae3700 1 --2- 172.21.15.102:0/3773915685 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7facf80f5750 0x7facf810c0f0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 rx=0x7facec002f70 tx=0x7facec00b9f0).ready entity=mon.1 client_cookie=402f3134b3d35f1c server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:09.535 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:09.546 7facf77fe700 1 -- 172.21.15.102:0/3773915685 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7facf81a82f0 con 0x7facf80f5750 2024-04-17T22:31:09.535 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:09.546 7facf77fe700 1 -- 172.21.15.102:0/3773915685 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facec00d570 con 0x7facf80f5750 2024-04-17T22:31:09.535 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:09.546 7facf77fe700 1 -- 172.21.15.102:0/3773915685 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7facec00d940 con 0x7facf80f5750 2024-04-17T22:31:09.535 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:09.546 7facffce8700 1 -- 172.21.15.102:0/3773915685 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7facf80f5750 msgr2=0x7facf810c0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:09.536 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:09.546 7facffce8700 1 --2- 172.21.15.102:0/3773915685 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7facf80f5750 0x7facf810c0f0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 rx=0x7facec002f70 tx=0x7facec00b9f0).stop 2024-04-17T22:31:09.536 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:09.547 7facffce8700 1 -- 172.21.15.102:0/3773915685 shutdown_connections 2024-04-17T22:31:09.536 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:09.547 7facffce8700 1 --2- 172.21.15.102:0/3773915685 >> v2:172.21.15.102:3300/0 conn(0x7facf810cb70 0x7facf81a1210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:09.536 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:09.547 7facffce8700 1 --2- 172.21.15.102:0/3773915685 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7facf80f5750 0x7facf810c0f0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:09.536 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:09.547 7facffce8700 1 --2- 172.21.15.102:0/3773915685 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7facf810c720 0x7facf81a0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:09.536 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:09.547 7facffce8700 1 -- 172.21.15.102:0/3773915685 >> 172.21.15.102:0/3773915685 conn(0x7facf81138a0 msgr2=0x7facf80f4f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:09.536 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:09.547 7facffce8700 1 -- 172.21.15.102:0/3773915685 shutdown_connections 2024-04-17T22:31:09.536 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:09.547 7facffce8700 1 -- 172.21.15.102:0/3773915685 wait complete. 2024-04-17T22:31:09.555 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:09.570 7facffce8700 1 Processor -- start 2024-04-17T22:31:09.556 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:09.570 7facffce8700 1 -- start start 2024-04-17T22:31:09.558 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:09.570 7facffce8700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7facf81ad100 0x7facf81ad550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:09.558 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:09.570 7facffce8700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7facf81adb80 0x7facf81b3b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:09.558 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:09.570 7facffce8700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7facf81b4130 0x7facf81b4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:09.558 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:09.570 7facffce8700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facf80f74b0 con 0x7facf81adb80 2024-04-17T22:31:09.559 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:09.570 7facffce8700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7facf80f77a0 con 0x7facf81b4130 2024-04-17T22:31:09.559 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:09.570 7facffce8700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7facf81a4920 con 0x7facf81ad100 2024-04-17T22:31:09.559 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:09.570 7facfd2e4700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7facf81b4130 0x7facf81b4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:09.559 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:09.570 7facf7fff700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7facf81adb80 0x7facf81b3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:09.559 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:09.570 7facfcae3700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7facf81ad100 0x7facf81ad550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:09.559 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:09.570 7facfd2e4700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7facf81b4130 0x7facf81b4580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:52148/0 (socket says 172.21.15.102:52148) 2024-04-17T22:31:09.559 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:09.570 7facf7fff700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7facf81adb80 0x7facf81b3b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:54218/0 (socket says 172.21.15.102:54218) 2024-04-17T22:31:09.559 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:09.570 7facfd2e4700 1 -- 172.21.15.102:0/1647623961 learned_addr learned my addr 172.21.15.102:0/1647623961 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:31:09.559 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:09.571 7facf7fff700 1 -- 172.21.15.102:0/1647623961 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7facf81ad100 msgr2=0x7facf81ad550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:09.559 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:09.571 7facf7fff700 1 --2- 172.21.15.102:0/1647623961 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7facf81ad100 0x7facf81ad550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:09.559 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:09.571 7facf7fff700 1 -- 172.21.15.102:0/1647623961 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7facf81b4130 msgr2=0x7facf81b4580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:09.560 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:09.571 7facf7fff700 1 --2- 172.21.15.102:0/1647623961 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7facf81b4130 0x7facf81b4580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:09.560 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:09.571 7facf7fff700 1 -- 172.21.15.102:0/1647623961 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facf81a4f20 con 0x7facf81adb80 2024-04-17T22:31:09.560 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:09.571 7facfd2e4700 1 --2- 172.21.15.102:0/1647623961 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7facf81b4130 0x7facf81b4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:09.560 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:09.571 7facf7fff700 1 --2- 172.21.15.102:0/1647623961 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7facf81adb80 0x7facf81b3b00 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 rx=0x7facf80f7430 tx=0x7face400ea20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:09.560 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:09.571 7facf5ffb700 1 -- 172.21.15.102:0/1647623961 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7facf81a4f20 con 0x7facf81adb80 2024-04-17T22:31:09.560 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:09.571 7facf5ffb700 1 -- 172.21.15.102:0/1647623961 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7face4012600 con 0x7facf81adb80 2024-04-17T22:31:09.560 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:09.571 7facffce8700 1 -- 172.21.15.102:0/1647623961 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7facf81bb270 con 0x7facf81adb80 2024-04-17T22:31:09.560 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:09.571 7facf5ffb700 1 -- 172.21.15.102:0/1647623961 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7face400f040 con 0x7facf81adb80 2024-04-17T22:31:09.560 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:09.571 7facffce8700 1 -- 172.21.15.102:0/1647623961 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7facf81bb650 con 0x7facf81adb80 2024-04-17T22:31:09.560 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:09.571 7facf5ffb700 1 -- 172.21.15.102:0/1647623961 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7face4011070 con 0x7facf81adb80 2024-04-17T22:31:09.560 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:09.571 7facf5ffb700 1 -- 172.21.15.102:0/1647623961 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3194+0+0 (secure 0 0 0) 0x7face4010040 con 0x7facf81adb80 2024-04-17T22:31:09.560 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:09.572 7facffce8700 1 -- 172.21.15.102:0/1647623961 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7facf80f0800 con 0x7facf81adb80 2024-04-17T22:31:09.561 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:09.575 7facf5ffb700 1 -- 172.21.15.102:0/1647623961 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+110455 (secure 0 0 0) 0x7face40123f0 con 0x7facf81adb80 2024-04-17T22:31:09.666 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:09.681 7facffce8700 1 -- 172.21.15.102:0/1647623961 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7facf80e21f0 con 0x7facf81adb80 2024-04-17T22:31:09.669 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:09.684 7facf5ffb700 1 -- 172.21.15.102:0/1647623961 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v2)=0 v2) v1 ==== 122+0+0 (secure 0 0 0) 0x7face4000c30 con 0x7facf81adb80 2024-04-17T22:31:09.671 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:09.685 7facffce8700 1 -- 172.21.15.102:0/1647623961 >> v1:172.21.15.102:6816/42360 conn(0x7face00145a0 legacy=0x7face0016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:09.672 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:09.685 7facffce8700 1 -- 172.21.15.102:0/1647623961 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7facf81adb80 msgr2=0x7facf81b3b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:09.672 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:09.685 7facffce8700 1 --2- 172.21.15.102:0/1647623961 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7facf81adb80 0x7facf81b3b00 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 rx=0x7facf80f7430 tx=0x7face400ea20).stop 2024-04-17T22:31:09.672 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:09.685 7facffce8700 1 -- 172.21.15.102:0/1647623961 shutdown_connections 2024-04-17T22:31:09.672 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:09.685 7facffce8700 1 --2- 172.21.15.102:0/1647623961 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7facf81b4130 0x7facf81b4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:09.672 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:09.685 7facffce8700 1 --2- 172.21.15.102:0/1647623961 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7facf81ad100 0x7facf81ad550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:09.672 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:09.685 7facffce8700 1 --2- 172.21.15.102:0/1647623961 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7facf81adb80 0x7facf81b3b00 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:09.672 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:09.685 7facffce8700 1 -- 172.21.15.102:0/1647623961 >> 172.21.15.102:0/1647623961 conn(0x7facf80e0d40 msgr2=0x7facf81a2030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:09.672 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:09.685 7facffce8700 1 -- 172.21.15.102:0/1647623961 shutdown_connections 2024-04-17T22:31:09.672 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:09.685 7facffce8700 1 -- 172.21.15.102:0/1647623961 wait complete. 2024-04-17T22:31:09.682 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-17T22:31:09.682 DEBUG:teuthology.orchestra.run.smithi102:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-17T22:31:09.800 DEBUG:teuthology.orchestra.run.smithi190:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-17T22:31:09.847 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-17T22:31:09.847 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-04-17T22:31:09.847 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T22:31:09.867 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-04-17T22:31:09.867 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T22:31:09.927 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T22:31:10.054 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.066 7f948bbb3700 1 Processor -- start 2024-04-17T22:31:10.054 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.066 7f948bbb3700 1 -- start start 2024-04-17T22:31:10.055 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.067 7f948bbb3700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f948410e260 0x7f948410e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:10.055 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.067 7f948bbb3700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f948410ece0 0x7f94840f1750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:10.055 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.067 7f948bbb3700 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94840f21d0 con 0x7f94840f1d80 2024-04-17T22:31:10.055 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.067 7f948bbb3700 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94840f9f30 con 0x7f948410e260 2024-04-17T22:31:10.055 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.067 7f948bbb3700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94840f9050 con 0x7f948410ece0 2024-04-17T22:31:10.055 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.067 7f94889ae700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f948410e260 0x7f948410e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:10.055 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.067 7f9483fff700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f948410ece0 0x7f94840f1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:10.055 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.067 7f94889ae700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f948410e260 0x7f948410e6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:54224/0 (socket says 172.21.15.102:54224) 2024-04-17T22:31:10.055 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.067 7f9483fff700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f948410ece0 0x7f94840f1750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:52150/0 (socket says 172.21.15.102:52150) 2024-04-17T22:31:10.055 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.067 7f94889ae700 1 -- 172.21.15.102:0/2396182401 learned_addr learned my addr 172.21.15.102:0/2396182401 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:31:10.055 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.067 7f9483fff700 1 -- 172.21.15.102:0/2396182401 >> v2:172.21.15.102:3300/0 conn(0x7f948410e260 msgr2=0x7f948410e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:31:10.055 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.067 7f9483fff700 1 --2- 172.21.15.102:0/2396182401 >> v2:172.21.15.102:3300/0 conn(0x7f948410e260 0x7f948410e6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:10.055 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.067 7f9483fff700 1 -- 172.21.15.102:0/2396182401 >> v1:172.21.15.102:6789/0 conn(0x7f94840f1d80 legacy=0x7f94841aa4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:10.055 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.067 7f9483fff700 1 -- 172.21.15.102:0/2396182401 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94840f23f0 con 0x7f948410ece0 2024-04-17T22:31:10.056 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.067 7f94837fe700 1 -- 172.21.15.102:0/2396182401 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2427906505 0 0) 0x7f947c001490 con 0x7f94840f1d80 2024-04-17T22:31:10.056 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.067 7f94889ae700 1 --2- 172.21.15.102:0/2396182401 >> v2:172.21.15.102:3300/0 conn(0x7f948410e260 0x7f948410e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:10.056 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.067 7f9483fff700 1 --2- 172.21.15.102:0/2396182401 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f948410ece0 0x7f94840f1750 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 rx=0x7f946c002f70 tx=0x7f946c00b970).ready entity=mon.1 client_cookie=945bb1b04e71123e server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:10.056 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.067 7f94837fe700 1 -- 172.21.15.102:0/2396182401 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f94840f23f0 con 0x7f948410ece0 2024-04-17T22:31:10.056 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.067 7f94837fe700 1 -- 172.21.15.102:0/2396182401 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f946c00d610 con 0x7f948410ece0 2024-04-17T22:31:10.056 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.067 7f94837fe700 1 -- 172.21.15.102:0/2396182401 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f946c00d9e0 con 0x7f948410ece0 2024-04-17T22:31:10.056 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.067 7f948bbb3700 1 -- 172.21.15.102:0/2396182401 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f948410ece0 msgr2=0x7f94840f1750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:10.056 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.067 7f948bbb3700 1 --2- 172.21.15.102:0/2396182401 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f948410ece0 0x7f94840f1750 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 rx=0x7f946c002f70 tx=0x7f946c00b970).stop 2024-04-17T22:31:10.056 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.068 7f948bbb3700 1 -- 172.21.15.102:0/2396182401 shutdown_connections 2024-04-17T22:31:10.056 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.068 7f948bbb3700 1 --2- 172.21.15.102:0/2396182401 >> v2:172.21.15.102:3300/0 conn(0x7f948410e260 0x7f948410e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:10.056 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.068 7f948bbb3700 1 --2- 172.21.15.102:0/2396182401 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f948410ece0 0x7f94840f1750 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:10.056 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.068 7f948bbb3700 1 -- 172.21.15.102:0/2396182401 >> 172.21.15.102:0/2396182401 conn(0x7f94841138c0 msgr2=0x7f94840f4f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:10.056 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.068 7f948bbb3700 1 -- 172.21.15.102:0/2396182401 shutdown_connections 2024-04-17T22:31:10.056 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.068 7f948bbb3700 1 -- 172.21.15.102:0/2396182401 wait complete. 2024-04-17T22:31:10.077 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.091 7f948bbb3700 1 Processor -- start 2024-04-17T22:31:10.077 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.091 7f948bbb3700 1 -- start start 2024-04-17T22:31:10.080 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.091 7f948bbb3700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f94840d6130 0x7f94840d6580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:10.080 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.091 7f948bbb3700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f94840d6bb0 0x7f94841b40a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:10.080 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.091 7f948bbb3700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f94841b46d0 0x7f94841b4b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:10.080 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.091 7f948bbb3700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94840fa1d0 con 0x7f94840d6bb0 2024-04-17T22:31:10.080 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.091 7f948bbb3700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94841a8f90 con 0x7f94841b46d0 2024-04-17T22:31:10.080 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.091 7f948bbb3700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f94840ddee0 con 0x7f94840d6130 2024-04-17T22:31:10.080 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.091 7f9483fff700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f94840d6bb0 0x7f94841b40a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:10.080 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.091 7f94891af700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f94841b46d0 0x7f94841b4b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:10.080 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.091 7f9483fff700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f94840d6bb0 0x7f94841b40a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:54228/0 (socket says 172.21.15.102:54228) 2024-04-17T22:31:10.080 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.091 7f9483fff700 1 -- 172.21.15.102:0/1839182421 learned_addr learned my addr 172.21.15.102:0/1839182421 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:31:10.080 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.091 7f94891af700 1 --2- 172.21.15.102:0/1839182421 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f94841b46d0 0x7f94841b4b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:52152/0 (socket says 172.21.15.102:52152) 2024-04-17T22:31:10.080 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.091 7f94889ae700 1 --2- 172.21.15.102:0/1839182421 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f94840d6130 0x7f94840d6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:10.081 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.092 7f9483fff700 1 -- 172.21.15.102:0/1839182421 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f94840d6130 msgr2=0x7f94840d6580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:10.081 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.092 7f9483fff700 1 --2- 172.21.15.102:0/1839182421 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f94840d6130 0x7f94840d6580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:10.081 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.092 7f9483fff700 1 -- 172.21.15.102:0/1839182421 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f94841b46d0 msgr2=0x7f94841b4b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:10.081 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.092 7f9483fff700 1 --2- 172.21.15.102:0/1839182421 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f94841b46d0 0x7f94841b4b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:10.081 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.092 7f9483fff700 1 -- 172.21.15.102:0/1839182421 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94841a89e0 con 0x7f94840d6bb0 2024-04-17T22:31:10.081 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.092 7f94889ae700 1 --2- 172.21.15.102:0/1839182421 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f94840d6130 0x7f94840d6580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:10.081 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.092 7f94891af700 1 --2- 172.21.15.102:0/1839182421 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f94841b46d0 0x7f94841b4b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:10.081 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.092 7f9483fff700 1 --2- 172.21.15.102:0/1839182421 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f94840d6bb0 0x7f94841b40a0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 rx=0x7f94840c8680 tx=0x7f946c00a040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:10.081 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.092 7f9481ffb700 1 -- 172.21.15.102:0/1839182421 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f94841a89e0 con 0x7f94840d6bb0 2024-04-17T22:31:10.081 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.092 7f9481ffb700 1 -- 172.21.15.102:0/1839182421 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f946c013a40 con 0x7f94840d6bb0 2024-04-17T22:31:10.081 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.092 7f9481ffb700 1 -- 172.21.15.102:0/1839182421 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f946c01f450 con 0x7f94840d6bb0 2024-04-17T22:31:10.081 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.092 7f948bbb3700 1 -- 172.21.15.102:0/1839182421 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94840dd8a0 con 0x7f94840d6bb0 2024-04-17T22:31:10.081 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.092 7f948bbb3700 1 -- 172.21.15.102:0/1839182421 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94841bb790 con 0x7f94840d6bb0 2024-04-17T22:31:10.082 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.092 7f9481ffb700 1 -- 172.21.15.102:0/1839182421 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7f946c0113c0 con 0x7f94840d6bb0 2024-04-17T22:31:10.082 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.092 7f9481ffb700 1 -- 172.21.15.102:0/1839182421 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3194+0+0 (secure 0 0 0) 0x7f946c008cc0 con 0x7f94840d6bb0 2024-04-17T22:31:10.082 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.093 7f948bbb3700 1 -- 172.21.15.102:0/1839182421 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94840f0820 con 0x7f94840d6bb0 2024-04-17T22:31:10.082 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.096 7f9481ffb700 1 -- 172.21.15.102:0/1839182421 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+110455 (secure 0 0 0) 0x7f946c013830 con 0x7f94840d6bb0 2024-04-17T22:31:10.187 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.201 7f948bbb3700 1 -- 172.21.15.102:0/1839182421 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f94840f5550 con 0x7f94840d6bb0 2024-04-17T22:31:10.187 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.201 7f9481ffb700 1 -- 172.21.15.102:0/1839182421 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) v1 ==== 74+0+8764 (secure 0 0 0) 0x7f946c00d400 con 0x7f94840d6bb0 2024-04-17T22:31:10.187 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-17T22:31:10.188 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":17,"fsid":"cb63e789-091d-49b1-b120-55f9db526871","created":"2024-04-17 22:28:48.209635","modified":"2024-04-17 22:29:23.257387","last_up_change":"2024-04-17 22:28:54.236215","last_in_change":"2024-04-17 22:28:54.236215","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":4,"require_min_compat_client":"nautilus","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-04-17 22:28:57.478545","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-17 22:29:06.144708","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-17 22:29:06.748924","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3351ef85-ade4-4b90-a8f4-213f4e00ba01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6812","nonce":42955}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6813","nonce":42955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6815","nonce":42955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6814","nonce":42955}]},"public_addr":"172.21.15.102:6812/42955","cluster_addr":"172.21.15.102:6813/42955","heartbeat_back_addr":"172.21.15.102:6815/42955","heartbeat_front_addr":"172.21.15.102:6814/42955","state":["exists","up"]},{"osd":1,"uuid":"04f67eac-f36b-48f5-982d-4cfc776596d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6808","nonce":42958}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6809","nonce":42958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6811","nonce":42958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6810","nonce":42958}]},"public_addr":"172.21.15.102:6808/42958","cluster_addr":"172.21.15.102:6809/42958","heartbeat_back_addr":"172.21.15.102:6811/42958","heartbeat_front_addr":"172.21.15.102:6810/42958","state":["exists","up"]},{"osd":2,"uuid":"d524204b-70b2-40a8-8f73-f9e7d44318af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6804","nonce":42954}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6805","nonce":42954}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6807","nonce":42954}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6806","nonce":42954}]},"public_addr":"172.21.15.102:6804/42954","cluster_addr":"172.21.15.102:6805/42954","heartbeat_back_addr":"172.21.15.102:6807/42954","heartbeat_front_addr":"172.21.15.102:6806/42954","state":["exists","up"]},{"osd":3,"uuid":"4fc9d20c-1ad0-484f-aafa-2c9a7574245b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6800","nonce":42953}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6801","nonce":42953}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6803","nonce":42953}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6802","nonce":42953}]},"public_addr":"172.21.15.102:6800/42953","cluster_addr":"172.21.15.102:6801/42953","heartbeat_back_addr":"172.21.15.102:6803/42953","heartbeat_front_addr":"172.21.15.102:6802/42953","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2024-04-17T22:31:10.189 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.202 7f948bbb3700 1 -- 172.21.15.102:0/1839182421 >> v1:172.21.15.102:6816/42360 conn(0x7f94680145a0 legacy=0x7f9468016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:10.189 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.202 7f948bbb3700 1 -- 172.21.15.102:0/1839182421 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f94840d6bb0 msgr2=0x7f94841b40a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:10.189 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.202 7f948bbb3700 1 --2- 172.21.15.102:0/1839182421 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f94840d6bb0 0x7f94841b40a0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 rx=0x7f94840c8680 tx=0x7f946c00a040).stop 2024-04-17T22:31:10.189 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.203 7f948bbb3700 1 -- 172.21.15.102:0/1839182421 shutdown_connections 2024-04-17T22:31:10.189 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.203 7f948bbb3700 1 --2- 172.21.15.102:0/1839182421 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f94841b46d0 0x7f94841b4b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:10.189 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.203 7f948bbb3700 1 --2- 172.21.15.102:0/1839182421 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f94840d6130 0x7f94840d6580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:10.189 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.203 7f948bbb3700 1 --2- 172.21.15.102:0/1839182421 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f94840d6bb0 0x7f94841b40a0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:10.190 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.203 7f948bbb3700 1 -- 172.21.15.102:0/1839182421 >> 172.21.15.102:0/1839182421 conn(0x7f94840c6a60 msgr2=0x7f94840c30e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:10.190 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.203 7f948bbb3700 1 -- 172.21.15.102:0/1839182421 shutdown_connections 2024-04-17T22:31:10.190 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.203 7f948bbb3700 1 -- 172.21.15.102:0/1839182421 wait complete. 2024-04-17T22:31:10.200 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2024-04-17 22:28:57.478545', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '12', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-17 22:29:06.144708', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-17 22:29:06.748924', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-17T22:31:10.200 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2024-04-17T22:31:10.329 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.341 7f7e333e6700 1 Processor -- start 2024-04-17T22:31:10.329 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.341 7f7e333e6700 1 -- start start 2024-04-17T22:31:10.329 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.341 7f7e333e6700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f7e2c10e240 0x7f7e2c10e690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:10.329 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.341 7f7e333e6700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f7e2c0f1f30 0x7f7e2c1ab4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:10.329 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.341 7f7e333e6700 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e2c0f2380 con 0x7f7e2c10ecc0 2024-04-17T22:31:10.329 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.341 7f7e333e6700 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e2c0f9f10 con 0x7f7e2c0f1f30 2024-04-17T22:31:10.329 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.341 7f7e333e6700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7e2c0f9030 con 0x7f7e2c10e240 2024-04-17T22:31:10.329 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.341 7f7e309e2700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f7e2c0f1f30 0x7f7e2c1ab4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:10.329 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.341 7f7e2bfff700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f7e2c10e240 0x7f7e2c10e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:10.329 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.341 7f7e309e2700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f7e2c0f1f30 0x7f7e2c1ab4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:54232/0 (socket says 172.21.15.102:54232) 2024-04-17T22:31:10.329 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.341 7f7e2bfff700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f7e2c10e240 0x7f7e2c10e690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:55012/0 (socket says 172.21.15.102:55012) 2024-04-17T22:31:10.330 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.341 7f7e2b7fe700 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f7e2c10ecc0 0x7f7e2c0f1730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:38770/0 (socket says 172.21.15.102:38770) 2024-04-17T22:31:10.330 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.341 7f7e309e2700 1 -- 172.21.15.102:0/2818176341 learned_addr learned my addr 172.21.15.102:0/2818176341 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:31:10.330 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.341 7f7e2bfff700 1 -- 172.21.15.102:0/2818176341 >> v2:172.21.15.102:3300/0 conn(0x7f7e2c0f1f30 msgr2=0x7f7e2c1ab4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:31:10.330 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.341 7f7e2bfff700 1 --2- 172.21.15.102:0/2818176341 >> v2:172.21.15.102:3300/0 conn(0x7f7e2c0f1f30 0x7f7e2c1ab4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:10.330 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.341 7f7e2bfff700 1 -- 172.21.15.102:0/2818176341 >> v1:172.21.15.102:6789/0 conn(0x7f7e2c10ecc0 legacy=0x7f7e2c0f1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:10.330 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.341 7f7e2bfff700 1 -- 172.21.15.102:0/2818176341 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e2c0f25a0 con 0x7f7e2c10e240 2024-04-17T22:31:10.330 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.341 7f7e309e2700 1 --2- 172.21.15.102:0/2818176341 >> v2:172.21.15.102:3300/0 conn(0x7f7e2c0f1f30 0x7f7e2c1ab4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:10.330 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.342 7f7e2bfff700 1 --2- 172.21.15.102:0/2818176341 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f7e2c10e240 0x7f7e2c10e690 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 rx=0x7f7e1c002f70 tx=0x7f7e1c00b970).ready entity=mon.2 client_cookie=2f0f4eca6fbf199b server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:10.330 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.342 7f7e2affd700 1 -- 172.21.15.102:0/2818176341 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f7e2c0f25a0 con 0x7f7e2c10e240 2024-04-17T22:31:10.330 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.342 7f7e2affd700 1 -- 172.21.15.102:0/2818176341 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e1c00d610 con 0x7f7e2c10e240 2024-04-17T22:31:10.330 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.342 7f7e2affd700 1 -- 172.21.15.102:0/2818176341 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f7e1c00d9e0 con 0x7f7e2c10e240 2024-04-17T22:31:10.330 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.342 7f7e333e6700 1 -- 172.21.15.102:0/2818176341 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f7e2c10e240 msgr2=0x7f7e2c10e690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:10.331 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.342 7f7e333e6700 1 --2- 172.21.15.102:0/2818176341 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f7e2c10e240 0x7f7e2c10e690 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 rx=0x7f7e1c002f70 tx=0x7f7e1c00b970).stop 2024-04-17T22:31:10.331 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.342 7f7e333e6700 1 -- 172.21.15.102:0/2818176341 shutdown_connections 2024-04-17T22:31:10.331 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.342 7f7e333e6700 1 --2- 172.21.15.102:0/2818176341 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f7e2c10e240 0x7f7e2c10e690 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:10.331 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.342 7f7e333e6700 1 --2- 172.21.15.102:0/2818176341 >> v2:172.21.15.102:3300/0 conn(0x7f7e2c0f1f30 0x7f7e2c1ab4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:10.331 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.342 7f7e333e6700 1 -- 172.21.15.102:0/2818176341 >> 172.21.15.102:0/2818176341 conn(0x7f7e2c1138a0 msgr2=0x7f7e2c0f4f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:10.331 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.342 7f7e333e6700 1 -- 172.21.15.102:0/2818176341 shutdown_connections 2024-04-17T22:31:10.331 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.342 7f7e333e6700 1 -- 172.21.15.102:0/2818176341 wait complete. 2024-04-17T22:31:10.351 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.365 7f7e333e6700 1 Processor -- start 2024-04-17T22:31:10.351 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.365 7f7e333e6700 1 -- start start 2024-04-17T22:31:10.353 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.365 7f7e333e6700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7e2c0c7a30 0x7f7e2c1ada90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:10.353 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.365 7f7e333e6700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f7e2c1ae0c0 0x7f7e2c1b44e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:10.353 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.365 7f7e333e6700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f7e2c1ae510 0x7f7e2c1b4b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:10.353 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.365 7f7e333e6700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e2c0e2c80 con 0x7f7e2c0c7a30 2024-04-17T22:31:10.353 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.365 7f7e333e6700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e2c0fa1b0 con 0x7f7e2c1ae510 2024-04-17T22:31:10.353 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.365 7f7e333e6700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7e2c0e0a20 con 0x7f7e2c1ae0c0 2024-04-17T22:31:10.354 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.365 7f7e2b7fe700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f7e2c1ae0c0 0x7f7e2c1b44e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:10.354 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.365 7f7e2bfff700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7e2c0c7a30 0x7f7e2c1ada90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:10.354 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.365 7f7e2b7fe700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f7e2c1ae0c0 0x7f7e2c1b44e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:55028/0 (socket says 172.21.15.102:55028) 2024-04-17T22:31:10.354 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.365 7f7e2bfff700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7e2c0c7a30 0x7f7e2c1ada90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:54246/0 (socket says 172.21.15.102:54246) 2024-04-17T22:31:10.354 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.365 7f7e2b7fe700 1 -- 172.21.15.102:0/2311682315 learned_addr learned my addr 172.21.15.102:0/2311682315 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:31:10.354 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.365 7f7e309e2700 1 --2- 172.21.15.102:0/2311682315 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f7e2c1ae510 0x7f7e2c1b4b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:10.354 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.366 7f7e2bfff700 1 -- 172.21.15.102:0/2311682315 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f7e2c1ae0c0 msgr2=0x7f7e2c1b44e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:10.354 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.366 7f7e2bfff700 1 --2- 172.21.15.102:0/2311682315 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f7e2c1ae0c0 0x7f7e2c1b44e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:10.354 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.366 7f7e2bfff700 1 -- 172.21.15.102:0/2311682315 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f7e2c1ae510 msgr2=0x7f7e2c1b4b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:10.354 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.366 7f7e2bfff700 1 --2- 172.21.15.102:0/2311682315 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f7e2c1ae510 0x7f7e2c1b4b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:10.354 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.366 7f7e2bfff700 1 -- 172.21.15.102:0/2311682315 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e2c0e0410 con 0x7f7e2c0c7a30 2024-04-17T22:31:10.354 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.366 7f7e2b7fe700 1 --2- 172.21.15.102:0/2311682315 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f7e2c1ae0c0 0x7f7e2c1b44e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:10.355 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.366 7f7e309e2700 1 --2- 172.21.15.102:0/2311682315 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f7e2c1ae510 0x7f7e2c1b4b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:10.355 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.366 7f7e2bfff700 1 --2- 172.21.15.102:0/2311682315 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7e2c0c7a30 0x7f7e2c1ada90 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 rx=0x7f7e2c0c74a0 tx=0x7f7e1c00a040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:10.355 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.366 7f7e297fa700 1 -- 172.21.15.102:0/2311682315 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f7e2c0e0410 con 0x7f7e2c0c7a30 2024-04-17T22:31:10.355 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.366 7f7e297fa700 1 -- 172.21.15.102:0/2311682315 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e1c0139b0 con 0x7f7e2c0c7a30 2024-04-17T22:31:10.355 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.366 7f7e333e6700 1 -- 172.21.15.102:0/2311682315 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e2c1b5140 con 0x7f7e2c0c7a30 2024-04-17T22:31:10.355 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.366 7f7e297fa700 1 -- 172.21.15.102:0/2311682315 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f7e1c013d80 con 0x7f7e2c0c7a30 2024-04-17T22:31:10.355 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.366 7f7e333e6700 1 -- 172.21.15.102:0/2311682315 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e2c1bbc50 con 0x7f7e2c0c7a30 2024-04-17T22:31:10.355 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.366 7f7e297fa700 1 -- 172.21.15.102:0/2311682315 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7f7e1c027070 con 0x7f7e2c0c7a30 2024-04-17T22:31:10.355 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.367 7f7e297fa700 1 -- 172.21.15.102:0/2311682315 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3194+0+0 (secure 0 0 0) 0x7f7e1c008750 con 0x7f7e2c0c7a30 2024-04-17T22:31:10.355 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.367 7f7e333e6700 1 -- 172.21.15.102:0/2311682315 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e2c0f0800 con 0x7f7e2c0c7a30 2024-04-17T22:31:10.356 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.370 7f7e297fa700 1 -- 172.21.15.102:0/2311682315 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+110455 (secure 0 0 0) 0x7f7e1c0137a0 con 0x7f7e2c0c7a30 2024-04-17T22:31:10.463 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.476 7f7e333e6700 1 -- 172.21.15.102:0/2311682315 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f7e2c0e21f0 con 0x7f7e2c0c7a30 2024-04-17T22:31:10.463 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.477 7f7e297fa700 1 -- 172.21.15.102:0/2311682315 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v17) v1 ==== 92+0+10 (secure 0 0 0) 0x7f7e1c00d400 con 0x7f7e2c0c7a30 2024-04-17T22:31:10.463 INFO:teuthology.orchestra.run.smithi102.stdout:pg_num: 8 2024-04-17T22:31:10.464 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.478 7f7e333e6700 1 -- 172.21.15.102:0/2311682315 >> v1:172.21.15.102:6816/42360 conn(0x7f7e0c0145a0 legacy=0x7f7e0c016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:10.465 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.478 7f7e333e6700 1 -- 172.21.15.102:0/2311682315 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7e2c0c7a30 msgr2=0x7f7e2c1ada90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:10.465 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.478 7f7e333e6700 1 --2- 172.21.15.102:0/2311682315 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7e2c0c7a30 0x7f7e2c1ada90 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 rx=0x7f7e2c0c74a0 tx=0x7f7e1c00a040).stop 2024-04-17T22:31:10.465 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.478 7f7e333e6700 1 -- 172.21.15.102:0/2311682315 shutdown_connections 2024-04-17T22:31:10.465 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.478 7f7e333e6700 1 --2- 172.21.15.102:0/2311682315 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f7e2c1ae510 0x7f7e2c1b4b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:10.465 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.478 7f7e333e6700 1 --2- 172.21.15.102:0/2311682315 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7e2c0c7a30 0x7f7e2c1ada90 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:10.465 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.478 7f7e333e6700 1 --2- 172.21.15.102:0/2311682315 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f7e2c1ae0c0 0x7f7e2c1b44e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:10.465 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.478 7f7e333e6700 1 -- 172.21.15.102:0/2311682315 >> 172.21.15.102:0/2311682315 conn(0x7f7e2c0c6c90 msgr2=0x7f7e2c0d7300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:10.465 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.478 7f7e333e6700 1 -- 172.21.15.102:0/2311682315 shutdown_connections 2024-04-17T22:31:10.465 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.478 7f7e333e6700 1 -- 172.21.15.102:0/2311682315 wait complete. 2024-04-17T22:31:10.475 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-17T22:31:10.605 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.616 7f920c1b9700 1 Processor -- start 2024-04-17T22:31:10.605 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.617 7f920c1b9700 1 -- start start 2024-04-17T22:31:10.605 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.617 7f920c1b9700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f920410e240 0x7f920410e690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:10.605 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.617 7f920c1b9700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f92040f1f30 0x7f92041ab4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:10.605 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.617 7f920c1b9700 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92040f2380 con 0x7f920410ecc0 2024-04-17T22:31:10.605 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.617 7f920c1b9700 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92040f9f10 con 0x7f920410e240 2024-04-17T22:31:10.605 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.617 7f920c1b9700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92040f9030 con 0x7f92040f1f30 2024-04-17T22:31:10.605 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.617 7f92097b5700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f92040f1f30 0x7f92041ab4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:10.606 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.617 7f9208fb4700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f920410e240 0x7f920410e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:10.606 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.617 7f92097b5700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f92040f1f30 0x7f92041ab4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:52162/0 (socket says 172.21.15.102:52162) 2024-04-17T22:31:10.606 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.617 7f92097b5700 1 -- 172.21.15.102:0/3724722531 learned_addr learned my addr 172.21.15.102:0/3724722531 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:31:10.606 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.617 7f9203fff700 1 --1- 172.21.15.102:0/3724722531 >> v1:172.21.15.102:6789/0 conn(0x7f920410ecc0 0x7f92040f1730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:38776/0 (socket says 172.21.15.102:38776) 2024-04-17T22:31:10.606 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.617 7f9208fb4700 1 -- 172.21.15.102:0/3724722531 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f92040f1f30 msgr2=0x7f92041ab4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:31:10.606 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.617 7f9208fb4700 1 --2- 172.21.15.102:0/3724722531 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f92040f1f30 0x7f92041ab4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:10.606 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.617 7f9208fb4700 1 -- 172.21.15.102:0/3724722531 >> v1:172.21.15.102:6789/0 conn(0x7f920410ecc0 legacy=0x7f92040f1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:10.606 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.617 7f9208fb4700 1 -- 172.21.15.102:0/3724722531 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92040f25a0 con 0x7f920410e240 2024-04-17T22:31:10.606 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.617 7f92037fe700 1 -- 172.21.15.102:0/3724722531 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3469598878 0 0) 0x7f91f0001110 con 0x7f920410ecc0 2024-04-17T22:31:10.606 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.617 7f92097b5700 1 --2- 172.21.15.102:0/3724722531 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f92040f1f30 0x7f92041ab4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:10.606 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.617 7f9208fb4700 1 --2- 172.21.15.102:0/3724722531 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f920410e240 0x7f920410e690 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 rx=0x7f92040f46a0 tx=0x7f91f800ba20).ready entity=mon.0 client_cookie=e4505f5b200adda2 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:10.606 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.618 7f92037fe700 1 -- 172.21.15.102:0/3724722531 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f92040f25a0 con 0x7f920410e240 2024-04-17T22:31:10.606 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.618 7f92037fe700 1 -- 172.21.15.102:0/3724722531 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91f800d5c0 con 0x7f920410e240 2024-04-17T22:31:10.607 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.618 7f92037fe700 1 -- 172.21.15.102:0/3724722531 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f91f800d990 con 0x7f920410e240 2024-04-17T22:31:10.607 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.618 7f920c1b9700 1 -- 172.21.15.102:0/3724722531 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f920410e240 msgr2=0x7f920410e690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:10.607 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.618 7f920c1b9700 1 --2- 172.21.15.102:0/3724722531 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f920410e240 0x7f920410e690 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 rx=0x7f92040f46a0 tx=0x7f91f800ba20).stop 2024-04-17T22:31:10.607 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.618 7f920c1b9700 1 -- 172.21.15.102:0/3724722531 shutdown_connections 2024-04-17T22:31:10.607 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.618 7f920c1b9700 1 --2- 172.21.15.102:0/3724722531 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f92040f1f30 0x7f92041ab4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:10.607 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.618 7f920c1b9700 1 --2- 172.21.15.102:0/3724722531 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f920410e240 0x7f920410e690 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:10.607 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.618 7f920c1b9700 1 -- 172.21.15.102:0/3724722531 >> 172.21.15.102:0/3724722531 conn(0x7f92041138a0 msgr2=0x7f92040f4f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:10.607 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.618 7f920c1b9700 1 -- 172.21.15.102:0/3724722531 shutdown_connections 2024-04-17T22:31:10.607 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.618 7f920c1b9700 1 -- 172.21.15.102:0/3724722531 wait complete. 2024-04-17T22:31:10.627 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.641 7f920c1b9700 1 Processor -- start 2024-04-17T22:31:10.627 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.641 7f920c1b9700 1 -- start start 2024-04-17T22:31:10.629 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.641 7f920c1b9700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f92041ad9e0 0x7f92041ade30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:10.629 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.641 7f920c1b9700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f92041ae460 0x7f92041b43e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:10.629 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.641 7f920c1b9700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f92041b4a10 0x7f92041b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:10.629 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.641 7f920c1b9700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92040fa1b0 con 0x7f92041b4a10 2024-04-17T22:31:10.629 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.641 7f920c1b9700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92040d7cf0 con 0x7f92041ae460 2024-04-17T22:31:10.629 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.641 7f920c1b9700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f92040d7ac0 con 0x7f92041ad9e0 2024-04-17T22:31:10.629 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.641 7f9203fff700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f92041ae460 0x7f92041b43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:10.630 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.641 7f9208fb4700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f92041ad9e0 0x7f92041ade30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:10.630 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.641 7f92097b5700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f92041b4a10 0x7f92041b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:10.630 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.641 7f9203fff700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f92041ae460 0x7f92041b43e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:52170/0 (socket says 172.21.15.102:52170) 2024-04-17T22:31:10.630 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.641 7f92097b5700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f92041b4a10 0x7f92041b4e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:54264/0 (socket says 172.21.15.102:54264) 2024-04-17T22:31:10.630 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.641 7f9208fb4700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f92041ad9e0 0x7f92041ade30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:55040/0 (socket says 172.21.15.102:55040) 2024-04-17T22:31:10.630 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.641 7f9203fff700 1 -- 172.21.15.102:0/2316087019 learned_addr learned my addr 172.21.15.102:0/2316087019 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:31:10.630 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.642 7f9203fff700 1 -- 172.21.15.102:0/2316087019 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f92041ad9e0 msgr2=0x7f92041ade30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:10.630 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.642 7f9203fff700 1 --2- 172.21.15.102:0/2316087019 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f92041ad9e0 0x7f92041ade30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:10.630 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.642 7f9203fff700 1 -- 172.21.15.102:0/2316087019 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f92041b4a10 msgr2=0x7f92041b4e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:10.630 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.642 7f9203fff700 1 --2- 172.21.15.102:0/2316087019 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f92041b4a10 0x7f92041b4e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:10.630 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.642 7f9203fff700 1 -- 172.21.15.102:0/2316087019 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92040d87f0 con 0x7f92041ae460 2024-04-17T22:31:10.630 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.642 7f92097b5700 1 --2- 172.21.15.102:0/2316087019 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f92041b4a10 0x7f92041b4e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:10.630 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.642 7f9203fff700 1 --2- 172.21.15.102:0/2316087019 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f92041ae460 0x7f92041b43e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 rx=0x7f92040d7930 tx=0x7f91f000c890).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:10.631 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.642 7f9201ffb700 1 -- 172.21.15.102:0/2316087019 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f92040d87f0 con 0x7f92041ae460 2024-04-17T22:31:10.631 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.642 7f9201ffb700 1 -- 172.21.15.102:0/2316087019 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91f00072e0 con 0x7f92041ae460 2024-04-17T22:31:10.631 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.642 7f920c1b9700 1 -- 172.21.15.102:0/2316087019 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92041bbb50 con 0x7f92041ae460 2024-04-17T22:31:10.631 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.642 7f9201ffb700 1 -- 172.21.15.102:0/2316087019 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f91f0005750 con 0x7f92041ae460 2024-04-17T22:31:10.631 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.642 7f920c1b9700 1 -- 172.21.15.102:0/2316087019 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92041bbf30 con 0x7f92041ae460 2024-04-17T22:31:10.631 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.642 7f9201ffb700 1 -- 172.21.15.102:0/2316087019 <== mon.1 v2:172.21.15.102:3301/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7f91f0005950 con 0x7f92041ae460 2024-04-17T22:31:10.631 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.642 7f9201ffb700 1 -- 172.21.15.102:0/2316087019 <== mon.1 v2:172.21.15.102:3301/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3194+0+0 (secure 0 0 0) 0x7f91f0014590 con 0x7f92041ae460 2024-04-17T22:31:10.631 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.643 7f920c1b9700 1 -- 172.21.15.102:0/2316087019 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92040f0800 con 0x7f92041ae460 2024-04-17T22:31:10.632 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.646 7f9201ffb700 1 -- 172.21.15.102:0/2316087019 <== mon.1 v2:172.21.15.102:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+110455 (secure 0 0 0) 0x7f91f00070d0 con 0x7f92041ae460 2024-04-17T22:31:10.737 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.751 7f920c1b9700 1 -- 172.21.15.102:0/2316087019 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f92040abc60 con 0x7f92041ae460 2024-04-17T22:31:10.737 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.751 7f9201ffb700 1 -- 172.21.15.102:0/2316087019 <== mon.1 v2:172.21.15.102:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v17) v1 ==== 104+0+11 (secure 0 0 0) 0x7f91f0000c30 con 0x7f92041ae460 2024-04-17T22:31:10.737 INFO:teuthology.orchestra.run.smithi102.stdout:pg_num: 64 2024-04-17T22:31:10.739 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.752 7f920c1b9700 1 -- 172.21.15.102:0/2316087019 >> v1:172.21.15.102:6816/42360 conn(0x7f91ec0145a0 legacy=0x7f91ec016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:10.739 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.752 7f920c1b9700 1 -- 172.21.15.102:0/2316087019 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f92041ae460 msgr2=0x7f92041b43e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:10.739 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.752 7f920c1b9700 1 --2- 172.21.15.102:0/2316087019 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f92041ae460 0x7f92041b43e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 rx=0x7f92040d7930 tx=0x7f91f000c890).stop 2024-04-17T22:31:10.739 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.752 7f920c1b9700 1 -- 172.21.15.102:0/2316087019 shutdown_connections 2024-04-17T22:31:10.739 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.752 7f920c1b9700 1 --2- 172.21.15.102:0/2316087019 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f92041b4a10 0x7f92041b4e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:10.739 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.752 7f920c1b9700 1 --2- 172.21.15.102:0/2316087019 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f92041ad9e0 0x7f92041ade30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:10.739 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.752 7f920c1b9700 1 --2- 172.21.15.102:0/2316087019 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f92041ae460 0x7f92041b43e0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:10.739 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.752 7f920c1b9700 1 -- 172.21.15.102:0/2316087019 >> 172.21.15.102:0/2316087019 conn(0x7f92040c6af0 msgr2=0x7f92041a84e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:10.739 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.752 7f920c1b9700 1 -- 172.21.15.102:0/2316087019 shutdown_connections 2024-04-17T22:31:10.739 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.752 7f920c1b9700 1 -- 172.21.15.102:0/2316087019 wait complete. 2024-04-17T22:31:10.749 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-17T22:31:10.878 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.890 7f8c815dd700 1 Processor -- start 2024-04-17T22:31:10.878 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.890 7f8c815dd700 1 -- start start 2024-04-17T22:31:10.878 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.891 7f8c815dd700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f8c7c1a64b0 0x7f8c7c1a8910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:10.878 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.891 7f8c815dd700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f8c7c1a8f40 0x7f8c7c1ab3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:10.878 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.891 7f8c815dd700 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c7c10ee90 con 0x7f8c7c10e240 2024-04-17T22:31:10.878 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.891 7f8c815dd700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c7c0f9f10 con 0x7f8c7c1a64b0 2024-04-17T22:31:10.878 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.891 7f8c815dd700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8c7c0f9030 con 0x7f8c7c1a8f40 2024-04-17T22:31:10.878 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.891 7f8c79ab0700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f8c7c1a64b0 0x7f8c7c1a8910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:10.878 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.891 7f8c7aab2700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f8c7c1a8f40 0x7f8c7c1ab3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:10.878 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.891 7f8c7a2b1700 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f8c7c10e240 0x7f8c7c10e690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:38778/0 (socket says 172.21.15.102:38778) 2024-04-17T22:31:10.879 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.891 7f8c7a2b1700 1 -- 172.21.15.102:0/2459942793 learned_addr learned my addr 172.21.15.102:0/2459942793 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-17T22:31:10.879 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.891 7f8c7aab2700 1 -- 172.21.15.102:0/2459942793 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f8c7c1a64b0 msgr2=0x7f8c7c1a8910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:31:10.879 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.891 7f8c792af700 1 -- 172.21.15.102:0/2459942793 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 978308942 0 0) 0x7f8c70001110 con 0x7f8c7c10e240 2024-04-17T22:31:10.879 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.891 7f8c7aab2700 1 --2- 172.21.15.102:0/2459942793 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f8c7c1a64b0 0x7f8c7c1a8910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:10.879 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.891 7f8c7aab2700 1 -- 172.21.15.102:0/2459942793 >> v1:172.21.15.102:6789/0 conn(0x7f8c7c10e240 legacy=0x7f8c7c10e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:10.879 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.891 7f8c7aab2700 1 -- 172.21.15.102:0/2459942793 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c7c1ac9e0 con 0x7f8c7c1a8f40 2024-04-17T22:31:10.879 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.891 7f8c79ab0700 1 --2- 172.21.15.102:0/2459942793 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f8c7c1a64b0 0x7f8c7c1a8910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:10.879 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.891 7f8c7aab2700 1 --2- 172.21.15.102:0/2459942793 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f8c7c1a8f40 0x7f8c7c1ab3a0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 rx=0x7f8c7c0f95d0 tx=0x7f8c74009cf0).ready entity=mon.2 client_cookie=b038a41dec5d8c9f server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:10.879 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.891 7f8c792af700 1 -- 172.21.15.102:0/2459942793 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8c7c1ac9e0 con 0x7f8c7c1a8f40 2024-04-17T22:31:10.879 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.891 7f8c792af700 1 -- 172.21.15.102:0/2459942793 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c7400b840 con 0x7f8c7c1a8f40 2024-04-17T22:31:10.879 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.891 7f8c792af700 1 -- 172.21.15.102:0/2459942793 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8c7400ba20 con 0x7f8c7c1a8f40 2024-04-17T22:31:10.879 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.891 7f8c815dd700 1 -- 172.21.15.102:0/2459942793 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f8c7c1a8f40 msgr2=0x7f8c7c1ab3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:10.879 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.891 7f8c815dd700 1 --2- 172.21.15.102:0/2459942793 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f8c7c1a8f40 0x7f8c7c1ab3a0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 rx=0x7f8c7c0f95d0 tx=0x7f8c74009cf0).stop 2024-04-17T22:31:10.880 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.892 7f8c815dd700 1 -- 172.21.15.102:0/2459942793 shutdown_connections 2024-04-17T22:31:10.880 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.892 7f8c815dd700 1 --2- 172.21.15.102:0/2459942793 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f8c7c1a8f40 0x7f8c7c1ab3a0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:10.880 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.892 7f8c815dd700 1 --2- 172.21.15.102:0/2459942793 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f8c7c1a64b0 0x7f8c7c1a8910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:10.880 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.892 7f8c815dd700 1 -- 172.21.15.102:0/2459942793 >> 172.21.15.102:0/2459942793 conn(0x7f8c7c1138a0 msgr2=0x7f8c7c0f4f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:10.880 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.892 7f8c815dd700 1 -- 172.21.15.102:0/2459942793 shutdown_connections 2024-04-17T22:31:10.880 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.892 7f8c815dd700 1 -- 172.21.15.102:0/2459942793 wait complete. 2024-04-17T22:31:10.903 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.915 7f8c815dd700 1 Processor -- start 2024-04-17T22:31:10.903 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.915 7f8c815dd700 1 -- start start 2024-04-17T22:31:10.903 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.915 7f8c815dd700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8c7c0cc360 0x7f8c7c0d51d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:10.903 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.915 7f8c815dd700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f8c7c0d5800 0x7f8c7c1b42a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:10.903 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.915 7f8c815dd700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f8c7c0d5c50 0x7f8c7c1b48d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:10.904 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.915 7f8c815dd700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c7c0fa1b0 con 0x7f8c7c0cc360 2024-04-17T22:31:10.904 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.915 7f8c815dd700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c7c0daa80 con 0x7f8c7c0d5c50 2024-04-17T22:31:10.904 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.915 7f8c815dd700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8c7c0cbca0 con 0x7f8c7c0d5800 2024-04-17T22:31:10.904 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.915 7f8c7a2b1700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8c7c0cc360 0x7f8c7c0d51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:10.904 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.915 7f8c7aab2700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f8c7c0d5c50 0x7f8c7c1b48d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:10.904 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.915 7f8c7a2b1700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8c7c0cc360 0x7f8c7c0d51d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:54280/0 (socket says 172.21.15.102:54280) 2024-04-17T22:31:10.904 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.915 7f8c79ab0700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f8c7c0d5800 0x7f8c7c1b42a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:10.904 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.915 7f8c7a2b1700 1 -- 172.21.15.102:0/2903473258 learned_addr learned my addr 172.21.15.102:0/2903473258 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:31:10.904 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.916 7f8c7aab2700 1 -- 172.21.15.102:0/2903473258 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f8c7c0d5800 msgr2=0x7f8c7c1b42a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:10.904 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.916 7f8c7aab2700 1 --2- 172.21.15.102:0/2903473258 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f8c7c0d5800 0x7f8c7c1b42a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:10.904 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.916 7f8c7aab2700 1 -- 172.21.15.102:0/2903473258 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8c7c0cc360 msgr2=0x7f8c7c0d51d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:10.904 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.916 7f8c7aab2700 1 --2- 172.21.15.102:0/2903473258 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8c7c0cc360 0x7f8c7c0d51d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:10.904 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.916 7f8c7aab2700 1 -- 172.21.15.102:0/2903473258 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c7c1b4f00 con 0x7f8c7c0d5c50 2024-04-17T22:31:10.904 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.916 7f8c79ab0700 1 --2- 172.21.15.102:0/2903473258 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f8c7c0d5800 0x7f8c7c1b42a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:10.905 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.916 7f8c7a2b1700 1 --2- 172.21.15.102:0/2903473258 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8c7c0cc360 0x7f8c7c0d51d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:10.905 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.916 7f8c7aab2700 1 --2- 172.21.15.102:0/2903473258 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f8c7c0d5c50 0x7f8c7c1b48d0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 rx=0x7f8c74006fc0 tx=0x7f8c74006c40).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:10.905 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.916 7f8c6b7fe700 1 -- 172.21.15.102:0/2903473258 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8c7c1b4f00 con 0x7f8c7c0d5c50 2024-04-17T22:31:10.905 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.916 7f8c6b7fe700 1 -- 172.21.15.102:0/2903473258 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c74011b60 con 0x7f8c7c0d5c50 2024-04-17T22:31:10.905 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.916 7f8c6b7fe700 1 -- 172.21.15.102:0/2903473258 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8c74017040 con 0x7f8c7c0d5c50 2024-04-17T22:31:10.905 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.916 7f8c815dd700 1 -- 172.21.15.102:0/2903473258 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c7c1b51c0 con 0x7f8c7c0d5c50 2024-04-17T22:31:10.905 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.916 7f8c815dd700 1 -- 172.21.15.102:0/2903473258 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c7c1bbbf0 con 0x7f8c7c0d5c50 2024-04-17T22:31:10.905 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.917 7f8c6b7fe700 1 -- 172.21.15.102:0/2903473258 <== mon.1 v2:172.21.15.102:3301/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7f8c7401c470 con 0x7f8c7c0d5c50 2024-04-17T22:31:10.905 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.917 7f8c6b7fe700 1 -- 172.21.15.102:0/2903473258 <== mon.1 v2:172.21.15.102:3301/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3194+0+0 (secure 0 0 0) 0x7f8c74018030 con 0x7f8c7c0d5c50 2024-04-17T22:31:10.905 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.917 7f8c815dd700 1 -- 172.21.15.102:0/2903473258 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c7c0f0800 con 0x7f8c7c0d5c50 2024-04-17T22:31:10.906 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:10.920 7f8c6b7fe700 1 -- 172.21.15.102:0/2903473258 <== mon.1 v2:172.21.15.102:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+110455 (secure 0 0 0) 0x7f8c74011950 con 0x7f8c7c0d5c50 2024-04-17T22:31:11.011 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.025 7f8c815dd700 1 -- 172.21.15.102:0/2903473258 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f8c7c1ae9c0 con 0x7f8c7c0d5c50 2024-04-17T22:31:11.012 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.026 7f8c6b7fe700 1 -- 172.21.15.102:0/2903473258 <== mon.1 v2:172.21.15.102:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v17) v1 ==== 100+0+11 (secure 0 0 0) 0x7f8c7400b630 con 0x7f8c7c0d5c50 2024-04-17T22:31:11.012 INFO:teuthology.orchestra.run.smithi102.stdout:pg_num: 64 2024-04-17T22:31:11.013 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.027 7f8c815dd700 1 -- 172.21.15.102:0/2903473258 >> v1:172.21.15.102:6816/42360 conn(0x7f8c580145a0 legacy=0x7f8c58016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:11.013 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.027 7f8c815dd700 1 -- 172.21.15.102:0/2903473258 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f8c7c0d5c50 msgr2=0x7f8c7c1b48d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:11.013 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.027 7f8c815dd700 1 --2- 172.21.15.102:0/2903473258 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f8c7c0d5c50 0x7f8c7c1b48d0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 rx=0x7f8c74006fc0 tx=0x7f8c74006c40).stop 2024-04-17T22:31:11.013 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.027 7f8c815dd700 1 -- 172.21.15.102:0/2903473258 shutdown_connections 2024-04-17T22:31:11.013 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.027 7f8c815dd700 1 --2- 172.21.15.102:0/2903473258 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f8c7c0d5c50 0x7f8c7c1b48d0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:11.013 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.027 7f8c815dd700 1 --2- 172.21.15.102:0/2903473258 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8c7c0cc360 0x7f8c7c0d51d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:11.013 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.027 7f8c815dd700 1 --2- 172.21.15.102:0/2903473258 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f8c7c0d5800 0x7f8c7c1b42a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:11.014 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.027 7f8c815dd700 1 -- 172.21.15.102:0/2903473258 >> 172.21.15.102:0/2903473258 conn(0x7f8c7c0c5630 msgr2=0x7f8c7c0d62b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:11.014 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.027 7f8c815dd700 1 -- 172.21.15.102:0/2903473258 shutdown_connections 2024-04-17T22:31:11.014 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.027 7f8c815dd700 1 -- 172.21.15.102:0/2903473258 wait complete. 2024-04-17T22:31:11.024 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-17T22:31:11.152 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.163 7f01f6c8c700 1 Processor -- start 2024-04-17T22:31:11.152 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.163 7f01f6c8c700 1 -- start start 2024-04-17T22:31:11.152 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.163 7f01f6c8c700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f01f00f5750 0x7f01f010c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:11.152 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.163 7f01f6c8c700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f01f010c720 0x7f01f01a0be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:11.153 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.164 7f01f6c8c700 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01f01a1a10 con 0x7f01f010cb70 2024-04-17T22:31:11.153 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.164 7f01f6c8c700 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01f00f93c0 con 0x7f01f010c720 2024-04-17T22:31:11.153 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.164 7f01f6c8c700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01f01a1c30 con 0x7f01f00f5750 2024-04-17T22:31:11.153 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.164 7f01eeffd700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f01f010c720 0x7f01f01a0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:11.154 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.164 7f01eeffd700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f01f010c720 0x7f01f01a0be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:54296/0 (socket says 172.21.15.102:54296) 2024-04-17T22:31:11.154 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.164 7f01effff700 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f01f010cb70 0x7f01f01a1210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:38780/0 (socket says 172.21.15.102:38780) 2024-04-17T22:31:11.154 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.164 7f01eeffd700 1 -- 172.21.15.102:0/3624574742 learned_addr learned my addr 172.21.15.102:0/3624574742 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:31:11.154 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.164 7f01ef7fe700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f01f00f5750 0x7f01f010c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:11.154 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.164 7f01ef7fe700 1 -- 172.21.15.102:0/3624574742 >> v2:172.21.15.102:3300/0 conn(0x7f01f010c720 msgr2=0x7f01f01a0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:31:11.154 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.164 7f01ef7fe700 1 --2- 172.21.15.102:0/3624574742 >> v2:172.21.15.102:3300/0 conn(0x7f01f010c720 0x7f01f01a0be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:11.154 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.164 7f01ef7fe700 1 -- 172.21.15.102:0/3624574742 >> v1:172.21.15.102:6789/0 conn(0x7f01f010cb70 legacy=0x7f01f01a1210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:11.154 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.164 7f01ef7fe700 1 -- 172.21.15.102:0/3624574742 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01f01aa310 con 0x7f01f00f5750 2024-04-17T22:31:11.154 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.164 7f01ee7fc700 1 -- 172.21.15.102:0/3624574742 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3004073053 0 0) 0x7f01e0001490 con 0x7f01f010cb70 2024-04-17T22:31:11.155 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.164 7f01eeffd700 1 --2- 172.21.15.102:0/3624574742 >> v2:172.21.15.102:3300/0 conn(0x7f01f010c720 0x7f01f01a0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:11.155 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.164 7f01ef7fe700 1 --2- 172.21.15.102:0/3624574742 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f01f00f5750 0x7f01f010c0f0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 rx=0x7f01e8002f70 tx=0x7f01e800b970).ready entity=mon.1 client_cookie=86f04780760dc29a server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:11.155 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.164 7f01ee7fc700 1 -- 172.21.15.102:0/3624574742 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f01f01aa310 con 0x7f01f00f5750 2024-04-17T22:31:11.155 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.164 7f01ee7fc700 1 -- 172.21.15.102:0/3624574742 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01e800d610 con 0x7f01f00f5750 2024-04-17T22:31:11.155 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.164 7f01ee7fc700 1 -- 172.21.15.102:0/3624574742 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f01e800d9e0 con 0x7f01f00f5750 2024-04-17T22:31:11.155 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.164 7f01f6c8c700 1 -- 172.21.15.102:0/3624574742 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f01f00f5750 msgr2=0x7f01f010c0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:11.155 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.164 7f01f6c8c700 1 --2- 172.21.15.102:0/3624574742 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f01f00f5750 0x7f01f010c0f0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 rx=0x7f01e8002f70 tx=0x7f01e800b970).stop 2024-04-17T22:31:11.155 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.165 7f01f6c8c700 1 -- 172.21.15.102:0/3624574742 shutdown_connections 2024-04-17T22:31:11.155 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.165 7f01f6c8c700 1 --2- 172.21.15.102:0/3624574742 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f01f00f5750 0x7f01f010c0f0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:11.157 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.165 7f01f6c8c700 1 --2- 172.21.15.102:0/3624574742 >> v2:172.21.15.102:3300/0 conn(0x7f01f010c720 0x7f01f01a0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:11.157 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.165 7f01f6c8c700 1 -- 172.21.15.102:0/3624574742 >> 172.21.15.102:0/3624574742 conn(0x7f01f01138a0 msgr2=0x7f01f00f4f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:11.157 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.165 7f01f6c8c700 1 -- 172.21.15.102:0/3624574742 shutdown_connections 2024-04-17T22:31:11.157 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.165 7f01f6c8c700 1 -- 172.21.15.102:0/3624574742 wait complete. 2024-04-17T22:31:11.174 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.188 7f01f6c8c700 1 Processor -- start 2024-04-17T22:31:11.174 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.188 7f01f6c8c700 1 -- start start 2024-04-17T22:31:11.176 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.188 7f01f6c8c700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f01f00d0030 0x7f01f01a8090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:11.176 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.188 7f01f6c8c700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f01f01a86c0 0x7f01f01b3ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:11.176 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.188 7f01f6c8c700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f01f01a8b10 0x7f01f01b4500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:11.176 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.188 7f01f6c8c700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01f00d88e0 con 0x7f01f00d0030 2024-04-17T22:31:11.176 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.188 7f01f6c8c700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01f01b4b30 con 0x7f01f01a86c0 2024-04-17T22:31:11.177 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.188 7f01f6c8c700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f01f01b4d60 con 0x7f01f01a8b10 2024-04-17T22:31:11.177 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.188 7f01eeffd700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f01f01a86c0 0x7f01f01b3ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:11.177 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.188 7f01effff700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f01f01a8b10 0x7f01f01b4500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:11.177 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.188 7f01eeffd700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f01f01a86c0 0x7f01f01b3ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:52198/0 (socket says 172.21.15.102:52198) 2024-04-17T22:31:11.177 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.188 7f01eeffd700 1 -- 172.21.15.102:0/3659412772 learned_addr learned my addr 172.21.15.102:0/3659412772 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:31:11.177 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.188 7f01ef7fe700 1 --2- 172.21.15.102:0/3659412772 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f01f00d0030 0x7f01f01a8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:11.177 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.189 7f01eeffd700 1 -- 172.21.15.102:0/3659412772 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f01f01a8b10 msgr2=0x7f01f01b4500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:11.177 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.189 7f01eeffd700 1 --2- 172.21.15.102:0/3659412772 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f01f01a8b10 0x7f01f01b4500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:11.177 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.189 7f01eeffd700 1 -- 172.21.15.102:0/3659412772 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f01f00d0030 msgr2=0x7f01f01a8090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:11.177 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.189 7f01eeffd700 1 --2- 172.21.15.102:0/3659412772 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f01f00d0030 0x7f01f01a8090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:11.177 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.189 7f01eeffd700 1 -- 172.21.15.102:0/3659412772 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01f01bb640 con 0x7f01f01a86c0 2024-04-17T22:31:11.177 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.189 7f01effff700 1 --2- 172.21.15.102:0/3659412772 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f01f01a8b10 0x7f01f01b4500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:11.178 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.189 7f01ef7fe700 1 --2- 172.21.15.102:0/3659412772 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f01f00d0030 0x7f01f01a8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:11.178 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.189 7f01eeffd700 1 --2- 172.21.15.102:0/3659412772 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f01f01a86c0 0x7f01f01b3ed0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 rx=0x7f01d800c000 tx=0x7f01d800eca0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:11.178 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.189 7f01ecff9700 1 -- 172.21.15.102:0/3659412772 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f01f01bb640 con 0x7f01f01a86c0 2024-04-17T22:31:11.178 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.189 7f01ecff9700 1 -- 172.21.15.102:0/3659412772 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01d800c5a0 con 0x7f01f01a86c0 2024-04-17T22:31:11.178 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.189 7f01ecff9700 1 -- 172.21.15.102:0/3659412772 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f01d800f040 con 0x7f01f01a86c0 2024-04-17T22:31:11.178 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.189 7f01f6c8c700 1 -- 172.21.15.102:0/3659412772 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01f01bb840 con 0x7f01f01a86c0 2024-04-17T22:31:11.178 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.189 7f01f6c8c700 1 -- 172.21.15.102:0/3659412772 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01f01bba40 con 0x7f01f01a86c0 2024-04-17T22:31:11.178 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.189 7f01ecff9700 1 -- 172.21.15.102:0/3659412772 <== mon.1 v2:172.21.15.102:3301/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7f01d8022060 con 0x7f01f01a86c0 2024-04-17T22:31:11.178 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.189 7f01ecff9700 1 -- 172.21.15.102:0/3659412772 <== mon.1 v2:172.21.15.102:3301/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3194+0+0 (secure 0 0 0) 0x7f01d8010040 con 0x7f01f01a86c0 2024-04-17T22:31:11.178 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.190 7f01f6c8c700 1 -- 172.21.15.102:0/3659412772 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01f00f0800 con 0x7f01f01a86c0 2024-04-17T22:31:11.179 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.193 7f01ecff9700 1 -- 172.21.15.102:0/3659412772 <== mon.1 v2:172.21.15.102:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+110455 (secure 0 0 0) 0x7f01d800c970 con 0x7f01f01a86c0 2024-04-17T22:31:11.324 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.337 7f01f6c8c700 1 -- 172.21.15.102:0/3659412772 >> v1:172.21.15.102:6816/42360 conn(0x7f01c80145a0 legacy=0x7f01c8016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:11.324 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.337 7f01f6c8c700 1 -- 172.21.15.102:0/3659412772 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f01f01a86c0 msgr2=0x7f01f01b3ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:11.324 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.337 7f01f6c8c700 1 --2- 172.21.15.102:0/3659412772 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f01f01a86c0 0x7f01f01b3ed0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 rx=0x7f01d800c000 tx=0x7f01d800eca0).stop 2024-04-17T22:31:11.324 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.338 7f01f6c8c700 1 -- 172.21.15.102:0/3659412772 shutdown_connections 2024-04-17T22:31:11.325 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.338 7f01f6c8c700 1 --2- 172.21.15.102:0/3659412772 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f01f01a8b10 0x7f01f01b4500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:11.325 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.338 7f01f6c8c700 1 --2- 172.21.15.102:0/3659412772 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f01f00d0030 0x7f01f01a8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:11.325 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.338 7f01f6c8c700 1 --2- 172.21.15.102:0/3659412772 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f01f01a86c0 0x7f01f01b3ed0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:11.325 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.338 7f01f6c8c700 1 -- 172.21.15.102:0/3659412772 >> 172.21.15.102:0/3659412772 conn(0x7f01f00d1630 msgr2=0x7f01f00c4c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:11.325 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.338 7f01f6c8c700 1 -- 172.21.15.102:0/3659412772 shutdown_connections 2024-04-17T22:31:11.325 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.338 7f01f6c8c700 1 -- 172.21.15.102:0/3659412772 wait complete. 2024-04-17T22:31:11.325 INFO:teuthology.orchestra.run.smithi102.stderr:no valid command found; 10 closest matches: 2024-04-17T22:31:11.325 INFO:teuthology.orchestra.run.smithi102.stderr:osd perf 2024-04-17T22:31:11.326 INFO:teuthology.orchestra.run.smithi102.stderr:osd df {plain|tree} {class|name} {} 2024-04-17T22:31:11.326 INFO:teuthology.orchestra.run.smithi102.stderr:osd blocked-by 2024-04-17T22:31:11.326 INFO:teuthology.orchestra.run.smithi102.stderr:osd pool stats {} 2024-04-17T22:31:11.326 INFO:teuthology.orchestra.run.smithi102.stderr:osd pool scrub [...] 2024-04-17T22:31:11.326 INFO:teuthology.orchestra.run.smithi102.stderr:osd pool deep-scrub [...] 2024-04-17T22:31:11.326 INFO:teuthology.orchestra.run.smithi102.stderr:osd pool repair [...] 2024-04-17T22:31:11.326 INFO:teuthology.orchestra.run.smithi102.stderr:osd pool force-recovery [...] 2024-04-17T22:31:11.326 INFO:teuthology.orchestra.run.smithi102.stderr:osd pool force-backfill [...] 2024-04-17T22:31:11.326 INFO:teuthology.orchestra.run.smithi102.stderr:osd pool cancel-force-recovery [...] 2024-04-17T22:31:11.326 INFO:teuthology.orchestra.run.smithi102.stderr:Error EINVAL: invalid command 2024-04-17T22:31:11.334 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-17T22:31:11.335 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blacklist ls 2024-04-17T22:31:11.464 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.476 7f3c7b1cb700 1 Processor -- start 2024-04-17T22:31:11.464 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.476 7f3c7b1cb700 1 -- start start 2024-04-17T22:31:11.464 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.476 7f3c7b1cb700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f3c740f5840 0x7f3c7410b010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:11.464 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.476 7f3c7b1cb700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f3c7410ba90 0x7f3c741a02f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:11.464 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.476 7f3c7b1cb700 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c74103b40 con 0x7f3c7410b640 2024-04-17T22:31:11.465 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.476 7f3c7b1cb700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c7410bee0 con 0x7f3c7410ba90 2024-04-17T22:31:11.465 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.476 7f3c7b1cb700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3c740fa370 con 0x7f3c740f5840 2024-04-17T22:31:11.465 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.476 7f3c737fe700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f3c740f5840 0x7f3c7410b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:11.465 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.476 7f3c73fff700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f3c7410ba90 0x7f3c741a02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:11.465 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.476 7f3c737fe700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f3c740f5840 0x7f3c7410b010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:55090/0 (socket says 172.21.15.102:55090) 2024-04-17T22:31:11.465 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.476 7f3c73fff700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f3c7410ba90 0x7f3c741a02f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:52202/0 (socket says 172.21.15.102:52202) 2024-04-17T22:31:11.465 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.476 7f3c737fe700 1 -- 172.21.15.102:0/2589073382 learned_addr learned my addr 172.21.15.102:0/2589073382 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:31:11.465 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.477 7f3c737fe700 1 -- 172.21.15.102:0/2589073382 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f3c7410ba90 msgr2=0x7f3c741a02f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:31:11.465 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.477 7f3c737fe700 1 --2- 172.21.15.102:0/2589073382 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f3c7410ba90 0x7f3c741a02f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:11.465 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.477 7f3c737fe700 1 -- 172.21.15.102:0/2589073382 >> v1:172.21.15.102:6789/0 conn(0x7f3c7410b640 legacy=0x7f3c7419faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:11.465 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.477 7f3c727fc700 1 -- 172.21.15.102:0/2589073382 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2679941691 0 0) 0x7f3c64001110 con 0x7f3c7410b640 2024-04-17T22:31:11.465 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.477 7f3c737fe700 1 -- 172.21.15.102:0/2589073382 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c741a0920 con 0x7f3c740f5840 2024-04-17T22:31:11.465 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.477 7f3c737fe700 1 --2- 172.21.15.102:0/2589073382 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f3c740f5840 0x7f3c7410b010 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 rx=0x7f3c740f46a0 tx=0x7f3c68009a30).ready entity=mon.2 client_cookie=b25886e5daa1749d server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:11.465 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.477 7f3c727fc700 1 -- 172.21.15.102:0/2589073382 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f3c741a0920 con 0x7f3c740f5840 2024-04-17T22:31:11.466 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.477 7f3c727fc700 1 -- 172.21.15.102:0/2589073382 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c6800f510 con 0x7f3c740f5840 2024-04-17T22:31:11.466 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.477 7f3c727fc700 1 -- 172.21.15.102:0/2589073382 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f3c6800f6f0 con 0x7f3c740f5840 2024-04-17T22:31:11.466 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.477 7f3c7b1cb700 1 -- 172.21.15.102:0/2589073382 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f3c740f5840 msgr2=0x7f3c7410b010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:11.466 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.477 7f3c7b1cb700 1 --2- 172.21.15.102:0/2589073382 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f3c740f5840 0x7f3c7410b010 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 rx=0x7f3c740f46a0 tx=0x7f3c68009a30).stop 2024-04-17T22:31:11.466 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.477 7f3c7b1cb700 1 -- 172.21.15.102:0/2589073382 shutdown_connections 2024-04-17T22:31:11.466 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.477 7f3c7b1cb700 1 --2- 172.21.15.102:0/2589073382 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f3c7410ba90 0x7f3c741a02f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:11.466 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.477 7f3c7b1cb700 1 --2- 172.21.15.102:0/2589073382 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f3c740f5840 0x7f3c7410b010 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:11.466 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.477 7f3c7b1cb700 1 -- 172.21.15.102:0/2589073382 >> 172.21.15.102:0/2589073382 conn(0x7f3c741138a0 msgr2=0x7f3c740f4f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:11.466 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.477 7f3c7b1cb700 1 -- 172.21.15.102:0/2589073382 shutdown_connections 2024-04-17T22:31:11.466 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.477 7f3c7b1cb700 1 -- 172.21.15.102:0/2589073382 wait complete. 2024-04-17T22:31:11.486 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.500 7f3c7b1cb700 1 Processor -- start 2024-04-17T22:31:11.486 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.500 7f3c7b1cb700 1 -- start start 2024-04-17T22:31:11.488 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.500 7f3c7b1cb700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f3c740ca5e0 0x7f3c740f17b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:11.489 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.500 7f3c7b1cb700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f3c740caa30 0x7f3c740f1de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:11.489 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.500 7f3c7b1cb700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3c741ad420 0x7f3c741ad870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:11.489 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.500 7f3c7b1cb700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c740e2ce0 con 0x7f3c741ad420 2024-04-17T22:31:11.489 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.500 7f3c7b1cb700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c740f2410 con 0x7f3c740ca5e0 2024-04-17T22:31:11.489 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.500 7f3c7b1cb700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3c741adea0 con 0x7f3c740caa30 2024-04-17T22:31:11.489 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.500 7f3c737fe700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f3c740ca5e0 0x7f3c740f17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:11.489 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.501 7f3c737fe700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f3c740ca5e0 0x7f3c740f17b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:52212/0 (socket says 172.21.15.102:52212) 2024-04-17T22:31:11.489 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.501 7f3c737fe700 1 -- 172.21.15.102:0/985829021 learned_addr learned my addr 172.21.15.102:0/985829021 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:31:11.489 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.501 7f3c72ffd700 1 --2- 172.21.15.102:0/985829021 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f3c740caa30 0x7f3c740f1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:11.489 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.501 7f3c73fff700 1 --2- 172.21.15.102:0/985829021 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3c741ad420 0x7f3c741ad870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:11.489 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.501 7f3c737fe700 1 -- 172.21.15.102:0/985829021 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f3c740caa30 msgr2=0x7f3c740f1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:11.489 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.501 7f3c737fe700 1 --2- 172.21.15.102:0/985829021 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f3c740caa30 0x7f3c740f1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:11.490 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.501 7f3c737fe700 1 -- 172.21.15.102:0/985829021 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3c741ad420 msgr2=0x7f3c741ad870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:11.490 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.501 7f3c737fe700 1 --2- 172.21.15.102:0/985829021 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3c741ad420 0x7f3c741ad870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:11.490 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.501 7f3c737fe700 1 -- 172.21.15.102:0/985829021 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c741ae0d0 con 0x7f3c740ca5e0 2024-04-17T22:31:11.490 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.501 7f3c72ffd700 1 --2- 172.21.15.102:0/985829021 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f3c740caa30 0x7f3c740f1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:11.490 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.501 7f3c737fe700 1 --2- 172.21.15.102:0/985829021 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f3c740ca5e0 0x7f3c740f17b0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 rx=0x7f3c740c3d10 tx=0x7f3c680068c0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:11.490 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.501 7f3c70ff9700 1 -- 172.21.15.102:0/985829021 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f3c741ae0d0 con 0x7f3c740ca5e0 2024-04-17T22:31:11.490 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.501 7f3c70ff9700 1 -- 172.21.15.102:0/985829021 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c680119f0 con 0x7f3c740ca5e0 2024-04-17T22:31:11.490 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.501 7f3c7b1cb700 1 -- 172.21.15.102:0/985829021 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c741bb510 con 0x7f3c740ca5e0 2024-04-17T22:31:11.490 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.501 7f3c70ff9700 1 -- 172.21.15.102:0/985829021 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f3c68011dc0 con 0x7f3c740ca5e0 2024-04-17T22:31:11.490 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.501 7f3c7b1cb700 1 -- 172.21.15.102:0/985829021 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c741bb8f0 con 0x7f3c740ca5e0 2024-04-17T22:31:11.490 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.502 7f3c70ff9700 1 -- 172.21.15.102:0/985829021 <== mon.1 v2:172.21.15.102:3301/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7f3c68027460 con 0x7f3c740ca5e0 2024-04-17T22:31:11.490 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.502 7f3c70ff9700 1 -- 172.21.15.102:0/985829021 <== mon.1 v2:172.21.15.102:3301/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3194+0+0 (secure 0 0 0) 0x7f3c68001b40 con 0x7f3c740ca5e0 2024-04-17T22:31:11.490 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.502 7f3c7b1cb700 1 -- 172.21.15.102:0/985829021 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c740c4140 con 0x7f3c740ca5e0 2024-04-17T22:31:11.491 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.505 7f3c70ff9700 1 -- 172.21.15.102:0/985829021 <== mon.1 v2:172.21.15.102:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+110455 (secure 0 0 0) 0x7f3c680117e0 con 0x7f3c740ca5e0 2024-04-17T22:31:11.608 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.622 7f3c7b1cb700 1 -- 172.21.15.102:0/985829021 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "osd blacklist ls"} v 0) v1 -- 0x7f3c740f0800 con 0x7f3c740ca5e0 2024-04-17T22:31:11.609 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.623 7f3c70ff9700 1 -- 172.21.15.102:0/985829021 <== mon.1 v2:172.21.15.102:3301/0 7 ==== mon_command_ack([{"prefix": "osd blacklist ls"}]=0 listed 0 entries v17) v1 ==== 80+0+0 (secure 0 0 0) 0x7f3c6800f300 con 0x7f3c740ca5e0 2024-04-17T22:31:11.610 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.624 7f3c7b1cb700 1 -- 172.21.15.102:0/985829021 >> v1:172.21.15.102:6816/42360 conn(0x7f3c540145a0 legacy=0x7f3c54016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:11.610 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.624 7f3c7b1cb700 1 -- 172.21.15.102:0/985829021 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f3c740ca5e0 msgr2=0x7f3c740f17b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:11.610 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.624 7f3c7b1cb700 1 --2- 172.21.15.102:0/985829021 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f3c740ca5e0 0x7f3c740f17b0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 rx=0x7f3c740c3d10 tx=0x7f3c680068c0).stop 2024-04-17T22:31:11.610 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.624 7f3c7b1cb700 1 -- 172.21.15.102:0/985829021 shutdown_connections 2024-04-17T22:31:11.611 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.624 7f3c7b1cb700 1 --2- 172.21.15.102:0/985829021 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3c741ad420 0x7f3c741ad870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:11.611 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.624 7f3c7b1cb700 1 --2- 172.21.15.102:0/985829021 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f3c740ca5e0 0x7f3c740f17b0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:11.611 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.624 7f3c7b1cb700 1 --2- 172.21.15.102:0/985829021 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f3c740caa30 0x7f3c740f1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:11.611 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.624 7f3c7b1cb700 1 -- 172.21.15.102:0/985829021 >> 172.21.15.102:0/985829021 conn(0x7f3c740e0dc0 msgr2=0x7f3c740daae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:11.611 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.624 7f3c7b1cb700 1 -- 172.21.15.102:0/985829021 shutdown_connections 2024-04-17T22:31:11.611 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.624 7f3c7b1cb700 1 -- 172.21.15.102:0/985829021 wait complete. 2024-04-17T22:31:11.611 INFO:teuthology.orchestra.run.smithi102.stderr:listed 0 entries 2024-04-17T22:31:11.621 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-04-17T22:31:11.621 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T22:31:11.643 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T22:31:11.769 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.781 7f7ca6e35700 1 Processor -- start 2024-04-17T22:31:11.769 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.781 7f7ca6e35700 1 -- start start 2024-04-17T22:31:11.769 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.782 7f7ca6e35700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f7ca00f5770 0x7f7ca010c110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:11.769 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.782 7f7ca6e35700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f7ca010cb90 0x7f7ca01a1400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:11.769 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.782 7f7ca6e35700 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ca01a1a30 con 0x7f7ca010c740 2024-04-17T22:31:11.769 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.782 7f7ca6e35700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ca00f93e0 con 0x7f7ca00f5770 2024-04-17T22:31:11.769 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.782 7f7ca6e35700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7ca01a1c50 con 0x7f7ca010cb90 2024-04-17T22:31:11.770 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.782 7f7c9ffff700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f7ca010cb90 0x7f7ca01a1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:11.770 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.782 7f7c9f7fe700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f7ca00f5770 0x7f7ca010c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:11.770 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.782 7f7c97fff700 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f7ca010c740 0x7f7ca01a0c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:38812/0 (socket says 172.21.15.102:38812) 2024-04-17T22:31:11.770 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.782 7f7c97fff700 1 -- 172.21.15.102:0/2910213203 learned_addr learned my addr 172.21.15.102:0/2910213203 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-17T22:31:11.770 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.782 7f7c9ffff700 1 -- 172.21.15.102:0/2910213203 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f7ca00f5770 msgr2=0x7f7ca010c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:31:11.770 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.782 7f7c9effd700 1 -- 172.21.15.102:0/2910213203 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3932712013 0 0) 0x7f7c88001110 con 0x7f7ca010c740 2024-04-17T22:31:11.770 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.782 7f7c9ffff700 1 --2- 172.21.15.102:0/2910213203 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f7ca00f5770 0x7f7ca010c110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:11.770 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.782 7f7c9ffff700 1 -- 172.21.15.102:0/2910213203 >> v1:172.21.15.102:6789/0 conn(0x7f7ca010c740 legacy=0x7f7ca01a0c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:11.770 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.782 7f7c9ffff700 1 -- 172.21.15.102:0/2910213203 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ca01aa330 con 0x7f7ca010cb90 2024-04-17T22:31:11.770 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.782 7f7c9f7fe700 1 --2- 172.21.15.102:0/2910213203 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f7ca00f5770 0x7f7ca010c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:11.770 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.782 7f7c9ffff700 1 --2- 172.21.15.102:0/2910213203 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f7ca010cb90 0x7f7ca01a1400 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0x7f7ca0089f10 tx=0x7f7c98009cf0).ready entity=mon.2 client_cookie=60b7b5bef71cb004 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:11.770 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.782 7f7c9effd700 1 -- 172.21.15.102:0/2910213203 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f7ca01aa330 con 0x7f7ca010cb90 2024-04-17T22:31:11.770 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.782 7f7c9effd700 1 -- 172.21.15.102:0/2910213203 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c9800b840 con 0x7f7ca010cb90 2024-04-17T22:31:11.771 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.782 7f7c9effd700 1 -- 172.21.15.102:0/2910213203 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f7c9800ba20 con 0x7f7ca010cb90 2024-04-17T22:31:11.771 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.782 7f7ca6e35700 1 -- 172.21.15.102:0/2910213203 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f7ca010cb90 msgr2=0x7f7ca01a1400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:11.771 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.782 7f7ca6e35700 1 --2- 172.21.15.102:0/2910213203 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f7ca010cb90 0x7f7ca01a1400 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0x7f7ca0089f10 tx=0x7f7c98009cf0).stop 2024-04-17T22:31:11.771 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.783 7f7ca6e35700 1 -- 172.21.15.102:0/2910213203 shutdown_connections 2024-04-17T22:31:11.771 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.783 7f7ca6e35700 1 --2- 172.21.15.102:0/2910213203 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f7ca010cb90 0x7f7ca01a1400 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:11.771 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.783 7f7ca6e35700 1 --2- 172.21.15.102:0/2910213203 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f7ca00f5770 0x7f7ca010c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:11.771 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.783 7f7ca6e35700 1 -- 172.21.15.102:0/2910213203 >> 172.21.15.102:0/2910213203 conn(0x7f7ca01138c0 msgr2=0x7f7ca00f4f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:11.771 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.783 7f7ca6e35700 1 -- 172.21.15.102:0/2910213203 shutdown_connections 2024-04-17T22:31:11.772 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.783 7f7ca6e35700 1 -- 172.21.15.102:0/2910213203 wait complete. 2024-04-17T22:31:11.794 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.806 7f7ca6e35700 1 Processor -- start 2024-04-17T22:31:11.794 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.806 7f7ca6e35700 1 -- start start 2024-04-17T22:31:11.794 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.806 7f7ca6e35700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f7ca00cda10 0x7f7ca00d5e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:11.794 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.806 7f7ca6e35700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f7ca00d64c0 0x7f7ca01b1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:11.794 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.806 7f7ca6e35700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7ca00d6910 0x7f7ca01b21f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:11.794 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.806 7f7ca6e35700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ca00e2ce0 con 0x7f7ca00d6910 2024-04-17T22:31:11.794 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.806 7f7ca6e35700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ca00ccdb0 con 0x7f7ca00cda10 2024-04-17T22:31:11.794 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.806 7f7ca6e35700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7ca00cd090 con 0x7f7ca00d64c0 2024-04-17T22:31:11.794 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.806 7f7c97fff700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f7ca00d64c0 0x7f7ca01b1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:11.794 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.806 7f7c9f7fe700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f7ca00cda10 0x7f7ca00d5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:11.794 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.806 7f7c97fff700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f7ca00d64c0 0x7f7ca01b1bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:55108/0 (socket says 172.21.15.102:55108) 2024-04-17T22:31:11.794 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.806 7f7c9f7fe700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f7ca00cda10 0x7f7ca00d5e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:52218/0 (socket says 172.21.15.102:52218) 2024-04-17T22:31:11.795 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.806 7f7c97fff700 1 -- 172.21.15.102:0/1234749003 learned_addr learned my addr 172.21.15.102:0/1234749003 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:31:11.795 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.806 7f7c9ffff700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7ca00d6910 0x7f7ca01b21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:11.795 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.806 7f7c97fff700 1 -- 172.21.15.102:0/1234749003 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f7ca00cda10 msgr2=0x7f7ca00d5e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:11.795 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.806 7f7c97fff700 1 --2- 172.21.15.102:0/1234749003 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f7ca00cda10 0x7f7ca00d5e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:11.795 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.806 7f7c97fff700 1 -- 172.21.15.102:0/1234749003 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7ca00d6910 msgr2=0x7f7ca01b21f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:11.795 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.806 7f7c97fff700 1 --2- 172.21.15.102:0/1234749003 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7ca00d6910 0x7f7ca01b21f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:11.795 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.806 7f7c97fff700 1 -- 172.21.15.102:0/1234749003 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ca01b2820 con 0x7f7ca00d64c0 2024-04-17T22:31:11.795 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.806 7f7c9f7fe700 1 --2- 172.21.15.102:0/1234749003 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f7ca00cda10 0x7f7ca00d5e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:11.795 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.806 7f7c9ffff700 1 --2- 172.21.15.102:0/1234749003 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7ca00d6910 0x7f7ca01b21f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:11.795 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.807 7f7c97fff700 1 --2- 172.21.15.102:0/1234749003 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f7ca00d64c0 0x7f7ca01b1bc0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 rx=0x7f7ca00d71f0 tx=0x7f7c88001a00).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:11.795 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.807 7f7c9d7fa700 1 -- 172.21.15.102:0/1234749003 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f7ca01b2820 con 0x7f7ca00d64c0 2024-04-17T22:31:11.795 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.807 7f7c9d7fa700 1 -- 172.21.15.102:0/1234749003 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c880072c0 con 0x7f7ca00d64c0 2024-04-17T22:31:11.795 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.807 7f7ca6e35700 1 -- 172.21.15.102:0/1234749003 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ca01b2ae0 con 0x7f7ca00d64c0 2024-04-17T22:31:11.795 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.807 7f7c9d7fa700 1 -- 172.21.15.102:0/1234749003 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f7c88014070 con 0x7f7ca00d64c0 2024-04-17T22:31:11.796 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.807 7f7ca6e35700 1 -- 172.21.15.102:0/1234749003 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ca00f63c0 con 0x7f7ca00d64c0 2024-04-17T22:31:11.796 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.807 7f7c9d7fa700 1 -- 172.21.15.102:0/1234749003 <== mon.2 v2:172.21.15.102:3302/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7f7c88005810 con 0x7f7ca00d64c0 2024-04-17T22:31:11.796 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.807 7f7c9d7fa700 1 -- 172.21.15.102:0/1234749003 <== mon.2 v2:172.21.15.102:3302/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3194+0+0 (secure 0 0 0) 0x7f7c8800dd20 con 0x7f7ca00d64c0 2024-04-17T22:31:11.796 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.808 7f7ca6e35700 1 -- 172.21.15.102:0/1234749003 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ca00f0820 con 0x7f7ca00d64c0 2024-04-17T22:31:11.797 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.811 7f7c9d7fa700 1 -- 172.21.15.102:0/1234749003 <== mon.2 v2:172.21.15.102:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+110455 (secure 0 0 0) 0x7f7c8800e0d0 con 0x7f7ca00d64c0 2024-04-17T22:31:11.903 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.917 7f7ca6e35700 1 -- 172.21.15.102:0/1234749003 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7ca00efef0 con 0x7f7ca00d64c0 2024-04-17T22:31:11.903 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.918 7f7c9d7fa700 1 -- 172.21.15.102:0/1234749003 <== mon.2 v2:172.21.15.102:3302/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) v1 ==== 74+0+8764 (secure 0 0 0) 0x7f7c88000c30 con 0x7f7ca00d64c0 2024-04-17T22:31:11.904 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-17T22:31:11.904 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":17,"fsid":"cb63e789-091d-49b1-b120-55f9db526871","created":"2024-04-17 22:28:48.209635","modified":"2024-04-17 22:29:23.257387","last_up_change":"2024-04-17 22:28:54.236215","last_in_change":"2024-04-17 22:28:54.236215","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":4,"require_min_compat_client":"nautilus","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-04-17 22:28:57.478545","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-17 22:29:06.144708","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-17 22:29:06.748924","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3351ef85-ade4-4b90-a8f4-213f4e00ba01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6812","nonce":42955}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6813","nonce":42955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6815","nonce":42955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6814","nonce":42955}]},"public_addr":"172.21.15.102:6812/42955","cluster_addr":"172.21.15.102:6813/42955","heartbeat_back_addr":"172.21.15.102:6815/42955","heartbeat_front_addr":"172.21.15.102:6814/42955","state":["exists","up"]},{"osd":1,"uuid":"04f67eac-f36b-48f5-982d-4cfc776596d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6808","nonce":42958}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6809","nonce":42958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6811","nonce":42958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6810","nonce":42958}]},"public_addr":"172.21.15.102:6808/42958","cluster_addr":"172.21.15.102:6809/42958","heartbeat_back_addr":"172.21.15.102:6811/42958","heartbeat_front_addr":"172.21.15.102:6810/42958","state":["exists","up"]},{"osd":2,"uuid":"d524204b-70b2-40a8-8f73-f9e7d44318af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6804","nonce":42954}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6805","nonce":42954}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6807","nonce":42954}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6806","nonce":42954}]},"public_addr":"172.21.15.102:6804/42954","cluster_addr":"172.21.15.102:6805/42954","heartbeat_back_addr":"172.21.15.102:6807/42954","heartbeat_front_addr":"172.21.15.102:6806/42954","state":["exists","up"]},{"osd":3,"uuid":"4fc9d20c-1ad0-484f-aafa-2c9a7574245b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6800","nonce":42953}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6801","nonce":42953}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6803","nonce":42953}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6802","nonce":42953}]},"public_addr":"172.21.15.102:6800/42953","cluster_addr":"172.21.15.102:6801/42953","heartbeat_back_addr":"172.21.15.102:6803/42953","heartbeat_front_addr":"172.21.15.102:6802/42953","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2024-04-17T22:31:11.905 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.919 7f7ca6e35700 1 -- 172.21.15.102:0/1234749003 >> v1:172.21.15.102:6816/42360 conn(0x7f7c840145a0 legacy=0x7f7c84016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:11.905 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.919 7f7ca6e35700 1 -- 172.21.15.102:0/1234749003 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f7ca00d64c0 msgr2=0x7f7ca01b1bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:11.905 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.919 7f7ca6e35700 1 --2- 172.21.15.102:0/1234749003 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f7ca00d64c0 0x7f7ca01b1bc0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 rx=0x7f7ca00d71f0 tx=0x7f7c88001a00).stop 2024-04-17T22:31:11.905 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.919 7f7ca6e35700 1 -- 172.21.15.102:0/1234749003 shutdown_connections 2024-04-17T22:31:11.905 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.919 7f7ca6e35700 1 --2- 172.21.15.102:0/1234749003 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7ca00d6910 0x7f7ca01b21f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:11.906 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.919 7f7ca6e35700 1 --2- 172.21.15.102:0/1234749003 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f7ca00cda10 0x7f7ca00d5e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:11.906 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.919 7f7ca6e35700 1 --2- 172.21.15.102:0/1234749003 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f7ca00d64c0 0x7f7ca01b1bc0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:11.906 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.919 7f7ca6e35700 1 -- 172.21.15.102:0/1234749003 >> 172.21.15.102:0/1234749003 conn(0x7f7ca00e0fd0 msgr2=0x7f7ca00dacf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:11.906 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.919 7f7ca6e35700 1 -- 172.21.15.102:0/1234749003 shutdown_connections 2024-04-17T22:31:11.906 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:11.919 7f7ca6e35700 1 -- 172.21.15.102:0/1234749003 wait complete. 2024-04-17T22:31:11.916 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2024-04-17 22:28:57.478545', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '12', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-17 22:29:06.144708', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-17 22:29:06.748924', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-17T22:31:11.916 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2024-04-17T22:31:12.046 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.058 7ff9afccb700 1 Processor -- start 2024-04-17T22:31:12.046 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.058 7ff9afccb700 1 -- start start 2024-04-17T22:31:12.046 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.058 7ff9afccb700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ff9a810c8f0 0x7ff9a81a62e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:12.046 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.058 7ff9afccb700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7ff9a81a6910 0x7ff9a81a8d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:12.047 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.058 7ff9afccb700 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9a81a1bf0 con 0x7ff9a80f5750 2024-04-17T22:31:12.047 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.058 7ff9afccb700 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9a80f93c0 con 0x7ff9a81a6910 2024-04-17T22:31:12.047 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.058 7ff9afccb700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9a810cd40 con 0x7ff9a810c8f0 2024-04-17T22:31:12.047 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.058 7ff9a7fff700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ff9a810c8f0 0x7ff9a81a62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:12.047 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.058 7ff9acac6700 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7ff9a80f5750 0x7ff9a810c0f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:38814/0 (socket says 172.21.15.102:38814) 2024-04-17T22:31:12.047 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.058 7ff9ad2c7700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7ff9a81a6910 0x7ff9a81a8d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:12.047 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.058 7ff9acac6700 1 -- 172.21.15.102:0/3349759584 learned_addr learned my addr 172.21.15.102:0/3349759584 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-17T22:31:12.047 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.059 7ff9ad2c7700 1 -- 172.21.15.102:0/3349759584 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ff9a810c8f0 msgr2=0x7ff9a81a62e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:31:12.047 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.059 7ff9ad2c7700 1 --2- 172.21.15.102:0/3349759584 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ff9a810c8f0 0x7ff9a81a62e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:12.047 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.059 7ff9ad2c7700 1 -- 172.21.15.102:0/3349759584 >> v1:172.21.15.102:6789/0 conn(0x7ff9a80f5750 legacy=0x7ff9a810c0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:12.047 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.059 7ff9ad2c7700 1 -- 172.21.15.102:0/3349759584 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9a81aa3b0 con 0x7ff9a81a6910 2024-04-17T22:31:12.047 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.059 7ff9a77fe700 1 -- 172.21.15.102:0/3349759584 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2701789499 0 0) 0x7ff9a81a1bf0 con 0x7ff9a80f5750 2024-04-17T22:31:12.047 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.059 7ff9a7fff700 1 --2- 172.21.15.102:0/3349759584 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ff9a810c8f0 0x7ff9a81a62e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:12.047 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.059 7ff9ad2c7700 1 --2- 172.21.15.102:0/3349759584 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff9a81a6910 0x7ff9a81a8d70 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 rx=0x7ff9a00032f0 tx=0x7ff9a0009d10).ready entity=mon.0 client_cookie=af28151724639e81 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:12.048 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.059 7ff9a77fe700 1 -- 172.21.15.102:0/3349759584 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff9a81aa3b0 con 0x7ff9a81a6910 2024-04-17T22:31:12.048 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.059 7ff9a77fe700 1 -- 172.21.15.102:0/3349759584 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9a000b800 con 0x7ff9a81a6910 2024-04-17T22:31:12.048 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.059 7ff9a77fe700 1 -- 172.21.15.102:0/3349759584 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff9a000ba30 con 0x7ff9a81a6910 2024-04-17T22:31:12.048 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.059 7ff9afccb700 1 -- 172.21.15.102:0/3349759584 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff9a81a6910 msgr2=0x7ff9a81a8d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:12.048 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.059 7ff9afccb700 1 --2- 172.21.15.102:0/3349759584 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff9a81a6910 0x7ff9a81a8d70 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 rx=0x7ff9a00032f0 tx=0x7ff9a0009d10).stop 2024-04-17T22:31:12.048 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.060 7ff9afccb700 1 -- 172.21.15.102:0/3349759584 shutdown_connections 2024-04-17T22:31:12.048 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.060 7ff9afccb700 1 --2- 172.21.15.102:0/3349759584 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff9a81a6910 0x7ff9a81a8d70 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:12.048 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.060 7ff9afccb700 1 --2- 172.21.15.102:0/3349759584 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ff9a810c8f0 0x7ff9a81a62e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:12.048 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.060 7ff9afccb700 1 -- 172.21.15.102:0/3349759584 >> 172.21.15.102:0/3349759584 conn(0x7ff9a81138a0 msgr2=0x7ff9a80f4f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:12.048 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.060 7ff9afccb700 1 -- 172.21.15.102:0/3349759584 shutdown_connections 2024-04-17T22:31:12.048 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.060 7ff9afccb700 1 -- 172.21.15.102:0/3349759584 wait complete. 2024-04-17T22:31:12.068 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.082 7ff9afccb700 1 Processor -- start 2024-04-17T22:31:12.068 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.082 7ff9afccb700 1 -- start start 2024-04-17T22:31:12.070 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.083 7ff9afccb700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff9a80ca570 0x7ff9a81ad220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:12.071 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.083 7ff9afccb700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7ff9a81ad850 0x7ff9a81b3c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:12.071 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.083 7ff9afccb700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ff9a81adca0 0x7ff9a81b4250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:12.071 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.083 7ff9afccb700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9a80f7df0 con 0x7ff9a80ca570 2024-04-17T22:31:12.071 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.083 7ff9afccb700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9a81b4880 con 0x7ff9a81adca0 2024-04-17T22:31:12.071 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.083 7ff9afccb700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff9a81b4ab0 con 0x7ff9a81ad850 2024-04-17T22:31:12.071 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.083 7ff9acac6700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff9a80ca570 0x7ff9a81ad220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:12.071 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.083 7ff9ad2c7700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ff9a81adca0 0x7ff9a81b4250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:12.071 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.083 7ff9acac6700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff9a80ca570 0x7ff9a81ad220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:54340/0 (socket says 172.21.15.102:54340) 2024-04-17T22:31:12.071 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.083 7ff9a7fff700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7ff9a81ad850 0x7ff9a81b3c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:12.071 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.083 7ff9acac6700 1 -- 172.21.15.102:0/1501902600 learned_addr learned my addr 172.21.15.102:0/1501902600 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:31:12.071 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.083 7ff9ad2c7700 1 -- 172.21.15.102:0/1501902600 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7ff9a81ad850 msgr2=0x7ff9a81b3c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:12.071 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.083 7ff9ad2c7700 1 --2- 172.21.15.102:0/1501902600 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7ff9a81ad850 0x7ff9a81b3c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:12.071 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.083 7ff9ad2c7700 1 -- 172.21.15.102:0/1501902600 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff9a80ca570 msgr2=0x7ff9a81ad220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:12.071 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.083 7ff9ad2c7700 1 --2- 172.21.15.102:0/1501902600 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff9a80ca570 0x7ff9a81ad220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:12.072 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.083 7ff9ad2c7700 1 -- 172.21.15.102:0/1501902600 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9a81bb310 con 0x7ff9a81adca0 2024-04-17T22:31:12.072 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.083 7ff9acac6700 1 --2- 172.21.15.102:0/1501902600 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff9a80ca570 0x7ff9a81ad220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:12.072 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.083 7ff9ad2c7700 1 --2- 172.21.15.102:0/1501902600 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ff9a81adca0 0x7ff9a81b4250 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 rx=0x7ff9a80c5220 tx=0x7ff9a0006c40).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:12.072 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.083 7ff9a5ffb700 1 -- 172.21.15.102:0/1501902600 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff9a81bb310 con 0x7ff9a81adca0 2024-04-17T22:31:12.072 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.083 7ff9a5ffb700 1 -- 172.21.15.102:0/1501902600 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9a0011ae0 con 0x7ff9a81adca0 2024-04-17T22:31:12.072 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.083 7ff9afccb700 1 -- 172.21.15.102:0/1501902600 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9a81bb510 con 0x7ff9a81adca0 2024-04-17T22:31:12.072 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.083 7ff9a5ffb700 1 -- 172.21.15.102:0/1501902600 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7ff9a0017040 con 0x7ff9a81adca0 2024-04-17T22:31:12.072 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.084 7ff9afccb700 1 -- 172.21.15.102:0/1501902600 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9a81bb8f0 con 0x7ff9a81adca0 2024-04-17T22:31:12.072 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.084 7ff9a5ffb700 1 -- 172.21.15.102:0/1501902600 <== mon.1 v2:172.21.15.102:3301/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7ff9a001c470 con 0x7ff9a81adca0 2024-04-17T22:31:12.072 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.084 7ff9a5ffb700 1 -- 172.21.15.102:0/1501902600 <== mon.1 v2:172.21.15.102:3301/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3194+0+0 (secure 0 0 0) 0x7ff9a0018030 con 0x7ff9a81adca0 2024-04-17T22:31:12.072 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.084 7ff9afccb700 1 -- 172.21.15.102:0/1501902600 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9a80f0800 con 0x7ff9a81adca0 2024-04-17T22:31:12.073 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.087 7ff9a5ffb700 1 -- 172.21.15.102:0/1501902600 <== mon.1 v2:172.21.15.102:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+110455 (secure 0 0 0) 0x7ff9a000b5f0 con 0x7ff9a81adca0 2024-04-17T22:31:12.179 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.193 7ff9afccb700 1 -- 172.21.15.102:0/1501902600 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7ff9a80a64e0 con 0x7ff9a81adca0 2024-04-17T22:31:12.179 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.193 7ff9a5ffb700 1 -- 172.21.15.102:0/1501902600 <== mon.1 v2:172.21.15.102:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v17) v1 ==== 92+0+10 (secure 0 0 0) 0x7ff9a00118d0 con 0x7ff9a81adca0 2024-04-17T22:31:12.180 INFO:teuthology.orchestra.run.smithi102.stdout:pg_num: 8 2024-04-17T22:31:12.181 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.194 7ff9afccb700 1 -- 172.21.15.102:0/1501902600 >> v1:172.21.15.102:6816/42360 conn(0x7ff9900145a0 legacy=0x7ff990016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:12.181 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.194 7ff9afccb700 1 -- 172.21.15.102:0/1501902600 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ff9a81adca0 msgr2=0x7ff9a81b4250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:12.181 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.194 7ff9afccb700 1 --2- 172.21.15.102:0/1501902600 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ff9a81adca0 0x7ff9a81b4250 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 rx=0x7ff9a80c5220 tx=0x7ff9a0006c40).stop 2024-04-17T22:31:12.181 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.194 7ff9afccb700 1 -- 172.21.15.102:0/1501902600 shutdown_connections 2024-04-17T22:31:12.181 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.194 7ff9afccb700 1 --2- 172.21.15.102:0/1501902600 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ff9a81adca0 0x7ff9a81b4250 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:12.181 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.194 7ff9afccb700 1 --2- 172.21.15.102:0/1501902600 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff9a80ca570 0x7ff9a81ad220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:12.181 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.194 7ff9afccb700 1 --2- 172.21.15.102:0/1501902600 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7ff9a81ad850 0x7ff9a81b3c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:12.181 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.194 7ff9afccb700 1 -- 172.21.15.102:0/1501902600 >> 172.21.15.102:0/1501902600 conn(0x7ff9a80e0d60 msgr2=0x7ff9a80daa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:12.181 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.195 7ff9afccb700 1 -- 172.21.15.102:0/1501902600 shutdown_connections 2024-04-17T22:31:12.181 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.195 7ff9afccb700 1 -- 172.21.15.102:0/1501902600 wait complete. 2024-04-17T22:31:12.191 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-17T22:31:12.321 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.333 7f128b164700 1 Processor -- start 2024-04-17T22:31:12.321 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.333 7f128b164700 1 -- start start 2024-04-17T22:31:12.321 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.333 7f128b164700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f12841a6690 0x7f12841a8af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:12.321 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.333 7f128b164700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f12841a9120 0x7f12841ab580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:12.321 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.333 7f128b164700 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12840ad030 con 0x7f128419fde0 2024-04-17T22:31:12.321 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.333 7f128b164700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12840f9d40 con 0x7f12841a9120 2024-04-17T22:31:12.321 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.333 7f128b164700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f12840f2c90 con 0x7f12841a6690 2024-04-17T22:31:12.321 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.333 7f1282ffd700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f12841a6690 0x7f12841a8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:12.321 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.333 7f12837fe700 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f128419fde0 0x7f12841a0230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:38820/0 (socket says 172.21.15.102:38820) 2024-04-17T22:31:12.321 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.333 7f12837fe700 1 -- 172.21.15.102:0/2645062551 learned_addr learned my addr 172.21.15.102:0/2645062551 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-17T22:31:12.321 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.333 7f1283fff700 1 --2- 172.21.15.102:0/2645062551 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f12841a9120 0x7f12841ab580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:12.322 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.333 7f12827fc700 1 -- 172.21.15.102:0/2645062551 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3536847146 0 0) 0x7f127c001490 con 0x7f128419fde0 2024-04-17T22:31:12.322 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.333 7f12827fc700 1 -- 172.21.15.102:0/2645062551 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1264003270 con 0x7f128419fde0 2024-04-17T22:31:12.322 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.333 7f1283fff700 1 -- 172.21.15.102:0/2645062551 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f12841a6690 msgr2=0x7f12841a8af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:31:12.322 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.333 7f1283fff700 1 --2- 172.21.15.102:0/2645062551 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f12841a6690 0x7f12841a8af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:12.322 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.333 7f1283fff700 1 -- 172.21.15.102:0/2645062551 >> v1:172.21.15.102:6789/0 conn(0x7f128419fde0 legacy=0x7f12841a0230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:12.322 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.334 7f1283fff700 1 -- 172.21.15.102:0/2645062551 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12841a4030 con 0x7f12841a9120 2024-04-17T22:31:12.322 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.334 7f1282ffd700 1 --2- 172.21.15.102:0/2645062551 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f12841a6690 0x7f12841a8af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:12.322 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.334 7f1283fff700 1 --2- 172.21.15.102:0/2645062551 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f12841a9120 0x7f12841ab580 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 rx=0x7f12840f4010 tx=0x7f127400ba50).ready entity=mon.1 client_cookie=8b281fa7c1ce3733 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:12.322 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.334 7f12827fc700 1 -- 172.21.15.102:0/2645062551 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f12841a4030 con 0x7f12841a9120 2024-04-17T22:31:12.322 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.334 7f12827fc700 1 -- 172.21.15.102:0/2645062551 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f127400d570 con 0x7f12841a9120 2024-04-17T22:31:12.322 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.334 7f12827fc700 1 -- 172.21.15.102:0/2645062551 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f127400d940 con 0x7f12841a9120 2024-04-17T22:31:12.322 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.334 7f128b164700 1 -- 172.21.15.102:0/2645062551 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f12841a9120 msgr2=0x7f12841ab580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:12.322 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.334 7f128b164700 1 --2- 172.21.15.102:0/2645062551 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f12841a9120 0x7f12841ab580 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 rx=0x7f12840f4010 tx=0x7f127400ba50).stop 2024-04-17T22:31:12.322 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.334 7f128b164700 1 -- 172.21.15.102:0/2645062551 shutdown_connections 2024-04-17T22:31:12.323 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.334 7f128b164700 1 --2- 172.21.15.102:0/2645062551 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f12841a9120 0x7f12841ab580 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:12.323 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.334 7f128b164700 1 --2- 172.21.15.102:0/2645062551 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f12841a6690 0x7f12841a8af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:12.323 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.334 7f128b164700 1 -- 172.21.15.102:0/2645062551 >> 172.21.15.102:0/2645062551 conn(0x7f1284113000 msgr2=0x7f1284113450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:12.323 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.334 7f128b164700 1 -- 172.21.15.102:0/2645062551 shutdown_connections 2024-04-17T22:31:12.323 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.334 7f128b164700 1 -- 172.21.15.102:0/2645062551 wait complete. 2024-04-17T22:31:12.343 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.357 7f128b164700 1 Processor -- start 2024-04-17T22:31:12.343 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.357 7f128b164700 1 -- start start 2024-04-17T22:31:12.345 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.357 7f128b164700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f12840de670 0x7f128419ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:12.345 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.357 7f128b164700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f128419e3f0 0x7f12841b3fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:12.345 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.357 7f128b164700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f128419e840 0x7f12841b45d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:12.345 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.357 7f128b164700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12840d7ae0 con 0x7f12840de670 2024-04-17T22:31:12.345 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.357 7f128b164700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12840ddfb0 con 0x7f128419e3f0 2024-04-17T22:31:12.345 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.357 7f128b164700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f12841b4c00 con 0x7f128419e840 2024-04-17T22:31:12.345 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.357 7f1282ffd700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f128419e3f0 0x7f12841b3fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:12.345 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.357 7f12837fe700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f12840de670 0x7f128419ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:12.346 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.357 7f1283fff700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f128419e840 0x7f12841b45d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:12.346 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.358 7f1282ffd700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f128419e3f0 0x7f12841b3fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:52246/0 (socket says 172.21.15.102:52246) 2024-04-17T22:31:12.346 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.358 7f1282ffd700 1 -- 172.21.15.102:0/2162747204 learned_addr learned my addr 172.21.15.102:0/2162747204 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:31:12.346 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.358 7f1282ffd700 1 -- 172.21.15.102:0/2162747204 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f128419e840 msgr2=0x7f12841b45d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:12.346 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.358 7f1282ffd700 1 --2- 172.21.15.102:0/2162747204 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f128419e840 0x7f12841b45d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:12.346 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.358 7f1282ffd700 1 -- 172.21.15.102:0/2162747204 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f12840de670 msgr2=0x7f128419ddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:12.346 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.358 7f1282ffd700 1 --2- 172.21.15.102:0/2162747204 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f12840de670 0x7f128419ddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:12.346 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.358 7f1282ffd700 1 -- 172.21.15.102:0/2162747204 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12841b4de0 con 0x7f128419e3f0 2024-04-17T22:31:12.346 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.358 7f1282ffd700 1 --2- 172.21.15.102:0/2162747204 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f128419e3f0 0x7f12841b3fa0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 rx=0x7f12840e1510 tx=0x7f126c00ecd0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:12.346 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.358 7f1280ff9700 1 -- 172.21.15.102:0/2162747204 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f12841b4de0 con 0x7f128419e3f0 2024-04-17T22:31:12.346 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.358 7f1280ff9700 1 -- 172.21.15.102:0/2162747204 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f126c00c9a0 con 0x7f128419e3f0 2024-04-17T22:31:12.346 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.358 7f128b164700 1 -- 172.21.15.102:0/2162747204 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12841bb690 con 0x7f128419e3f0 2024-04-17T22:31:12.346 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.358 7f1280ff9700 1 -- 172.21.15.102:0/2162747204 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f126c00f040 con 0x7f128419e3f0 2024-04-17T22:31:12.346 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.358 7f128b164700 1 -- 172.21.15.102:0/2162747204 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12841bba70 con 0x7f128419e3f0 2024-04-17T22:31:12.346 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.358 7f1280ff9700 1 -- 172.21.15.102:0/2162747204 <== mon.1 v2:172.21.15.102:3301/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7f126c011070 con 0x7f128419e3f0 2024-04-17T22:31:12.347 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.359 7f1280ff9700 1 -- 172.21.15.102:0/2162747204 <== mon.1 v2:172.21.15.102:3301/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3194+0+0 (secure 0 0 0) 0x7f126c010040 con 0x7f128419e3f0 2024-04-17T22:31:12.347 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.359 7f128b164700 1 -- 172.21.15.102:0/2162747204 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12840f09f0 con 0x7f128419e3f0 2024-04-17T22:31:12.348 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.362 7f1280ff9700 1 -- 172.21.15.102:0/2162747204 <== mon.1 v2:172.21.15.102:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+110455 (secure 0 0 0) 0x7f126c00c5a0 con 0x7f128419e3f0 2024-04-17T22:31:12.456 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.470 7f128b164700 1 -- 172.21.15.102:0/2162747204 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f12841ae6c0 con 0x7f128419e3f0 2024-04-17T22:31:12.456 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.470 7f1280ff9700 1 -- 172.21.15.102:0/2162747204 <== mon.1 v2:172.21.15.102:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v17) v1 ==== 104+0+11 (secure 0 0 0) 0x7f126c000c30 con 0x7f128419e3f0 2024-04-17T22:31:12.457 INFO:teuthology.orchestra.run.smithi102.stdout:pg_num: 64 2024-04-17T22:31:12.458 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.472 7f128b164700 1 -- 172.21.15.102:0/2162747204 >> v1:172.21.15.102:6816/42360 conn(0x7f12680145a0 legacy=0x7f1268016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:12.458 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.472 7f128b164700 1 -- 172.21.15.102:0/2162747204 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f128419e3f0 msgr2=0x7f12841b3fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:12.458 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.472 7f128b164700 1 --2- 172.21.15.102:0/2162747204 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f128419e3f0 0x7f12841b3fa0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 rx=0x7f12840e1510 tx=0x7f126c00ecd0).stop 2024-04-17T22:31:12.458 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.472 7f128b164700 1 -- 172.21.15.102:0/2162747204 shutdown_connections 2024-04-17T22:31:12.458 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.472 7f128b164700 1 --2- 172.21.15.102:0/2162747204 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f128419e840 0x7f12841b45d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:12.458 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.472 7f128b164700 1 --2- 172.21.15.102:0/2162747204 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f12840de670 0x7f128419ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:12.458 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.472 7f128b164700 1 --2- 172.21.15.102:0/2162747204 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f128419e3f0 0x7f12841b3fa0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:12.459 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.472 7f128b164700 1 -- 172.21.15.102:0/2162747204 >> 172.21.15.102:0/2162747204 conn(0x7f12840e0d60 msgr2=0x7f12841a0620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:12.459 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.472 7f128b164700 1 -- 172.21.15.102:0/2162747204 shutdown_connections 2024-04-17T22:31:12.459 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.472 7f128b164700 1 -- 172.21.15.102:0/2162747204 wait complete. 2024-04-17T22:31:12.468 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-17T22:31:12.595 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.610 7f67e6277700 1 Processor -- start 2024-04-17T22:31:12.598 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.610 7f67e6277700 1 -- start start 2024-04-17T22:31:12.598 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.610 7f67e6277700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f67e01055c0 0x7f67e010b130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:12.598 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.610 7f67e6277700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f67e00f59b0 0x7f67e010b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:12.598 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.610 7f67e6277700 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67e00ad030 con 0x7f67e019fc80 2024-04-17T22:31:12.598 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.610 7f67e6277700 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67e010bd90 con 0x7f67e01055c0 2024-04-17T22:31:12.598 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.610 7f67e6277700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f67e01a08d0 con 0x7f67e00f59b0 2024-04-17T22:31:12.598 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.610 7f67d7fff700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f67e00f59b0 0x7f67e010b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:12.598 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.610 7f67d7fff700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f67e00f59b0 0x7f67e010b760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:55136/0 (socket says 172.21.15.102:55136) 2024-04-17T22:31:12.599 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.610 7f67d7fff700 1 -- 172.21.15.102:0/3056714701 learned_addr learned my addr 172.21.15.102:0/3056714701 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:31:12.599 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.611 7f67df001700 1 --2- 172.21.15.102:0/3056714701 >> v2:172.21.15.102:3300/0 conn(0x7f67e01055c0 0x7f67e010b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:12.599 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.611 7f67d7fff700 1 -- 172.21.15.102:0/3056714701 >> v2:172.21.15.102:3300/0 conn(0x7f67e01055c0 msgr2=0x7f67e010b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:31:12.599 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.611 7f67d7fff700 1 --2- 172.21.15.102:0/3056714701 >> v2:172.21.15.102:3300/0 conn(0x7f67e01055c0 0x7f67e010b130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:12.599 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.611 7f67d7fff700 1 -- 172.21.15.102:0/3056714701 >> v1:172.21.15.102:6789/0 conn(0x7f67e019fc80 legacy=0x7f67e01a00d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:12.599 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.611 7f67d7fff700 1 -- 172.21.15.102:0/3056714701 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67e01a0b00 con 0x7f67e00f59b0 2024-04-17T22:31:12.599 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.611 7f67d7fff700 1 --2- 172.21.15.102:0/3056714701 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f67e00f59b0 0x7f67e010b760 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0x7f67c8002f70 tx=0x7f67c800bc00).ready entity=mon.2 client_cookie=5af55e0cda0f3aa8 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:12.599 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.611 7f67de800700 1 -- 172.21.15.102:0/3056714701 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f67e01a0b00 con 0x7f67e00f59b0 2024-04-17T22:31:12.599 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.611 7f67de800700 1 -- 172.21.15.102:0/3056714701 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67c800dc80 con 0x7f67e00f59b0 2024-04-17T22:31:12.599 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.611 7f67de800700 1 -- 172.21.15.102:0/3056714701 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f67c8010660 con 0x7f67e00f59b0 2024-04-17T22:31:12.599 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.611 7f67e6277700 1 -- 172.21.15.102:0/3056714701 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f67e00f59b0 msgr2=0x7f67e010b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:12.599 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.611 7f67e6277700 1 --2- 172.21.15.102:0/3056714701 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f67e00f59b0 0x7f67e010b760 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0x7f67c8002f70 tx=0x7f67c800bc00).stop 2024-04-17T22:31:12.599 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.612 7f67e6277700 1 -- 172.21.15.102:0/3056714701 shutdown_connections 2024-04-17T22:31:12.599 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.612 7f67e6277700 1 --2- 172.21.15.102:0/3056714701 >> v2:172.21.15.102:3300/0 conn(0x7f67e01055c0 0x7f67e010b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:12.600 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.612 7f67e6277700 1 --2- 172.21.15.102:0/3056714701 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f67e00f59b0 0x7f67e010b760 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:12.600 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.612 7f67e6277700 1 -- 172.21.15.102:0/3056714701 >> 172.21.15.102:0/3056714701 conn(0x7f67e0113a90 msgr2=0x7f67e00f5140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:12.600 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.612 7f67e6277700 1 -- 172.21.15.102:0/3056714701 shutdown_connections 2024-04-17T22:31:12.600 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.612 7f67e6277700 1 -- 172.21.15.102:0/3056714701 wait complete. 2024-04-17T22:31:12.622 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.635 7f67e6277700 1 Processor -- start 2024-04-17T22:31:12.623 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.635 7f67e6277700 1 -- start start 2024-04-17T22:31:12.623 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.635 7f67e6277700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f67e00d9e70 0x7f67e01ad1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:12.623 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.635 7f67e6277700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f67e01ad820 0x7f67e01b3b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:12.623 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.635 7f67e6277700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f67e01adc70 0x7f67e01b4130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:12.623 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.635 7f67e6277700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67e00d8f40 con 0x7f67e00d9e70 2024-04-17T22:31:12.623 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.635 7f67e6277700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67e00d9240 con 0x7f67e01ad820 2024-04-17T22:31:12.623 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.635 7f67e6277700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f67e01b4760 con 0x7f67e01adc70 2024-04-17T22:31:12.623 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.635 7f67d7fff700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f67e01ad820 0x7f67e01b3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:12.623 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.635 7f67df802700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f67e01adc70 0x7f67e01b4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:12.623 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.635 7f67d7fff700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f67e01ad820 0x7f67e01b3b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:52250/0 (socket says 172.21.15.102:52250) 2024-04-17T22:31:12.623 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.635 7f67d7fff700 1 -- 172.21.15.102:0/420350848 learned_addr learned my addr 172.21.15.102:0/420350848 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:31:12.623 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.635 7f67df001700 1 --2- 172.21.15.102:0/420350848 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f67e00d9e70 0x7f67e01ad1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:12.623 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.635 7f67df802700 1 -- 172.21.15.102:0/420350848 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f67e01ad820 msgr2=0x7f67e01b3b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:12.623 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.635 7f67df802700 1 --2- 172.21.15.102:0/420350848 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f67e01ad820 0x7f67e01b3b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:12.623 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.635 7f67df802700 1 -- 172.21.15.102:0/420350848 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f67e00d9e70 msgr2=0x7f67e01ad1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:12.624 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.635 7f67df802700 1 --2- 172.21.15.102:0/420350848 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f67e00d9e70 0x7f67e01ad1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:12.624 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.636 7f67df802700 1 -- 172.21.15.102:0/420350848 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67e01b4940 con 0x7f67e01adc70 2024-04-17T22:31:12.624 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.636 7f67df001700 1 --2- 172.21.15.102:0/420350848 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f67e00d9e70 0x7f67e01ad1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:12.624 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.636 7f67d7fff700 1 --2- 172.21.15.102:0/420350848 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f67e01ad820 0x7f67e01b3b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:12.624 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.636 7f67df802700 1 --2- 172.21.15.102:0/420350848 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f67e01adc70 0x7f67e01b4130 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 rx=0x7f67e00c4200 tx=0x7f67d800c900).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:12.624 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.636 7f67dcffd700 1 -- 172.21.15.102:0/420350848 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f67e01b4940 con 0x7f67e01adc70 2024-04-17T22:31:12.624 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.636 7f67dcffd700 1 -- 172.21.15.102:0/420350848 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67d8006ab0 con 0x7f67e01adc70 2024-04-17T22:31:12.624 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.636 7f67e6277700 1 -- 172.21.15.102:0/420350848 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67e01bb1f0 con 0x7f67e01adc70 2024-04-17T22:31:12.624 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.636 7f67dcffd700 1 -- 172.21.15.102:0/420350848 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f67d8007030 con 0x7f67e01adc70 2024-04-17T22:31:12.624 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.636 7f67e6277700 1 -- 172.21.15.102:0/420350848 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67e01bb5d0 con 0x7f67e01adc70 2024-04-17T22:31:12.624 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.636 7f67dcffd700 1 -- 172.21.15.102:0/420350848 <== mon.2 v2:172.21.15.102:3302/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7f67d801a060 con 0x7f67e01adc70 2024-04-17T22:31:12.624 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.636 7f67dcffd700 1 -- 172.21.15.102:0/420350848 <== mon.2 v2:172.21.15.102:3302/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3194+0+0 (secure 0 0 0) 0x7f67d8002f70 con 0x7f67e01adc70 2024-04-17T22:31:12.624 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.637 7f67e6277700 1 -- 172.21.15.102:0/420350848 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67e00f09f0 con 0x7f67e01adc70 2024-04-17T22:31:12.626 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.640 7f67dcffd700 1 -- 172.21.15.102:0/420350848 <== mon.2 v2:172.21.15.102:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+110455 (secure 0 0 0) 0x7f67d80068a0 con 0x7f67e01adc70 2024-04-17T22:31:12.731 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.744 7f67e6277700 1 -- 172.21.15.102:0/420350848 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f67e00f00c0 con 0x7f67e01adc70 2024-04-17T22:31:12.731 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.745 7f67dcffd700 1 -- 172.21.15.102:0/420350848 <== mon.2 v2:172.21.15.102:3302/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v17) v1 ==== 100+0+11 (secure 0 0 0) 0x7f67e0114610 con 0x7f67e01adc70 2024-04-17T22:31:12.731 INFO:teuthology.orchestra.run.smithi102.stdout:pg_num: 64 2024-04-17T22:31:12.732 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.746 7f67e6277700 1 -- 172.21.15.102:0/420350848 >> v1:172.21.15.102:6816/42360 conn(0x7f67c40145a0 legacy=0x7f67c4016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:12.732 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.746 7f67e6277700 1 -- 172.21.15.102:0/420350848 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f67e01adc70 msgr2=0x7f67e01b4130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:12.732 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.746 7f67e6277700 1 --2- 172.21.15.102:0/420350848 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f67e01adc70 0x7f67e01b4130 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 rx=0x7f67e00c4200 tx=0x7f67d800c900).stop 2024-04-17T22:31:12.732 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.746 7f67e6277700 1 -- 172.21.15.102:0/420350848 shutdown_connections 2024-04-17T22:31:12.732 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.746 7f67e6277700 1 --2- 172.21.15.102:0/420350848 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f67e01adc70 0x7f67e01b4130 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:12.733 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.746 7f67e6277700 1 --2- 172.21.15.102:0/420350848 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f67e00d9e70 0x7f67e01ad1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:12.733 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.746 7f67e6277700 1 --2- 172.21.15.102:0/420350848 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f67e01ad820 0x7f67e01b3b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:12.733 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.746 7f67e6277700 1 -- 172.21.15.102:0/420350848 >> 172.21.15.102:0/420350848 conn(0x7f67e00c5a80 msgr2=0x7f67e00df2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:12.733 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.746 7f67e6277700 1 -- 172.21.15.102:0/420350848 shutdown_connections 2024-04-17T22:31:12.733 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.746 7f67e6277700 1 -- 172.21.15.102:0/420350848 wait complete. 2024-04-17T22:31:12.743 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-17T22:31:12.870 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.881 7fb80c347700 1 Processor -- start 2024-04-17T22:31:12.870 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.882 7fb80c347700 1 -- start start 2024-04-17T22:31:12.870 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.882 7fb80c347700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb8041a6440 0x7fb8041a88a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:12.870 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.882 7fb80c347700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fb8041a8ed0 0x7fb8041ab330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:12.870 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.882 7fb80c347700 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb80410ee20 con 0x7fb80410e1d0 2024-04-17T22:31:12.871 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.882 7fb80c347700 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8040f9ea0 con 0x7fb8041a8ed0 2024-04-17T22:31:12.871 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.882 7fb80c347700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8040f8fc0 con 0x7fb8041a6440 2024-04-17T22:31:12.871 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.882 7fb808941700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb8041a6440 0x7fb8041a88a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:12.871 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.882 7fb809943700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fb8041a8ed0 0x7fb8041ab330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:12.871 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.882 7fb809943700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fb8041a8ed0 0x7fb8041ab330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:54366/0 (socket says 172.21.15.102:54366) 2024-04-17T22:31:12.871 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.882 7fb808941700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb8041a6440 0x7fb8041a88a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:52256/0 (socket says 172.21.15.102:52256) 2024-04-17T22:31:12.871 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.882 7fb809943700 1 -- 172.21.15.102:0/2963058401 learned_addr learned my addr 172.21.15.102:0/2963058401 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:31:12.871 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.882 7fb809142700 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7fb80410e1d0 0x7fb80410e620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:38838/0 (socket says 172.21.15.102:38838) 2024-04-17T22:31:12.871 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.882 7fb808941700 1 -- 172.21.15.102:0/2963058401 >> v2:172.21.15.102:3300/0 conn(0x7fb8041a8ed0 msgr2=0x7fb8041ab330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:31:12.871 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.882 7fb808941700 1 --2- 172.21.15.102:0/2963058401 >> v2:172.21.15.102:3300/0 conn(0x7fb8041a8ed0 0x7fb8041ab330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:12.871 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.882 7fb808941700 1 -- 172.21.15.102:0/2963058401 >> v1:172.21.15.102:6789/0 conn(0x7fb80410e1d0 legacy=0x7fb80410e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:12.871 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.882 7fb7f3fff700 1 -- 172.21.15.102:0/2963058401 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3449981867 0 0) 0x7fb7f4001110 con 0x7fb80410e1d0 2024-04-17T22:31:12.871 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.882 7fb808941700 1 -- 172.21.15.102:0/2963058401 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8041ac970 con 0x7fb8041a6440 2024-04-17T22:31:12.872 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.882 7fb809943700 1 --2- 172.21.15.102:0/2963058401 >> v2:172.21.15.102:3300/0 conn(0x7fb8041a8ed0 0x7fb8041ab330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:12.872 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.883 7fb808941700 1 --2- 172.21.15.102:0/2963058401 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb8041a6440 0x7fb8041a88a0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 rx=0x7fb800002f70 tx=0x7fb80000b970).ready entity=mon.1 client_cookie=32f6a25ff062c753 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:12.872 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.883 7fb7f3fff700 1 -- 172.21.15.102:0/2963058401 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb8041ac970 con 0x7fb8041a6440 2024-04-17T22:31:12.872 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.883 7fb7f3fff700 1 -- 172.21.15.102:0/2963058401 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb80000d610 con 0x7fb8041a6440 2024-04-17T22:31:12.872 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.883 7fb7f3fff700 1 -- 172.21.15.102:0/2963058401 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb80000d9e0 con 0x7fb8041a6440 2024-04-17T22:31:12.872 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.883 7fb80c347700 1 -- 172.21.15.102:0/2963058401 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb8041a6440 msgr2=0x7fb8041a88a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:12.872 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.883 7fb80c347700 1 --2- 172.21.15.102:0/2963058401 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb8041a6440 0x7fb8041a88a0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 rx=0x7fb800002f70 tx=0x7fb80000b970).stop 2024-04-17T22:31:12.872 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.883 7fb80c347700 1 -- 172.21.15.102:0/2963058401 shutdown_connections 2024-04-17T22:31:12.872 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.883 7fb80c347700 1 --2- 172.21.15.102:0/2963058401 >> v2:172.21.15.102:3300/0 conn(0x7fb8041a8ed0 0x7fb8041ab330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:12.872 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.883 7fb80c347700 1 --2- 172.21.15.102:0/2963058401 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb8041a6440 0x7fb8041a88a0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:12.872 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.883 7fb80c347700 1 -- 172.21.15.102:0/2963058401 >> 172.21.15.102:0/2963058401 conn(0x7fb804113830 msgr2=0x7fb8040f4ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:12.872 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.883 7fb80c347700 1 -- 172.21.15.102:0/2963058401 shutdown_connections 2024-04-17T22:31:12.873 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.883 7fb80c347700 1 -- 172.21.15.102:0/2963058401 wait complete. 2024-04-17T22:31:12.894 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.906 7fb80c347700 1 Processor -- start 2024-04-17T22:31:12.895 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.906 7fb80c347700 1 -- start start 2024-04-17T22:31:12.895 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.907 7fb80c347700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb8040ca8a0 0x7fb8040cacf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:12.895 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.907 7fb80c347700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb8040cb320 0x7fb8040db340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:12.895 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.907 7fb80c347700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb8040db970 0x7fb8040dbdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:12.895 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.907 7fb80c347700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb80408a5a0 con 0x7fb8040db970 2024-04-17T22:31:12.895 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.907 7fb80c347700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8040fa140 con 0x7fb8040cb320 2024-04-17T22:31:12.895 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.907 7fb80c347700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb8040e07d0 con 0x7fb8040ca8a0 2024-04-17T22:31:12.895 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.907 7fb809142700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb8040ca8a0 0x7fb8040cacf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:12.895 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.907 7fb808941700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb8040cb320 0x7fb8040db340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:12.895 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.907 7fb809943700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb8040db970 0x7fb8040dbdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:12.895 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.907 7fb809142700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb8040ca8a0 0x7fb8040cacf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:55146/0 (socket says 172.21.15.102:55146) 2024-04-17T22:31:12.896 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.907 7fb808941700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb8040cb320 0x7fb8040db340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:52270/0 (socket says 172.21.15.102:52270) 2024-04-17T22:31:12.896 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.907 7fb809142700 1 -- 172.21.15.102:0/1081234395 learned_addr learned my addr 172.21.15.102:0/1081234395 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:31:12.896 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.907 7fb808941700 1 -- 172.21.15.102:0/1081234395 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb8040ca8a0 msgr2=0x7fb8040cacf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:12.896 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.907 7fb808941700 1 --2- 172.21.15.102:0/1081234395 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb8040ca8a0 0x7fb8040cacf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:12.896 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.907 7fb808941700 1 -- 172.21.15.102:0/1081234395 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb8040db970 msgr2=0x7fb8040dbdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:12.896 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.907 7fb808941700 1 --2- 172.21.15.102:0/1081234395 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb8040db970 0x7fb8040dbdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:12.896 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.907 7fb808941700 1 -- 172.21.15.102:0/1081234395 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8040dfee0 con 0x7fb8040cb320 2024-04-17T22:31:12.896 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.907 7fb809142700 1 --2- 172.21.15.102:0/1081234395 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb8040ca8a0 0x7fb8040cacf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:12.896 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.907 7fb809943700 1 --2- 172.21.15.102:0/1081234395 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb8040db970 0x7fb8040dbdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:12.896 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.907 7fb808941700 1 --2- 172.21.15.102:0/1081234395 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb8040cb320 0x7fb8040db340 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 rx=0x7fb8040c7290 tx=0x7fb80000a040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:12.896 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.907 7fb7f27fc700 1 -- 172.21.15.102:0/1081234395 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb8040dfee0 con 0x7fb8040cb320 2024-04-17T22:31:12.896 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.907 7fb7f27fc700 1 -- 172.21.15.102:0/1081234395 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8000139b0 con 0x7fb8040cb320 2024-04-17T22:31:12.896 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.907 7fb80c347700 1 -- 172.21.15.102:0/1081234395 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8041bbb50 con 0x7fb8040cb320 2024-04-17T22:31:12.897 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.907 7fb7f27fc700 1 -- 172.21.15.102:0/1081234395 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb800013d80 con 0x7fb8040cb320 2024-04-17T22:31:12.897 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.908 7fb80c347700 1 -- 172.21.15.102:0/1081234395 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8041bbf30 con 0x7fb8040cb320 2024-04-17T22:31:12.897 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.908 7fb7f27fc700 1 -- 172.21.15.102:0/1081234395 <== mon.1 v2:172.21.15.102:3301/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7fb800027070 con 0x7fb8040cb320 2024-04-17T22:31:12.897 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.908 7fb7f27fc700 1 -- 172.21.15.102:0/1081234395 <== mon.1 v2:172.21.15.102:3301/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3194+0+0 (secure 0 0 0) 0x7fb800008930 con 0x7fb8040cb320 2024-04-17T22:31:12.897 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.909 7fb80c347700 1 -- 172.21.15.102:0/1081234395 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8040f0790 con 0x7fb8040cb320 2024-04-17T22:31:12.897 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:12.912 7fb7f27fc700 1 -- 172.21.15.102:0/1081234395 <== mon.1 v2:172.21.15.102:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+110455 (secure 0 0 0) 0x7fb8000137a0 con 0x7fb8040cb320 2024-04-17T22:31:13.041 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.054 7fb80c347700 1 -- 172.21.15.102:0/1081234395 >> v1:172.21.15.102:6816/42360 conn(0x7fb7e40145a0 legacy=0x7fb7e4016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:13.041 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.054 7fb80c347700 1 -- 172.21.15.102:0/1081234395 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb8040cb320 msgr2=0x7fb8040db340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:13.041 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.054 7fb80c347700 1 --2- 172.21.15.102:0/1081234395 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb8040cb320 0x7fb8040db340 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 rx=0x7fb8040c7290 tx=0x7fb80000a040).stop 2024-04-17T22:31:13.041 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.055 7fb80c347700 1 -- 172.21.15.102:0/1081234395 shutdown_connections 2024-04-17T22:31:13.041 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.055 7fb80c347700 1 --2- 172.21.15.102:0/1081234395 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb8040db970 0x7fb8040dbdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:13.042 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.055 7fb80c347700 1 --2- 172.21.15.102:0/1081234395 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb8040ca8a0 0x7fb8040cacf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:13.042 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.055 7fb80c347700 1 --2- 172.21.15.102:0/1081234395 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb8040cb320 0x7fb8040db340 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:13.042 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.055 7fb80c347700 1 -- 172.21.15.102:0/1081234395 >> 172.21.15.102:0/1081234395 conn(0x7fb8040c6c90 msgr2=0x7fb8040d7140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:13.042 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.055 7fb80c347700 1 -- 172.21.15.102:0/1081234395 shutdown_connections 2024-04-17T22:31:13.042 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.055 7fb80c347700 1 -- 172.21.15.102:0/1081234395 wait complete. 2024-04-17T22:31:13.042 INFO:teuthology.orchestra.run.smithi102.stderr:no valid command found; 10 closest matches: 2024-04-17T22:31:13.042 INFO:teuthology.orchestra.run.smithi102.stderr:osd perf 2024-04-17T22:31:13.042 INFO:teuthology.orchestra.run.smithi102.stderr:osd df {plain|tree} {class|name} {} 2024-04-17T22:31:13.043 INFO:teuthology.orchestra.run.smithi102.stderr:osd blocked-by 2024-04-17T22:31:13.043 INFO:teuthology.orchestra.run.smithi102.stderr:osd pool stats {} 2024-04-17T22:31:13.043 INFO:teuthology.orchestra.run.smithi102.stderr:osd pool scrub [...] 2024-04-17T22:31:13.043 INFO:teuthology.orchestra.run.smithi102.stderr:osd pool deep-scrub [...] 2024-04-17T22:31:13.043 INFO:teuthology.orchestra.run.smithi102.stderr:osd pool repair [...] 2024-04-17T22:31:13.043 INFO:teuthology.orchestra.run.smithi102.stderr:osd pool force-recovery [...] 2024-04-17T22:31:13.043 INFO:teuthology.orchestra.run.smithi102.stderr:osd pool force-backfill [...] 2024-04-17T22:31:13.043 INFO:teuthology.orchestra.run.smithi102.stderr:osd pool cancel-force-recovery [...] 2024-04-17T22:31:13.043 INFO:teuthology.orchestra.run.smithi102.stderr:Error EINVAL: invalid command 2024-04-17T22:31:13.051 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-17T22:31:13.052 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blacklist ls 2024-04-17T22:31:13.178 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.192 7fc12e237700 1 Processor -- start 2024-04-17T22:31:13.178 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.192 7fc12e237700 1 -- start start 2024-04-17T22:31:13.181 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.193 7fc12e237700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc1280f5750 0x7fc12810c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:13.181 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.193 7fc12e237700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fc12810cb70 0x7fc1281a13e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:13.181 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.193 7fc12e237700 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1281a1a10 con 0x7fc12810c720 2024-04-17T22:31:13.181 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.193 7fc12e237700 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1280f93c0 con 0x7fc12810cb70 2024-04-17T22:31:13.181 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.193 7fc12e237700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1281a1c30 con 0x7fc1280f5750 2024-04-17T22:31:13.181 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.193 7fc11ffff700 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7fc12810c720 0x7fc1281a0be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:38842/0 (socket says 172.21.15.102:38842) 2024-04-17T22:31:13.181 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.193 7fc11ffff700 1 -- 172.21.15.102:0/218879281 learned_addr learned my addr 172.21.15.102:0/218879281 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-17T22:31:13.182 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.193 7fc127001700 1 --2- 172.21.15.102:0/218879281 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc1280f5750 0x7fc12810c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:13.182 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.193 7fc127802700 1 --2- 172.21.15.102:0/218879281 >> v2:172.21.15.102:3300/0 conn(0x7fc12810cb70 0x7fc1281a13e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:13.182 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.193 7fc126800700 1 -- 172.21.15.102:0/218879281 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1878691601 0 0) 0x7fc110001110 con 0x7fc12810c720 2024-04-17T22:31:13.182 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.193 7fc126800700 1 -- 172.21.15.102:0/218879281 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc108003270 con 0x7fc12810c720 2024-04-17T22:31:13.182 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.193 7fc127001700 1 -- 172.21.15.102:0/218879281 >> v2:172.21.15.102:3300/0 conn(0x7fc12810cb70 msgr2=0x7fc1281a13e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:31:13.182 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.193 7fc127001700 1 --2- 172.21.15.102:0/218879281 >> v2:172.21.15.102:3300/0 conn(0x7fc12810cb70 0x7fc1281a13e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:13.182 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.193 7fc127001700 1 -- 172.21.15.102:0/218879281 >> v1:172.21.15.102:6789/0 conn(0x7fc12810c720 legacy=0x7fc1281a0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:13.182 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.193 7fc127001700 1 -- 172.21.15.102:0/218879281 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1281aa310 con 0x7fc1280f5750 2024-04-17T22:31:13.182 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.193 7fc127802700 1 --2- 172.21.15.102:0/218879281 >> v2:172.21.15.102:3300/0 conn(0x7fc12810cb70 0x7fc1281a13e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:13.182 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.193 7fc127001700 1 --2- 172.21.15.102:0/218879281 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc1280f5750 0x7fc12810c0f0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 rx=0x7fc118002f70 tx=0x7fc11800b970).ready entity=mon.1 client_cookie=fb6013bbd3e2bc54 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:13.182 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.193 7fc126800700 1 -- 172.21.15.102:0/218879281 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc1281aa310 con 0x7fc1280f5750 2024-04-17T22:31:13.183 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.193 7fc126800700 1 -- 172.21.15.102:0/218879281 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc11800d610 con 0x7fc1280f5750 2024-04-17T22:31:13.183 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.193 7fc126800700 1 -- 172.21.15.102:0/218879281 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc11800d9e0 con 0x7fc1280f5750 2024-04-17T22:31:13.183 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.193 7fc12e237700 1 -- 172.21.15.102:0/218879281 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc1280f5750 msgr2=0x7fc12810c0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:13.183 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.193 7fc12e237700 1 --2- 172.21.15.102:0/218879281 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc1280f5750 0x7fc12810c0f0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 rx=0x7fc118002f70 tx=0x7fc11800b970).stop 2024-04-17T22:31:13.183 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.194 7fc12e237700 1 -- 172.21.15.102:0/218879281 shutdown_connections 2024-04-17T22:31:13.183 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.194 7fc12e237700 1 --2- 172.21.15.102:0/218879281 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc1280f5750 0x7fc12810c0f0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:13.183 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.194 7fc12e237700 1 --2- 172.21.15.102:0/218879281 >> v2:172.21.15.102:3300/0 conn(0x7fc12810cb70 0x7fc1281a13e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:13.183 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.194 7fc12e237700 1 -- 172.21.15.102:0/218879281 >> 172.21.15.102:0/218879281 conn(0x7fc1281138a0 msgr2=0x7fc1280f4f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:13.183 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.194 7fc12e237700 1 -- 172.21.15.102:0/218879281 shutdown_connections 2024-04-17T22:31:13.183 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.194 7fc12e237700 1 -- 172.21.15.102:0/218879281 wait complete. 2024-04-17T22:31:13.203 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.217 7fc12e237700 1 Processor -- start 2024-04-17T22:31:13.203 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.217 7fc12e237700 1 -- start start 2024-04-17T22:31:13.205 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.217 7fc12e237700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fc1281ad2d0 0x7fc1281ad720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:13.205 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.217 7fc12e237700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc1281add50 0x7fc1280f61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:13.205 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.217 7fc12e237700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc1280f67f0 0x7fc1280f6c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:13.206 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.217 7fc12e237700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1280caff0 con 0x7fc1280f67f0 2024-04-17T22:31:13.206 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.217 7fc12e237700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1280c42e0 con 0x7fc1281add50 2024-04-17T22:31:13.206 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.217 7fc12e237700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc1280cb230 con 0x7fc1281ad2d0 2024-04-17T22:31:13.206 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.218 7fc127802700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc1280f67f0 0x7fc1280f6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:13.206 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.218 7fc11ffff700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc1281add50 0x7fc1280f61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:13.206 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.218 7fc127001700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fc1281ad2d0 0x7fc1281ad720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:13.206 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.218 7fc127802700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc1280f67f0 0x7fc1280f6c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:54374/0 (socket says 172.21.15.102:54374) 2024-04-17T22:31:13.206 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.218 7fc127802700 1 -- 172.21.15.102:0/136038171 learned_addr learned my addr 172.21.15.102:0/136038171 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:31:13.206 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.218 7fc127802700 1 -- 172.21.15.102:0/136038171 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fc1281ad2d0 msgr2=0x7fc1281ad720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:13.206 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.218 7fc127802700 1 --2- 172.21.15.102:0/136038171 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fc1281ad2d0 0x7fc1281ad720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:13.206 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.218 7fc127802700 1 -- 172.21.15.102:0/136038171 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc1281add50 msgr2=0x7fc1280f61c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:13.206 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.218 7fc127802700 1 --2- 172.21.15.102:0/136038171 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc1281add50 0x7fc1280f61c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:13.207 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.218 7fc127802700 1 -- 172.21.15.102:0/136038171 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1280ca750 con 0x7fc1280f67f0 2024-04-17T22:31:13.207 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.218 7fc11ffff700 1 --2- 172.21.15.102:0/136038171 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc1281add50 0x7fc1280f61c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:13.207 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.218 7fc127802700 1 --2- 172.21.15.102:0/136038171 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc1280f67f0 0x7fc1280f6c40 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 rx=0x7fc1280d7140 tx=0x7fc120009e90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:13.207 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.218 7fc124ffd700 1 -- 172.21.15.102:0/136038171 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc1280ca750 con 0x7fc1280f67f0 2024-04-17T22:31:13.207 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.218 7fc124ffd700 1 -- 172.21.15.102:0/136038171 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc12000c880 con 0x7fc1280f67f0 2024-04-17T22:31:13.207 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.218 7fc124ffd700 1 -- 172.21.15.102:0/136038171 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc12000f040 con 0x7fc1280f67f0 2024-04-17T22:31:13.207 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.218 7fc12e237700 1 -- 172.21.15.102:0/136038171 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1281bb420 con 0x7fc1280f67f0 2024-04-17T22:31:13.207 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.218 7fc12e237700 1 -- 172.21.15.102:0/136038171 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1281bb800 con 0x7fc1280f67f0 2024-04-17T22:31:13.207 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.219 7fc124ffd700 1 -- 172.21.15.102:0/136038171 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7fc120022060 con 0x7fc1280f67f0 2024-04-17T22:31:13.207 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.219 7fc124ffd700 1 -- 172.21.15.102:0/136038171 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3194+0+0 (secure 0 0 0) 0x7fc120010040 con 0x7fc1280f67f0 2024-04-17T22:31:13.207 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.219 7fc12e237700 1 -- 172.21.15.102:0/136038171 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1280f0800 con 0x7fc1280f67f0 2024-04-17T22:31:13.208 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.222 7fc124ffd700 1 -- 172.21.15.102:0/136038171 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+110455 (secure 0 0 0) 0x7fc12000c670 con 0x7fc1280f67f0 2024-04-17T22:31:13.327 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.341 7fc12e237700 1 -- 172.21.15.102:0/136038171 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd blacklist ls"} v 0) v1 -- 0x7fc1280abc90 con 0x7fc1280f67f0 2024-04-17T22:31:13.327 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.341 7fc124ffd700 1 -- 172.21.15.102:0/136038171 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd blacklist ls"}]=0 listed 0 entries v17) v1 ==== 80+0+0 (secure 0 0 0) 0x7fc128114420 con 0x7fc1280f67f0 2024-04-17T22:31:13.329 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.342 7fc12e237700 1 -- 172.21.15.102:0/136038171 >> v1:172.21.15.102:6816/42360 conn(0x7fc10c0145a0 legacy=0x7fc10c016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:13.329 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.342 7fc12e237700 1 -- 172.21.15.102:0/136038171 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc1280f67f0 msgr2=0x7fc1280f6c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:13.329 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.342 7fc12e237700 1 --2- 172.21.15.102:0/136038171 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc1280f67f0 0x7fc1280f6c40 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 rx=0x7fc1280d7140 tx=0x7fc120009e90).stop 2024-04-17T22:31:13.329 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.342 7fc12e237700 1 -- 172.21.15.102:0/136038171 shutdown_connections 2024-04-17T22:31:13.329 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.342 7fc12e237700 1 --2- 172.21.15.102:0/136038171 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc1280f67f0 0x7fc1280f6c40 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:13.329 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.342 7fc12e237700 1 --2- 172.21.15.102:0/136038171 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fc1281ad2d0 0x7fc1281ad720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:13.329 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.342 7fc12e237700 1 --2- 172.21.15.102:0/136038171 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc1281add50 0x7fc1280f61c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:13.329 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.342 7fc12e237700 1 -- 172.21.15.102:0/136038171 >> 172.21.15.102:0/136038171 conn(0x7fc1280e0f20 msgr2=0x7fc1280dac40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:13.329 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.342 7fc12e237700 1 -- 172.21.15.102:0/136038171 shutdown_connections 2024-04-17T22:31:13.329 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.342 7fc12e237700 1 -- 172.21.15.102:0/136038171 wait complete. 2024-04-17T22:31:13.330 INFO:teuthology.orchestra.run.smithi102.stderr:listed 0 entries 2024-04-17T22:31:13.339 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi190.front.sepia.ceph.com... 2024-04-17T22:31:13.339 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T22:31:13.339 DEBUG:teuthology.orchestra.run.smithi190:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-17T22:31:13.382 INFO:teuthology.orchestra.run:waiting for 300 2024-04-17T22:31:13.383 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi190.front.sepia.ceph.com 2024-04-17T22:31:13.383 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T22:31:13.383 DEBUG:teuthology.orchestra.run.smithi190:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-17T22:31:13.444 INFO:tasks.cephfs.cephfs_test_case:[] 2024-04-17T22:31:13.444 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T22:31:13.568 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.582 7f303879a700 1 Processor -- start 2024-04-17T22:31:13.568 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.582 7f303879a700 1 -- start start 2024-04-17T22:31:13.571 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.583 7f303879a700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f303010e260 0x7f303010e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:13.571 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.583 7f303879a700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f30300f1f50 0x7f30301ab500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:13.571 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.583 7f303879a700 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30300f23a0 con 0x7f303010ece0 2024-04-17T22:31:13.571 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.583 7f303879a700 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30300f9f30 con 0x7f303010e260 2024-04-17T22:31:13.571 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.583 7f303879a700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30300f9050 con 0x7f30300f1f50 2024-04-17T22:31:13.571 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.583 7f3035595700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f303010e260 0x7f303010e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:13.571 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.583 7f3035d96700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f30300f1f50 0x7f30301ab500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:13.571 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.583 7f3034d94700 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f303010ece0 0x7f30300f1750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:36322/0 (socket says 172.21.15.102:36322) 2024-04-17T22:31:13.571 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.583 7f3034d94700 1 -- 172.21.15.102:0/2080529768 learned_addr learned my addr 172.21.15.102:0/2080529768 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-17T22:31:13.571 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.583 7f3035595700 1 -- 172.21.15.102:0/2080529768 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f30300f1f50 msgr2=0x7f30301ab500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:31:13.571 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.583 7f3035595700 1 --2- 172.21.15.102:0/2080529768 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f30300f1f50 0x7f30301ab500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:13.571 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.583 7f3035595700 1 -- 172.21.15.102:0/2080529768 >> v1:172.21.15.102:6789/0 conn(0x7f303010ece0 legacy=0x7f30300f1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:13.571 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.583 7f3035595700 1 -- 172.21.15.102:0/2080529768 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30300f25c0 con 0x7f303010e260 2024-04-17T22:31:13.572 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.583 7f301ffff700 1 -- 172.21.15.102:0/2080529768 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2052378944 0 0) 0x7f302c001110 con 0x7f303010ece0 2024-04-17T22:31:13.572 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.583 7f3035595700 1 --2- 172.21.15.102:0/2080529768 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f303010e260 0x7f303010e6b0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 rx=0x7f30300f46c0 tx=0x7f302000ba20).ready entity=mon.0 client_cookie=5e7a6df3f5a90c0a server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:13.572 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.583 7f301ffff700 1 -- 172.21.15.102:0/2080529768 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f30300f25c0 con 0x7f303010e260 2024-04-17T22:31:13.572 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.583 7f301ffff700 1 -- 172.21.15.102:0/2080529768 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f302000d5c0 con 0x7f303010e260 2024-04-17T22:31:13.572 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.583 7f301ffff700 1 -- 172.21.15.102:0/2080529768 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f302000d990 con 0x7f303010e260 2024-04-17T22:31:13.572 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.583 7f303879a700 1 -- 172.21.15.102:0/2080529768 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f303010e260 msgr2=0x7f303010e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:13.572 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.583 7f303879a700 1 --2- 172.21.15.102:0/2080529768 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f303010e260 0x7f303010e6b0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 rx=0x7f30300f46c0 tx=0x7f302000ba20).stop 2024-04-17T22:31:13.572 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.584 7f303879a700 1 -- 172.21.15.102:0/2080529768 shutdown_connections 2024-04-17T22:31:13.572 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.584 7f303879a700 1 --2- 172.21.15.102:0/2080529768 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f30300f1f50 0x7f30301ab500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:13.572 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.584 7f303879a700 1 --2- 172.21.15.102:0/2080529768 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f303010e260 0x7f303010e6b0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:13.572 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.584 7f303879a700 1 -- 172.21.15.102:0/2080529768 >> 172.21.15.102:0/2080529768 conn(0x7f30301138c0 msgr2=0x7f30300f4f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:13.572 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.584 7f303879a700 1 -- 172.21.15.102:0/2080529768 shutdown_connections 2024-04-17T22:31:13.572 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.584 7f303879a700 1 -- 172.21.15.102:0/2080529768 wait complete. 2024-04-17T22:31:13.595 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.607 7f303879a700 1 Processor -- start 2024-04-17T22:31:13.595 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.607 7f303879a700 1 -- start start 2024-04-17T22:31:13.595 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.607 7f303879a700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f30300d5280 0x7f30300d56d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:13.595 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.607 7f303879a700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f30300d5d00 0x7f30301b4220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:13.595 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.607 7f303879a700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f30301b4850 0x7f30301b4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:13.595 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.607 7f303879a700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30300fa1d0 con 0x7f30300d5d00 2024-04-17T22:31:13.595 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.607 7f303879a700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30301b52d0 con 0x7f30301b4850 2024-04-17T22:31:13.595 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.607 7f303879a700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f30300c4700 con 0x7f30300d5280 2024-04-17T22:31:13.595 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.607 7f3034d94700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f30300d5d00 0x7f30301b4220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:13.596 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.607 7f3035595700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f30300d5280 0x7f30300d56d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:13.596 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.607 7f3034d94700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f30300d5d00 0x7f30301b4220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:44008/0 (socket says 172.21.15.102:44008) 2024-04-17T22:31:13.596 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.607 7f3034d94700 1 -- 172.21.15.102:0/755737942 learned_addr learned my addr 172.21.15.102:0/755737942 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:31:13.596 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.607 7f3035d96700 1 --2- 172.21.15.102:0/755737942 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f30301b4850 0x7f30301b4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:13.596 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.607 7f3034d94700 1 -- 172.21.15.102:0/755737942 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f30300d5280 msgr2=0x7f30300d56d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:13.596 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.607 7f3034d94700 1 --2- 172.21.15.102:0/755737942 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f30300d5280 0x7f30300d56d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:13.596 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.607 7f3034d94700 1 -- 172.21.15.102:0/755737942 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f30301b4850 msgr2=0x7f30301b4ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:13.596 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.607 7f3034d94700 1 --2- 172.21.15.102:0/755737942 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f30301b4850 0x7f30301b4ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:13.596 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.607 7f3034d94700 1 -- 172.21.15.102:0/755737942 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30300dadc0 con 0x7f30300d5d00 2024-04-17T22:31:13.596 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.607 7f3035595700 1 --2- 172.21.15.102:0/755737942 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f30300d5280 0x7f30300d56d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:13.596 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.607 7f3035d96700 1 --2- 172.21.15.102:0/755737942 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f30301b4850 0x7f30301b4ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:13.596 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.608 7f3034d94700 1 --2- 172.21.15.102:0/755737942 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f30300d5d00 0x7f30301b4220 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 rx=0x7f30300c25c0 tx=0x7f302c00c890).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:13.596 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.608 7f301e7fc700 1 -- 172.21.15.102:0/755737942 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f30300dadc0 con 0x7f30300d5d00 2024-04-17T22:31:13.596 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.608 7f301e7fc700 1 -- 172.21.15.102:0/755737942 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f302c0072e0 con 0x7f30300d5d00 2024-04-17T22:31:13.597 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.608 7f303879a700 1 -- 172.21.15.102:0/755737942 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30300da9a0 con 0x7f30300d5d00 2024-04-17T22:31:13.597 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.608 7f301e7fc700 1 -- 172.21.15.102:0/755737942 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f302c005810 con 0x7f30300d5d00 2024-04-17T22:31:13.597 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.608 7f303879a700 1 -- 172.21.15.102:0/755737942 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30301bb9a0 con 0x7f30300d5d00 2024-04-17T22:31:13.597 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.608 7f301e7fc700 1 -- 172.21.15.102:0/755737942 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7f302c005a10 con 0x7f30300d5d00 2024-04-17T22:31:13.597 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.608 7f301e7fc700 1 -- 172.21.15.102:0/755737942 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3194+0+0 (secure 0 0 0) 0x7f302c009d60 con 0x7f30300d5d00 2024-04-17T22:31:13.597 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.609 7f303879a700 1 -- 172.21.15.102:0/755737942 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30300f0820 con 0x7f30300d5d00 2024-04-17T22:31:13.598 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.612 7f301e7fc700 1 -- 172.21.15.102:0/755737942 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+110455 (secure 0 0 0) 0x7f302c0070d0 con 0x7f30300d5d00 2024-04-17T22:31:13.731 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.745 7f303879a700 1 -- 172.21.15.102:0/755737942 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f30301ae920 con 0x7f30300d5d00 2024-04-17T22:31:13.731 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.745 7f301e7fc700 1 -- 172.21.15.102:0/755737942 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) v1 ==== 93+0+1704 (secure 0 0 0) 0x7f302c000c30 con 0x7f30300d5d00 2024-04-17T22:31:13.732 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-17T22:31:13.732 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":5,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[],"filesystems":[{"mdsmap":{"epoch":5,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17 22:29:07.787730","modified":"2024-04-17 22:29:09.795249","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"-1 (unspecified)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4177},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4177":{"gid":4177,"name":"a","rank":0,"incarnation":3,"state":"up:active","state_seq":4,"addr":"172.21.15.102:6817/1944542598","addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6817","nonce":1944542598}]},"export_targets":[],"features":4611087854035861503,"flags":0}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-04-17T22:31:13.733 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.746 7f303879a700 1 -- 172.21.15.102:0/755737942 >> v1:172.21.15.102:6816/42360 conn(0x7f30100145a0 legacy=0x7f3010016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:13.733 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.746 7f303879a700 1 -- 172.21.15.102:0/755737942 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f30300d5d00 msgr2=0x7f30301b4220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:13.733 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.746 7f303879a700 1 --2- 172.21.15.102:0/755737942 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f30300d5d00 0x7f30301b4220 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 rx=0x7f30300c25c0 tx=0x7f302c00c890).stop 2024-04-17T22:31:13.733 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.747 7f303879a700 1 -- 172.21.15.102:0/755737942 shutdown_connections 2024-04-17T22:31:13.733 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.747 7f303879a700 1 --2- 172.21.15.102:0/755737942 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f30301b4850 0x7f30301b4ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:13.733 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.747 7f303879a700 1 --2- 172.21.15.102:0/755737942 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f30300d5280 0x7f30300d56d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:13.733 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.747 7f303879a700 1 --2- 172.21.15.102:0/755737942 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f30300d5d00 0x7f30301b4220 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:13.733 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.747 7f303879a700 1 -- 172.21.15.102:0/755737942 >> 172.21.15.102:0/755737942 conn(0x7f30300c6af0 msgr2=0x7f30300d62b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:13.733 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.747 7f303879a700 1 -- 172.21.15.102:0/755737942 shutdown_connections 2024-04-17T22:31:13.733 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.747 7f303879a700 1 -- 172.21.15.102:0/755737942 wait complete. 2024-04-17T22:31:13.734 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 5 2024-04-17T22:31:13.743 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T22:31:13.870 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.882 7f35d152c700 1 Processor -- start 2024-04-17T22:31:13.870 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.882 7f35d152c700 1 -- start start 2024-04-17T22:31:13.870 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.882 7f35d152c700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f35cc10e260 0x7f35cc10e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:13.870 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.882 7f35d152c700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f35cc10ece0 0x7f35cc0f1750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:13.870 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.882 7f35d152c700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f35cc0f1d80 0x7f35cc1aa4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:13.870 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.882 7f35d152c700 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35cc0f9f30 con 0x7f35cc10ece0 2024-04-17T22:31:13.870 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.882 7f35d152c700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35cc0f9050 con 0x7f35cc0f1d80 2024-04-17T22:31:13.870 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.882 7f35d152c700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f35cc0fa1d0 con 0x7f35cc10e260 2024-04-17T22:31:13.871 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.882 7f35caab2700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f35cc0f1d80 0x7f35cc1aa4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:13.871 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.882 7f35ca2b1700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f35cc10e260 0x7f35cc10e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:13.871 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.882 7f35c9ab0700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f35cc10ece0 0x7f35cc0f1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:13.871 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.882 7f35ca2b1700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f35cc10e260 0x7f35cc10e6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:48806/0 (socket says 172.21.15.102:48806) 2024-04-17T22:31:13.871 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.882 7f35caab2700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f35cc0f1d80 0x7f35cc1aa4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:58696/0 (socket says 172.21.15.102:58696) 2024-04-17T22:31:13.871 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.882 7f35c9ab0700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f35cc10ece0 0x7f35cc0f1750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:44020/0 (socket says 172.21.15.102:44020) 2024-04-17T22:31:13.871 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.882 7f35ca2b1700 1 -- 172.21.15.102:0/2746008081 learned_addr learned my addr 172.21.15.102:0/2746008081 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:31:13.871 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.883 7f35caab2700 1 -- 172.21.15.102:0/2746008081 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f35cc10e260 msgr2=0x7f35cc10e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:31:13.871 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.883 7f35caab2700 1 --2- 172.21.15.102:0/2746008081 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f35cc10e260 0x7f35cc10e6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:13.871 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.883 7f35caab2700 1 -- 172.21.15.102:0/2746008081 >> v2:172.21.15.102:3300/0 conn(0x7f35cc10ece0 msgr2=0x7f35cc0f1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:31:13.871 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.883 7f35caab2700 1 --2- 172.21.15.102:0/2746008081 >> v2:172.21.15.102:3300/0 conn(0x7f35cc10ece0 0x7f35cc0f1750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:13.871 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.883 7f35caab2700 1 -- 172.21.15.102:0/2746008081 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35cc0f21d0 con 0x7f35cc0f1d80 2024-04-17T22:31:13.871 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.883 7f35ca2b1700 1 --2- 172.21.15.102:0/2746008081 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f35cc10e260 0x7f35cc10e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:13.871 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.883 7f35c9ab0700 1 --2- 172.21.15.102:0/2746008081 >> v2:172.21.15.102:3300/0 conn(0x7f35cc10ece0 0x7f35cc0f1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:13.871 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.883 7f35caab2700 1 --2- 172.21.15.102:0/2746008081 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f35cc0f1d80 0x7f35cc1aa4f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 rx=0x7f35cc0f4490 tx=0x7f35c4009d70).ready entity=mon.1 client_cookie=15299d5be98de7f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:13.872 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.883 7f35c92af700 1 -- 172.21.15.102:0/2746008081 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f35cc0f21d0 con 0x7f35cc0f1d80 2024-04-17T22:31:13.872 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.883 7f35c92af700 1 -- 172.21.15.102:0/2746008081 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35c400b780 con 0x7f35cc0f1d80 2024-04-17T22:31:13.872 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.883 7f35c92af700 1 -- 172.21.15.102:0/2746008081 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f35c400b960 con 0x7f35cc0f1d80 2024-04-17T22:31:13.872 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.883 7f35d152c700 1 -- 172.21.15.102:0/2746008081 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f35cc0f1d80 msgr2=0x7f35cc1aa4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:13.872 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.883 7f35d152c700 1 --2- 172.21.15.102:0/2746008081 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f35cc0f1d80 0x7f35cc1aa4f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 rx=0x7f35cc0f4490 tx=0x7f35c4009d70).stop 2024-04-17T22:31:13.872 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.884 7f35d152c700 1 -- 172.21.15.102:0/2746008081 shutdown_connections 2024-04-17T22:31:13.872 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.884 7f35d152c700 1 --2- 172.21.15.102:0/2746008081 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f35cc0f1d80 0x7f35cc1aa4f0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:13.872 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.884 7f35d152c700 1 --2- 172.21.15.102:0/2746008081 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f35cc10e260 0x7f35cc10e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:13.872 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.884 7f35d152c700 1 --2- 172.21.15.102:0/2746008081 >> v2:172.21.15.102:3300/0 conn(0x7f35cc10ece0 0x7f35cc0f1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:13.872 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.884 7f35d152c700 1 -- 172.21.15.102:0/2746008081 >> 172.21.15.102:0/2746008081 conn(0x7f35cc1138c0 msgr2=0x7f35cc0f4f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:13.872 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.884 7f35d152c700 1 -- 172.21.15.102:0/2746008081 shutdown_connections 2024-04-17T22:31:13.872 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.884 7f35d152c700 1 -- 172.21.15.102:0/2746008081 wait complete. 2024-04-17T22:31:13.892 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.906 7f35d152c700 1 Processor -- start 2024-04-17T22:31:13.892 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.906 7f35d152c700 1 -- start start 2024-04-17T22:31:13.895 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.907 7f35d152c700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f35cc0cd9c0 0x7f35cc1ad400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:13.895 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.907 7f35d152c700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f35cc1ada30 0x7f35cc1b3d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:13.895 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.907 7f35d152c700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f35cc1ade80 0x7f35cc1b43a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:13.895 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.907 7f35d152c700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35cc08f2e0 con 0x7f35cc1ade80 2024-04-17T22:31:13.895 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.907 7f35d152c700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35cc0cd360 con 0x7f35cc0cd9c0 2024-04-17T22:31:13.895 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.907 7f35d152c700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f35cc0cde10 con 0x7f35cc1ada30 2024-04-17T22:31:13.895 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.907 7f35ca2b1700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f35cc0cd9c0 0x7f35cc1ad400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:13.895 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.907 7f35c9ab0700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f35cc1ada30 0x7f35cc1b3d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:13.895 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.907 7f35ca2b1700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f35cc0cd9c0 0x7f35cc1ad400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:58708/0 (socket says 172.21.15.102:58708) 2024-04-17T22:31:13.895 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.907 7f35ca2b1700 1 -- 172.21.15.102:0/1036185259 learned_addr learned my addr 172.21.15.102:0/1036185259 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:31:13.896 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.907 7f35caab2700 1 --2- 172.21.15.102:0/1036185259 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f35cc1ade80 0x7f35cc1b43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:13.896 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.907 7f35c9ab0700 1 -- 172.21.15.102:0/1036185259 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f35cc0cd9c0 msgr2=0x7f35cc1ad400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:13.896 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.907 7f35c9ab0700 1 --2- 172.21.15.102:0/1036185259 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f35cc0cd9c0 0x7f35cc1ad400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:13.896 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.907 7f35c9ab0700 1 -- 172.21.15.102:0/1036185259 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f35cc1ade80 msgr2=0x7f35cc1b43a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:13.896 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.907 7f35c9ab0700 1 --2- 172.21.15.102:0/1036185259 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f35cc1ade80 0x7f35cc1b43a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:13.896 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.907 7f35c9ab0700 1 -- 172.21.15.102:0/1036185259 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35cc1b49d0 con 0x7f35cc1ada30 2024-04-17T22:31:13.896 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.907 7f35ca2b1700 1 --2- 172.21.15.102:0/1036185259 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f35cc0cd9c0 0x7f35cc1ad400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:13.896 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.907 7f35caab2700 1 --2- 172.21.15.102:0/1036185259 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f35cc1ade80 0x7f35cc1b43a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:13.896 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.908 7f35c9ab0700 1 --2- 172.21.15.102:0/1036185259 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f35cc1ada30 0x7f35cc1b3d70 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0x7f35cc0cd8b0 tx=0x7f35bc00ec50).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:13.896 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.908 7f35bb7fe700 1 -- 172.21.15.102:0/1036185259 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f35cc1b49d0 con 0x7f35cc1ada30 2024-04-17T22:31:13.896 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.908 7f35bb7fe700 1 -- 172.21.15.102:0/1036185259 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35bc00c810 con 0x7f35cc1ada30 2024-04-17T22:31:13.896 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.908 7f35d152c700 1 -- 172.21.15.102:0/1036185259 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35cc1b4cf0 con 0x7f35cc1ada30 2024-04-17T22:31:13.896 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.908 7f35bb7fe700 1 -- 172.21.15.102:0/1036185259 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f35bc00f040 con 0x7f35cc1ada30 2024-04-17T22:31:13.896 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.908 7f35d152c700 1 -- 172.21.15.102:0/1036185259 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35cc1bb6d0 con 0x7f35cc1ada30 2024-04-17T22:31:13.897 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.908 7f35bb7fe700 1 -- 172.21.15.102:0/1036185259 <== mon.2 v2:172.21.15.102:3302/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7f35bc011070 con 0x7f35cc1ada30 2024-04-17T22:31:13.897 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.908 7f35bb7fe700 1 -- 172.21.15.102:0/1036185259 <== mon.2 v2:172.21.15.102:3302/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3194+0+0 (secure 0 0 0) 0x7f35bc010040 con 0x7f35cc1ada30 2024-04-17T22:31:13.897 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.909 7f35d152c700 1 -- 172.21.15.102:0/1036185259 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35cc0f0820 con 0x7f35cc1ada30 2024-04-17T22:31:13.898 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:13.912 7f35bb7fe700 1 -- 172.21.15.102:0/1036185259 <== mon.2 v2:172.21.15.102:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+110455 (secure 0 0 0) 0x7f35bc00ed90 con 0x7f35cc1ada30 2024-04-17T22:31:14.003 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.017 7f35d152c700 1 -- 172.21.15.102:0/1036185259 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f35cc1ae490 con 0x7f35cc1ada30 2024-04-17T22:31:14.004 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.018 7f35bb7fe700 1 -- 172.21.15.102:0/1036185259 <== mon.2 v2:172.21.15.102:3302/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) v1 ==== 74+0+8764 (secure 0 0 0) 0x7f35bc000c30 con 0x7f35cc1ada30 2024-04-17T22:31:14.004 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-17T22:31:14.004 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":17,"fsid":"cb63e789-091d-49b1-b120-55f9db526871","created":"2024-04-17 22:28:48.209635","modified":"2024-04-17 22:29:23.257387","last_up_change":"2024-04-17 22:28:54.236215","last_in_change":"2024-04-17 22:28:54.236215","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":4,"require_min_compat_client":"nautilus","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-04-17 22:28:57.478545","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-17 22:29:06.144708","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-17 22:29:06.748924","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3351ef85-ade4-4b90-a8f4-213f4e00ba01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6812","nonce":42955}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6813","nonce":42955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6815","nonce":42955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6814","nonce":42955}]},"public_addr":"172.21.15.102:6812/42955","cluster_addr":"172.21.15.102:6813/42955","heartbeat_back_addr":"172.21.15.102:6815/42955","heartbeat_front_addr":"172.21.15.102:6814/42955","state":["exists","up"]},{"osd":1,"uuid":"04f67eac-f36b-48f5-982d-4cfc776596d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6808","nonce":42958}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6809","nonce":42958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6811","nonce":42958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6810","nonce":42958}]},"public_addr":"172.21.15.102:6808/42958","cluster_addr":"172.21.15.102:6809/42958","heartbeat_back_addr":"172.21.15.102:6811/42958","heartbeat_front_addr":"172.21.15.102:6810/42958","state":["exists","up"]},{"osd":2,"uuid":"d524204b-70b2-40a8-8f73-f9e7d44318af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6804","nonce":42954}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6805","nonce":42954}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6807","nonce":42954}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6806","nonce":42954}]},"public_addr":"172.21.15.102:6804/42954","cluster_addr":"172.21.15.102:6805/42954","heartbeat_back_addr":"172.21.15.102:6807/42954","heartbeat_front_addr":"172.21.15.102:6806/42954","state":["exists","up"]},{"osd":3,"uuid":"4fc9d20c-1ad0-484f-aafa-2c9a7574245b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6800","nonce":42953}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6801","nonce":42953}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6803","nonce":42953}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6802","nonce":42953}]},"public_addr":"172.21.15.102:6800/42953","cluster_addr":"172.21.15.102:6801/42953","heartbeat_back_addr":"172.21.15.102:6803/42953","heartbeat_front_addr":"172.21.15.102:6802/42953","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2024-04-17T22:31:14.006 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.019 7f35d152c700 1 -- 172.21.15.102:0/1036185259 >> v1:172.21.15.102:6816/42360 conn(0x7f35a80145a0 legacy=0x7f35a8016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:14.006 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.019 7f35d152c700 1 -- 172.21.15.102:0/1036185259 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f35cc1ada30 msgr2=0x7f35cc1b3d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:14.006 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.019 7f35d152c700 1 --2- 172.21.15.102:0/1036185259 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f35cc1ada30 0x7f35cc1b3d70 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0x7f35cc0cd8b0 tx=0x7f35bc00ec50).stop 2024-04-17T22:31:14.006 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.019 7f35d152c700 1 -- 172.21.15.102:0/1036185259 shutdown_connections 2024-04-17T22:31:14.006 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.019 7f35d152c700 1 --2- 172.21.15.102:0/1036185259 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f35cc1ade80 0x7f35cc1b43a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:14.006 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.019 7f35d152c700 1 --2- 172.21.15.102:0/1036185259 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f35cc0cd9c0 0x7f35cc1ad400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:14.006 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.019 7f35d152c700 1 --2- 172.21.15.102:0/1036185259 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f35cc1ada30 0x7f35cc1b3d70 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:14.006 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.019 7f35d152c700 1 -- 172.21.15.102:0/1036185259 >> 172.21.15.102:0/1036185259 conn(0x7f35cc0e0f20 msgr2=0x7f35cc0dac40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:14.006 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.019 7f35d152c700 1 -- 172.21.15.102:0/1036185259 shutdown_connections 2024-04-17T22:31:14.006 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.019 7f35d152c700 1 -- 172.21.15.102:0/1036185259 wait complete. 2024-04-17T22:31:14.016 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2024-04-17 22:28:57.478545', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '12', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-17 22:29:06.144708', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-17 22:29:06.748924', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-17T22:31:14.017 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2024-04-17T22:31:14.146 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.158 7f812e38f700 1 Processor -- start 2024-04-17T22:31:14.146 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.158 7f812e38f700 1 -- start start 2024-04-17T22:31:14.146 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.158 7f812e38f700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f81280f5750 0x7f812810c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:14.146 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.158 7f812e38f700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f812810c720 0x7f81281a0be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:14.146 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.158 7f812e38f700 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81281a1a10 con 0x7f812810cb70 2024-04-17T22:31:14.146 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.158 7f812e38f700 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81280f93c0 con 0x7f812810c720 2024-04-17T22:31:14.146 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.158 7f812e38f700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f81281a1c30 con 0x7f81280f5750 2024-04-17T22:31:14.146 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.158 7f811ffff700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f812810c720 0x7f81281a0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:14.146 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.158 7f8127086700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f81280f5750 0x7f812810c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:14.146 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.158 7f811ffff700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f812810c720 0x7f81281a0be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:44036/0 (socket says 172.21.15.102:44036) 2024-04-17T22:31:14.146 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.158 7f8127086700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f81280f5750 0x7f812810c0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:48832/0 (socket says 172.21.15.102:48832) 2024-04-17T22:31:14.147 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.158 7f8127887700 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f812810cb70 0x7f81281a1210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:36338/0 (socket says 172.21.15.102:36338) 2024-04-17T22:31:14.147 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.158 7f811ffff700 1 -- 172.21.15.102:0/1944883864 learned_addr learned my addr 172.21.15.102:0/1944883864 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:31:14.147 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.158 7f811ffff700 1 -- 172.21.15.102:0/1944883864 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f81280f5750 msgr2=0x7f812810c0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:31:14.147 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.158 7f811ffff700 1 --2- 172.21.15.102:0/1944883864 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f81280f5750 0x7f812810c0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:14.147 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.158 7f811ffff700 1 -- 172.21.15.102:0/1944883864 >> v1:172.21.15.102:6789/0 conn(0x7f812810cb70 legacy=0x7f81281a1210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:14.147 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.158 7f8126885700 1 -- 172.21.15.102:0/1944883864 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3130478869 0 0) 0x7f8120001490 con 0x7f812810cb70 2024-04-17T22:31:14.147 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.158 7f811ffff700 1 -- 172.21.15.102:0/1944883864 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81281aa310 con 0x7f812810c720 2024-04-17T22:31:14.147 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.158 7f8127086700 1 --2- 172.21.15.102:0/1944883864 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f81280f5750 0x7f812810c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:14.147 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.159 7f811ffff700 1 --2- 172.21.15.102:0/1944883864 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f812810c720 0x7f81281a0be0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 rx=0x7f81280f9570 tx=0x7f8110009a80).ready entity=mon.0 client_cookie=e4b9c2c1ac5f0d2f server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:14.147 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.159 7f8126885700 1 -- 172.21.15.102:0/1944883864 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f81281aa310 con 0x7f812810c720 2024-04-17T22:31:14.147 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.159 7f8126885700 1 -- 172.21.15.102:0/1944883864 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f811000f510 con 0x7f812810c720 2024-04-17T22:31:14.147 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.159 7f8126885700 1 -- 172.21.15.102:0/1944883864 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f811000f740 con 0x7f812810c720 2024-04-17T22:31:14.147 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.159 7f812e38f700 1 -- 172.21.15.102:0/1944883864 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f812810c720 msgr2=0x7f81281a0be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:14.147 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.159 7f812e38f700 1 --2- 172.21.15.102:0/1944883864 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f812810c720 0x7f81281a0be0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 rx=0x7f81280f9570 tx=0x7f8110009a80).stop 2024-04-17T22:31:14.148 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.159 7f812e38f700 1 -- 172.21.15.102:0/1944883864 shutdown_connections 2024-04-17T22:31:14.148 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.159 7f812e38f700 1 --2- 172.21.15.102:0/1944883864 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f81280f5750 0x7f812810c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:14.148 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.159 7f812e38f700 1 --2- 172.21.15.102:0/1944883864 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f812810c720 0x7f81281a0be0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:14.148 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.159 7f812e38f700 1 -- 172.21.15.102:0/1944883864 >> 172.21.15.102:0/1944883864 conn(0x7f81281138a0 msgr2=0x7f81280f4f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:14.148 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.159 7f812e38f700 1 -- 172.21.15.102:0/1944883864 shutdown_connections 2024-04-17T22:31:14.148 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.159 7f812e38f700 1 -- 172.21.15.102:0/1944883864 wait complete. 2024-04-17T22:31:14.168 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.182 7f812e38f700 1 Processor -- start 2024-04-17T22:31:14.168 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.182 7f812e38f700 1 -- start start 2024-04-17T22:31:14.170 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.182 7f812e38f700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f81280cf240 0x7f81281ab0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:14.170 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.182 7f812e38f700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f81281ab6f0 0x7f81281b1a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:14.170 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.182 7f812e38f700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f81281abb40 0x7f81281b20b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:14.170 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.182 7f812e38f700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81280cd100 con 0x7f81281ab6f0 2024-04-17T22:31:14.170 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.182 7f812e38f700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81280cf690 con 0x7f81281abb40 2024-04-17T22:31:14.171 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.182 7f812e38f700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f81280cd340 con 0x7f81280cf240 2024-04-17T22:31:14.171 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.182 7f8127086700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f81280cf240 0x7f81281ab0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:14.171 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.183 7f811ffff700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f81281ab6f0 0x7f81281b1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:14.171 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.183 7f8127086700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f81280cf240 0x7f81281ab0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:48840/0 (socket says 172.21.15.102:48840) 2024-04-17T22:31:14.171 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.183 7f8127086700 1 -- 172.21.15.102:0/335027318 learned_addr learned my addr 172.21.15.102:0/335027318 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:31:14.171 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.183 7f8127887700 1 --2- 172.21.15.102:0/335027318 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f81281abb40 0x7f81281b20b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:14.171 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.183 7f8127086700 1 -- 172.21.15.102:0/335027318 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f81281abb40 msgr2=0x7f81281b20b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:14.171 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.183 7f8127086700 1 --2- 172.21.15.102:0/335027318 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f81281abb40 0x7f81281b20b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:14.171 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.183 7f8127086700 1 -- 172.21.15.102:0/335027318 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f81281ab6f0 msgr2=0x7f81281b1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:14.171 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.183 7f8127086700 1 --2- 172.21.15.102:0/335027318 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f81281ab6f0 0x7f81281b1a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:14.171 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.183 7f8127086700 1 -- 172.21.15.102:0/335027318 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81281b26e0 con 0x7f81280cf240 2024-04-17T22:31:14.171 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.183 7f8127086700 1 --2- 172.21.15.102:0/335027318 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f81280cf240 0x7f81281ab0c0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 rx=0x7f81280cf840 tx=0x7f811800bf70).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:14.171 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.183 7f8125082700 1 -- 172.21.15.102:0/335027318 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f81281b26e0 con 0x7f81280cf240 2024-04-17T22:31:14.172 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.183 7f8125082700 1 -- 172.21.15.102:0/335027318 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f811800c870 con 0x7f81280cf240 2024-04-17T22:31:14.172 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.183 7f812e38f700 1 -- 172.21.15.102:0/335027318 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81281b2a00 con 0x7f81280cf240 2024-04-17T22:31:14.172 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.183 7f8125082700 1 -- 172.21.15.102:0/335027318 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f811800f040 con 0x7f81280cf240 2024-04-17T22:31:14.172 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.183 7f812e38f700 1 -- 172.21.15.102:0/335027318 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81281b93d0 con 0x7f81280cf240 2024-04-17T22:31:14.172 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.183 7f8125082700 1 -- 172.21.15.102:0/335027318 <== mon.2 v2:172.21.15.102:3302/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7f8118011070 con 0x7f81280cf240 2024-04-17T22:31:14.172 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.184 7f8125082700 1 -- 172.21.15.102:0/335027318 <== mon.2 v2:172.21.15.102:3302/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3194+0+0 (secure 0 0 0) 0x7f8118010040 con 0x7f81280cf240 2024-04-17T22:31:14.172 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.184 7f812e38f700 1 -- 172.21.15.102:0/335027318 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81280f0800 con 0x7f81280cf240 2024-04-17T22:31:14.173 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.187 7f8125082700 1 -- 172.21.15.102:0/335027318 <== mon.2 v2:172.21.15.102:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+110455 (secure 0 0 0) 0x7f811800c470 con 0x7f81280cf240 2024-04-17T22:31:14.279 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.293 7f812e38f700 1 -- 172.21.15.102:0/335027318 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f81280a64e0 con 0x7f81280cf240 2024-04-17T22:31:14.279 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.293 7f8125082700 1 -- 172.21.15.102:0/335027318 <== mon.2 v2:172.21.15.102:3302/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v17) v1 ==== 92+0+10 (secure 0 0 0) 0x7f8118000c30 con 0x7f81280cf240 2024-04-17T22:31:14.279 INFO:teuthology.orchestra.run.smithi102.stdout:pg_num: 8 2024-04-17T22:31:14.281 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.294 7f812e38f700 1 -- 172.21.15.102:0/335027318 >> v1:172.21.15.102:6816/42360 conn(0x7f810c0145a0 legacy=0x7f810c016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:14.281 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.294 7f812e38f700 1 -- 172.21.15.102:0/335027318 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f81280cf240 msgr2=0x7f81281ab0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:14.281 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.294 7f812e38f700 1 --2- 172.21.15.102:0/335027318 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f81280cf240 0x7f81281ab0c0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 rx=0x7f81280cf840 tx=0x7f811800bf70).stop 2024-04-17T22:31:14.281 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.294 7f812e38f700 1 -- 172.21.15.102:0/335027318 shutdown_connections 2024-04-17T22:31:14.281 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.294 7f812e38f700 1 --2- 172.21.15.102:0/335027318 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f81281abb40 0x7f81281b20b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:14.281 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.294 7f812e38f700 1 --2- 172.21.15.102:0/335027318 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f81280cf240 0x7f81281ab0c0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:14.281 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.294 7f812e38f700 1 --2- 172.21.15.102:0/335027318 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f81281ab6f0 0x7f81281b1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:14.281 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.294 7f812e38f700 1 -- 172.21.15.102:0/335027318 >> 172.21.15.102:0/335027318 conn(0x7f81280e0dc0 msgr2=0x7f81280daae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:14.281 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.294 7f812e38f700 1 -- 172.21.15.102:0/335027318 shutdown_connections 2024-04-17T22:31:14.281 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.294 7f812e38f700 1 -- 172.21.15.102:0/335027318 wait complete. 2024-04-17T22:31:14.291 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-17T22:31:14.421 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.433 7f5d9f1fc700 1 Processor -- start 2024-04-17T22:31:14.421 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.433 7f5d9f1fc700 1 -- start start 2024-04-17T22:31:14.421 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.433 7f5d9f1fc700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f5d980f5940 0x7f5d9810c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:14.421 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.433 7f5d9f1fc700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5d9810c970 0x7f5d980f1920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:14.421 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.433 7f5d9f1fc700 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d980ad030 con 0x7f5d9810cdc0 2024-04-17T22:31:14.421 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.433 7f5d9f1fc700 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d980f94f0 con 0x7f5d980f5940 2024-04-17T22:31:14.422 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.433 7f5d9f1fc700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d980f9250 con 0x7f5d9810c970 2024-04-17T22:31:14.422 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.433 7f5d977fe700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f5d980f5940 0x7f5d9810c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:14.422 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.434 7f5d96ffd700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5d9810c970 0x7f5d980f1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:14.422 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.434 7f5d977fe700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f5d980f5940 0x7f5d9810c340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:44054/0 (socket says 172.21.15.102:44054) 2024-04-17T22:31:14.422 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.434 7f5d97fff700 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f5d9810cdc0 0x7f5d980f1f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:36342/0 (socket says 172.21.15.102:36342) 2024-04-17T22:31:14.422 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.434 7f5d96ffd700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5d9810c970 0x7f5d980f1920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:58732/0 (socket says 172.21.15.102:58732) 2024-04-17T22:31:14.422 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.434 7f5d977fe700 1 -- 172.21.15.102:0/3311012638 learned_addr learned my addr 172.21.15.102:0/3311012638 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:31:14.422 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.434 7f5d977fe700 1 -- 172.21.15.102:0/3311012638 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5d9810c970 msgr2=0x7f5d980f1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:31:14.422 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.434 7f5d977fe700 1 --2- 172.21.15.102:0/3311012638 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5d9810c970 0x7f5d980f1920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:14.422 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.434 7f5d977fe700 1 -- 172.21.15.102:0/3311012638 >> v1:172.21.15.102:6789/0 conn(0x7f5d9810cdc0 legacy=0x7f5d980f1f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:14.422 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.434 7f5d967fc700 1 -- 172.21.15.102:0/3311012638 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2457003284 0 0) 0x7f5d90001490 con 0x7f5d9810cdc0 2024-04-17T22:31:14.422 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.434 7f5d977fe700 1 -- 172.21.15.102:0/3311012638 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d980f2750 con 0x7f5d980f5940 2024-04-17T22:31:14.422 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.434 7f5d96ffd700 1 --2- 172.21.15.102:0/3311012638 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5d9810c970 0x7f5d980f1920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:14.422 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.434 7f5d977fe700 1 --2- 172.21.15.102:0/3311012638 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5d980f5940 0x7f5d9810c340 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 rx=0x7f5d8c002f70 tx=0x7f5d8c009a50).ready entity=mon.0 client_cookie=f9807a0fdd9d4ab0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:14.423 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.434 7f5d967fc700 1 -- 172.21.15.102:0/3311012638 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f5d980f2750 con 0x7f5d980f5940 2024-04-17T22:31:14.423 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.434 7f5d967fc700 1 -- 172.21.15.102:0/3311012638 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d8c00f510 con 0x7f5d980f5940 2024-04-17T22:31:14.423 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.434 7f5d967fc700 1 -- 172.21.15.102:0/3311012638 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f5d8c00f6f0 con 0x7f5d980f5940 2024-04-17T22:31:14.423 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.434 7f5d9f1fc700 1 -- 172.21.15.102:0/3311012638 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5d980f5940 msgr2=0x7f5d9810c340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:14.423 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.434 7f5d9f1fc700 1 --2- 172.21.15.102:0/3311012638 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5d980f5940 0x7f5d9810c340 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 rx=0x7f5d8c002f70 tx=0x7f5d8c009a50).stop 2024-04-17T22:31:14.423 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.435 7f5d9f1fc700 1 -- 172.21.15.102:0/3311012638 shutdown_connections 2024-04-17T22:31:14.423 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.435 7f5d9f1fc700 1 --2- 172.21.15.102:0/3311012638 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5d980f5940 0x7f5d9810c340 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:14.423 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.435 7f5d9f1fc700 1 --2- 172.21.15.102:0/3311012638 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5d9810c970 0x7f5d980f1920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:14.423 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.435 7f5d9f1fc700 1 -- 172.21.15.102:0/3311012638 >> 172.21.15.102:0/3311012638 conn(0x7f5d98113a90 msgr2=0x7f5d980f5140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:14.423 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.435 7f5d9f1fc700 1 -- 172.21.15.102:0/3311012638 shutdown_connections 2024-04-17T22:31:14.423 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.435 7f5d9f1fc700 1 -- 172.21.15.102:0/3311012638 wait complete. 2024-04-17T22:31:14.446 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.458 7f5d9f1fc700 1 Processor -- start 2024-04-17T22:31:14.446 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.458 7f5d9f1fc700 1 -- start start 2024-04-17T22:31:14.446 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.458 7f5d9f1fc700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5d980ca730 0x7f5d981ad170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:14.446 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.458 7f5d9f1fc700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5d981ad7a0 0x7f5d981b3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:14.446 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.458 7f5d9f1fc700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5d981adbf0 0x7f5d981b4300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:14.446 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.458 7f5d9f1fc700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d980cb0d0 con 0x7f5d981adbf0 2024-04-17T22:31:14.446 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.458 7f5d9f1fc700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d981b4930 con 0x7f5d981ad7a0 2024-04-17T22:31:14.446 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.458 7f5d9f1fc700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5d981b4b60 con 0x7f5d980ca730 2024-04-17T22:31:14.446 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.458 7f5d96ffd700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5d981ad7a0 0x7f5d981b3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:14.446 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.458 7f5d97fff700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5d981adbf0 0x7f5d981b4300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:14.447 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.458 7f5d96ffd700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5d981ad7a0 0x7f5d981b3cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:58738/0 (socket says 172.21.15.102:58738) 2024-04-17T22:31:14.447 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.458 7f5d96ffd700 1 -- 172.21.15.102:0/1752686772 learned_addr learned my addr 172.21.15.102:0/1752686772 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:31:14.447 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.458 7f5d977fe700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5d980ca730 0x7f5d981ad170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:14.447 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.459 7f5d96ffd700 1 -- 172.21.15.102:0/1752686772 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5d980ca730 msgr2=0x7f5d981ad170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:14.447 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.459 7f5d96ffd700 1 --2- 172.21.15.102:0/1752686772 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5d980ca730 0x7f5d981ad170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:14.447 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.459 7f5d96ffd700 1 -- 172.21.15.102:0/1752686772 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5d981adbf0 msgr2=0x7f5d981b4300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:14.447 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.459 7f5d96ffd700 1 --2- 172.21.15.102:0/1752686772 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5d981adbf0 0x7f5d981b4300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:14.447 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.459 7f5d96ffd700 1 -- 172.21.15.102:0/1752686772 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d981bb3c0 con 0x7f5d981ad7a0 2024-04-17T22:31:14.447 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.459 7f5d97fff700 1 --2- 172.21.15.102:0/1752686772 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5d981adbf0 0x7f5d981b4300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:14.447 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.459 7f5d977fe700 1 --2- 172.21.15.102:0/1752686772 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5d980ca730 0x7f5d981ad170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:14.447 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.459 7f5d96ffd700 1 --2- 172.21.15.102:0/1752686772 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5d981ad7a0 0x7f5d981b3cd0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 rx=0x7f5d8800c000 tx=0x7f5d8800bf70).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:14.447 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.459 7f5d94ff9700 1 -- 172.21.15.102:0/1752686772 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f5d981bb3c0 con 0x7f5d981ad7a0 2024-04-17T22:31:14.447 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.459 7f5d94ff9700 1 -- 172.21.15.102:0/1752686772 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d8800c9a0 con 0x7f5d981ad7a0 2024-04-17T22:31:14.448 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.459 7f5d94ff9700 1 -- 172.21.15.102:0/1752686772 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f5d8800f040 con 0x7f5d981ad7a0 2024-04-17T22:31:14.448 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.459 7f5d9f1fc700 1 -- 172.21.15.102:0/1752686772 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d981bb5c0 con 0x7f5d981ad7a0 2024-04-17T22:31:14.448 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.459 7f5d9f1fc700 1 -- 172.21.15.102:0/1752686772 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d981bb9a0 con 0x7f5d981ad7a0 2024-04-17T22:31:14.448 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.459 7f5d94ff9700 1 -- 172.21.15.102:0/1752686772 <== mon.1 v2:172.21.15.102:3301/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7f5d88011070 con 0x7f5d981ad7a0 2024-04-17T22:31:14.448 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.459 7f5d94ff9700 1 -- 172.21.15.102:0/1752686772 <== mon.1 v2:172.21.15.102:3301/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3194+0+0 (secure 0 0 0) 0x7f5d88010040 con 0x7f5d981ad7a0 2024-04-17T22:31:14.448 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.460 7f5d9f1fc700 1 -- 172.21.15.102:0/1752686772 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d980f09f0 con 0x7f5d981ad7a0 2024-04-17T22:31:14.449 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.463 7f5d94ff9700 1 -- 172.21.15.102:0/1752686772 <== mon.1 v2:172.21.15.102:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+110455 (secure 0 0 0) 0x7f5d8800c5a0 con 0x7f5d981ad7a0 2024-04-17T22:31:14.554 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.568 7f5d9f1fc700 1 -- 172.21.15.102:0/1752686772 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f5d980a64e0 con 0x7f5d981ad7a0 2024-04-17T22:31:14.555 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.569 7f5d94ff9700 1 -- 172.21.15.102:0/1752686772 <== mon.1 v2:172.21.15.102:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v17) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5d88000c30 con 0x7f5d981ad7a0 2024-04-17T22:31:14.555 INFO:teuthology.orchestra.run.smithi102.stdout:pg_num: 64 2024-04-17T22:31:14.556 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.570 7f5d9f1fc700 1 -- 172.21.15.102:0/1752686772 >> v1:172.21.15.102:6816/42360 conn(0x7f5d780145a0 legacy=0x7f5d78016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:14.556 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.570 7f5d9f1fc700 1 -- 172.21.15.102:0/1752686772 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5d981ad7a0 msgr2=0x7f5d981b3cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:14.556 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.570 7f5d9f1fc700 1 --2- 172.21.15.102:0/1752686772 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5d981ad7a0 0x7f5d981b3cd0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 rx=0x7f5d8800c000 tx=0x7f5d8800bf70).stop 2024-04-17T22:31:14.556 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.570 7f5d9f1fc700 1 -- 172.21.15.102:0/1752686772 shutdown_connections 2024-04-17T22:31:14.556 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.570 7f5d9f1fc700 1 --2- 172.21.15.102:0/1752686772 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5d981adbf0 0x7f5d981b4300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:14.556 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.570 7f5d9f1fc700 1 --2- 172.21.15.102:0/1752686772 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5d980ca730 0x7f5d981ad170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:14.556 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.570 7f5d9f1fc700 1 --2- 172.21.15.102:0/1752686772 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5d981ad7a0 0x7f5d981b3cd0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:14.556 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.570 7f5d9f1fc700 1 -- 172.21.15.102:0/1752686772 >> 172.21.15.102:0/1752686772 conn(0x7f5d980c5bb0 msgr2=0x7f5d980d6890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:14.556 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.570 7f5d9f1fc700 1 -- 172.21.15.102:0/1752686772 shutdown_connections 2024-04-17T22:31:14.556 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.570 7f5d9f1fc700 1 -- 172.21.15.102:0/1752686772 wait complete. 2024-04-17T22:31:14.567 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-17T22:31:14.696 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.708 7f4d1813e700 1 Processor -- start 2024-04-17T22:31:14.696 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.708 7f4d1813e700 1 -- start start 2024-04-17T22:31:14.696 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.708 7f4d1813e700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4d100f5840 0x7f4d1010b010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:14.696 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.708 7f4d1813e700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f4d1010b640 0x7f4d1019faf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:14.697 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.708 7f4d1813e700 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d10103b40 con 0x7f4d1010ba90 2024-04-17T22:31:14.697 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.708 7f4d1813e700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d1010bee0 con 0x7f4d100f5840 2024-04-17T22:31:14.697 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.708 7f4d1813e700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4d100fa370 con 0x7f4d1010b640 2024-04-17T22:31:14.697 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.709 7f4d14f39700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4d100f5840 0x7f4d1010b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:14.697 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.709 7f4d0ffff700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f4d1010b640 0x7f4d1019faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:14.697 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.709 7f4d14f39700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4d100f5840 0x7f4d1010b010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:58746/0 (socket says 172.21.15.102:58746) 2024-04-17T22:31:14.697 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.709 7f4d14f39700 1 -- 172.21.15.102:0/2124211434 learned_addr learned my addr 172.21.15.102:0/2124211434 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:31:14.697 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.709 7f4d14f39700 1 -- 172.21.15.102:0/2124211434 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f4d1010b640 msgr2=0x7f4d1019faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:31:14.697 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.709 7f4d14f39700 1 --2- 172.21.15.102:0/2124211434 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f4d1010b640 0x7f4d1019faf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:14.697 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.709 7f4d14f39700 1 -- 172.21.15.102:0/2124211434 >> v1:172.21.15.102:6789/0 conn(0x7f4d1010ba90 legacy=0x7f4d101a0120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:14.697 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.709 7f4d14f39700 1 -- 172.21.15.102:0/2124211434 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d101a0920 con 0x7f4d100f5840 2024-04-17T22:31:14.697 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.709 7f4d0f7fe700 1 -- 172.21.15.102:0/2124211434 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1189060731 0 0) 0x7f4d08001490 con 0x7f4d1010ba90 2024-04-17T22:31:14.697 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.709 7f4d0ffff700 1 --2- 172.21.15.102:0/2124211434 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f4d1010b640 0x7f4d1019faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:14.697 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.709 7f4d14f39700 1 --2- 172.21.15.102:0/2124211434 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4d100f5840 0x7f4d1010b010 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 rx=0x7f4d10103d60 tx=0x7f4d0400bce0).ready entity=mon.1 client_cookie=8ab1ebdf2323a545 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:14.698 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.709 7f4d0f7fe700 1 -- 172.21.15.102:0/2124211434 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f4d101a0920 con 0x7f4d100f5840 2024-04-17T22:31:14.698 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.709 7f4d0f7fe700 1 -- 172.21.15.102:0/2124211434 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d0400dbc0 con 0x7f4d100f5840 2024-04-17T22:31:14.698 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.709 7f4d0f7fe700 1 -- 172.21.15.102:0/2124211434 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f4d04010470 con 0x7f4d100f5840 2024-04-17T22:31:14.698 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.709 7f4d1813e700 1 -- 172.21.15.102:0/2124211434 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4d100f5840 msgr2=0x7f4d1010b010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:14.698 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.709 7f4d1813e700 1 --2- 172.21.15.102:0/2124211434 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4d100f5840 0x7f4d1010b010 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 rx=0x7f4d10103d60 tx=0x7f4d0400bce0).stop 2024-04-17T22:31:14.698 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.710 7f4d1813e700 1 -- 172.21.15.102:0/2124211434 shutdown_connections 2024-04-17T22:31:14.698 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.710 7f4d1813e700 1 --2- 172.21.15.102:0/2124211434 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4d100f5840 0x7f4d1010b010 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:14.698 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.710 7f4d1813e700 1 --2- 172.21.15.102:0/2124211434 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f4d1010b640 0x7f4d1019faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:14.698 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.710 7f4d1813e700 1 -- 172.21.15.102:0/2124211434 >> 172.21.15.102:0/2124211434 conn(0x7f4d101138a0 msgr2=0x7f4d100f4f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:14.698 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.710 7f4d1813e700 1 -- 172.21.15.102:0/2124211434 shutdown_connections 2024-04-17T22:31:14.698 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.710 7f4d1813e700 1 -- 172.21.15.102:0/2124211434 wait complete. 2024-04-17T22:31:14.718 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.733 7f4d1813e700 1 Processor -- start 2024-04-17T22:31:14.719 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.733 7f4d1813e700 1 -- start start 2024-04-17T22:31:14.721 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.733 7f4d1813e700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f4d100d4e80 0x7f4d100d52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:14.721 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.733 7f4d1813e700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4d100d5900 0x7f4d101afc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:14.721 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.733 7f4d1813e700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4d101b0270 0x7f4d101b06c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:14.721 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.733 7f4d1813e700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d100d9a00 con 0x7f4d101b0270 2024-04-17T22:31:14.721 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.733 7f4d1813e700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d100d97c0 con 0x7f4d100d5900 2024-04-17T22:31:14.722 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.733 7f4d1813e700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4d100c3330 con 0x7f4d100d4e80 2024-04-17T22:31:14.722 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.733 7f4d14f39700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f4d100d4e80 0x7f4d100d52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:14.722 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.733 7f4d1573a700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4d101b0270 0x7f4d101b06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:14.722 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.733 7f4d0ffff700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4d100d5900 0x7f4d101afc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:14.722 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.734 7f4d14f39700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f4d100d4e80 0x7f4d100d52d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:48862/0 (socket says 172.21.15.102:48862) 2024-04-17T22:31:14.722 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.734 7f4d1573a700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4d101b0270 0x7f4d101b06c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:44076/0 (socket says 172.21.15.102:44076) 2024-04-17T22:31:14.722 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.734 7f4d0ffff700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4d100d5900 0x7f4d101afc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:58758/0 (socket says 172.21.15.102:58758) 2024-04-17T22:31:14.722 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.734 7f4d14f39700 1 -- 172.21.15.102:0/900802935 learned_addr learned my addr 172.21.15.102:0/900802935 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:31:14.722 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.734 7f4d1573a700 1 -- 172.21.15.102:0/900802935 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f4d100d4e80 msgr2=0x7f4d100d52d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:14.722 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.734 7f4d1573a700 1 --2- 172.21.15.102:0/900802935 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f4d100d4e80 0x7f4d100d52d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:14.722 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.734 7f4d1573a700 1 -- 172.21.15.102:0/900802935 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4d100d5900 msgr2=0x7f4d101afc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:14.722 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.734 7f4d1573a700 1 --2- 172.21.15.102:0/900802935 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4d100d5900 0x7f4d101afc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:14.722 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.734 7f4d1573a700 1 -- 172.21.15.102:0/900802935 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d101a1900 con 0x7f4d101b0270 2024-04-17T22:31:14.723 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.734 7f4d0ffff700 1 --2- 172.21.15.102:0/900802935 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4d100d5900 0x7f4d101afc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:14.723 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.734 7f4d14f39700 1 --2- 172.21.15.102:0/900802935 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f4d100d4e80 0x7f4d100d52d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:14.723 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.734 7f4d1573a700 1 --2- 172.21.15.102:0/900802935 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4d101b0270 0x7f4d101b06c0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 rx=0x7f4d100c5c10 tx=0x7f4d0800ca10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:14.723 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.734 7f4d0dffb700 1 -- 172.21.15.102:0/900802935 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f4d101a1900 con 0x7f4d101b0270 2024-04-17T22:31:14.723 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.734 7f4d0dffb700 1 -- 172.21.15.102:0/900802935 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d08006ab0 con 0x7f4d101b0270 2024-04-17T22:31:14.723 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.734 7f4d1813e700 1 -- 172.21.15.102:0/900802935 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d101b7330 con 0x7f4d101b0270 2024-04-17T22:31:14.723 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.734 7f4d0dffb700 1 -- 172.21.15.102:0/900802935 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f4d08007030 con 0x7f4d101b0270 2024-04-17T22:31:14.723 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.734 7f4d1813e700 1 -- 172.21.15.102:0/900802935 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d101b7710 con 0x7f4d101b0270 2024-04-17T22:31:14.723 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.735 7f4d0dffb700 1 -- 172.21.15.102:0/900802935 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7f4d0801a020 con 0x7f4d101b0270 2024-04-17T22:31:14.723 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.735 7f4d0dffb700 1 -- 172.21.15.102:0/900802935 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3194+0+0 (secure 0 0 0) 0x7f4d08003fa0 con 0x7f4d101b0270 2024-04-17T22:31:14.723 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.735 7f4d1813e700 1 -- 172.21.15.102:0/900802935 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d100f0800 con 0x7f4d101b0270 2024-04-17T22:31:14.724 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.738 7f4d0dffb700 1 -- 172.21.15.102:0/900802935 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+110455 (secure 0 0 0) 0x7f4d080068a0 con 0x7f4d101b0270 2024-04-17T22:31:14.829 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.843 7f4d1813e700 1 -- 172.21.15.102:0/900802935 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f4d101aa3a0 con 0x7f4d101b0270 2024-04-17T22:31:14.830 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.844 7f4d0dffb700 1 -- 172.21.15.102:0/900802935 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v17) v1 ==== 100+0+11 (secure 0 0 0) 0x7f4d10114420 con 0x7f4d101b0270 2024-04-17T22:31:14.830 INFO:teuthology.orchestra.run.smithi102.stdout:pg_num: 64 2024-04-17T22:31:14.831 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.845 7f4d1813e700 1 -- 172.21.15.102:0/900802935 >> v1:172.21.15.102:6816/42360 conn(0x7f4cf8014540 legacy=0x7f4cf8016a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:14.831 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.845 7f4d1813e700 1 -- 172.21.15.102:0/900802935 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4d101b0270 msgr2=0x7f4d101b06c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:14.831 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.845 7f4d1813e700 1 --2- 172.21.15.102:0/900802935 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4d101b0270 0x7f4d101b06c0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 rx=0x7f4d100c5c10 tx=0x7f4d0800ca10).stop 2024-04-17T22:31:14.832 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.845 7f4d1813e700 1 -- 172.21.15.102:0/900802935 shutdown_connections 2024-04-17T22:31:14.832 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.845 7f4d1813e700 1 --2- 172.21.15.102:0/900802935 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4d101b0270 0x7f4d101b06c0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:14.832 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.845 7f4d1813e700 1 --2- 172.21.15.102:0/900802935 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f4d100d4e80 0x7f4d100d52d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:14.832 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.845 7f4d1813e700 1 --2- 172.21.15.102:0/900802935 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4d100d5900 0x7f4d101afc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:14.832 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.845 7f4d1813e700 1 -- 172.21.15.102:0/900802935 >> 172.21.15.102:0/900802935 conn(0x7f4d100dff20 msgr2=0x7f4d100c4b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:14.832 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.845 7f4d1813e700 1 -- 172.21.15.102:0/900802935 shutdown_connections 2024-04-17T22:31:14.832 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.845 7f4d1813e700 1 -- 172.21.15.102:0/900802935 wait complete. 2024-04-17T22:31:14.842 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T22:31:14.969 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.980 7f6179d5d700 1 Processor -- start 2024-04-17T22:31:14.969 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.980 7f6179d5d700 1 -- start start 2024-04-17T22:31:14.969 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.981 7f6179d5d700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f61740f57d0 0x7f617410afa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:14.969 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.981 7f6179d5d700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f617410b5d0 0x7f617419fa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:14.969 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.981 7f6179d5d700 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6174103ad0 con 0x7f617410ba20 2024-04-17T22:31:14.969 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.981 7f6179d5d700 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f617410be70 con 0x7f617410b5d0 2024-04-17T22:31:14.969 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.981 7f6179d5d700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61740fa300 con 0x7f61740f57d0 2024-04-17T22:31:14.969 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.981 7f6172ab2700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f61740f57d0 0x7f617410afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:14.969 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.981 7f61722b1700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f617410b5d0 0x7f617419fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:14.969 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.981 7f6172ab2700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f61740f57d0 0x7f617410afa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:58764/0 (socket says 172.21.15.102:58764) 2024-04-17T22:31:14.969 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.981 7f61722b1700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f617410b5d0 0x7f617419fa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:44090/0 (socket says 172.21.15.102:44090) 2024-04-17T22:31:14.969 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.981 7f6172ab2700 1 -- 172.21.15.102:0/90972513 learned_addr learned my addr 172.21.15.102:0/90972513 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:31:14.970 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.981 7f6171ab0700 1 -- 172.21.15.102:0/90972513 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 793757733 0 0) 0x7f616c001490 con 0x7f617410ba20 2024-04-17T22:31:14.970 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.981 7f6172ab2700 1 -- 172.21.15.102:0/90972513 >> v2:172.21.15.102:3300/0 conn(0x7f617410b5d0 msgr2=0x7f617419fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:31:14.970 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.981 7f6172ab2700 1 --2- 172.21.15.102:0/90972513 >> v2:172.21.15.102:3300/0 conn(0x7f617410b5d0 0x7f617419fa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:14.970 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.981 7f6172ab2700 1 -- 172.21.15.102:0/90972513 >> v1:172.21.15.102:6789/0 conn(0x7f617410ba20 legacy=0x7f61741a00b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:14.970 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.981 7f6172ab2700 1 -- 172.21.15.102:0/90972513 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61741a08b0 con 0x7f61740f57d0 2024-04-17T22:31:14.970 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.981 7f61722b1700 1 --2- 172.21.15.102:0/90972513 >> v2:172.21.15.102:3300/0 conn(0x7f617410b5d0 0x7f617419fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:14.970 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.981 7f6172ab2700 1 --2- 172.21.15.102:0/90972513 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f61740f57d0 0x7f617410afa0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 rx=0x7f6168002f70 tx=0x7f61680099d0).ready entity=mon.1 client_cookie=3032f24600b48ef9 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:14.970 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.981 7f6171ab0700 1 -- 172.21.15.102:0/90972513 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f61741a08b0 con 0x7f61740f57d0 2024-04-17T22:31:14.970 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.981 7f6171ab0700 1 -- 172.21.15.102:0/90972513 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f616800f510 con 0x7f61740f57d0 2024-04-17T22:31:14.970 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.981 7f6171ab0700 1 -- 172.21.15.102:0/90972513 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f616800f6f0 con 0x7f61740f57d0 2024-04-17T22:31:14.970 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.981 7f6179d5d700 1 -- 172.21.15.102:0/90972513 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f61740f57d0 msgr2=0x7f617410afa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:14.970 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.981 7f6179d5d700 1 --2- 172.21.15.102:0/90972513 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f61740f57d0 0x7f617410afa0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 rx=0x7f6168002f70 tx=0x7f61680099d0).stop 2024-04-17T22:31:14.970 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.982 7f6179d5d700 1 -- 172.21.15.102:0/90972513 shutdown_connections 2024-04-17T22:31:14.970 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.982 7f6179d5d700 1 --2- 172.21.15.102:0/90972513 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f61740f57d0 0x7f617410afa0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:14.971 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.982 7f6179d5d700 1 --2- 172.21.15.102:0/90972513 >> v2:172.21.15.102:3300/0 conn(0x7f617410b5d0 0x7f617419fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:14.971 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.982 7f6179d5d700 1 -- 172.21.15.102:0/90972513 >> 172.21.15.102:0/90972513 conn(0x7f6174113830 msgr2=0x7f61740f4ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:14.971 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.982 7f6179d5d700 1 -- 172.21.15.102:0/90972513 shutdown_connections 2024-04-17T22:31:14.971 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:14.982 7f6179d5d700 1 -- 172.21.15.102:0/90972513 wait complete. 2024-04-17T22:31:14.990 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.005 7f6179d5d700 1 Processor -- start 2024-04-17T22:31:14.990 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.005 7f6179d5d700 1 -- start start 2024-04-17T22:31:14.993 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.005 7f6179d5d700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f61740d7160 0x7f61740ce1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:14.993 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.005 7f6179d5d700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f61740ce810 0x7f61741afee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:14.993 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.005 7f6179d5d700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f61740cec60 0x7f61741b0510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:14.993 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.005 7f6179d5d700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61740dd010 con 0x7f61740ce810 2024-04-17T22:31:14.993 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.005 7f6179d5d700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61740dd310 con 0x7f61740cec60 2024-04-17T22:31:14.993 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.005 7f6179d5d700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f61741b0b40 con 0x7f61740d7160 2024-04-17T22:31:14.993 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.005 7f6172ab2700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f61740d7160 0x7f61740ce1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:14.993 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.005 7f61732b3700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f61740cec60 0x7f61741b0510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:14.993 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.005 7f61722b1700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f61740ce810 0x7f61741afee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:14.993 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.005 7f6172ab2700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f61740d7160 0x7f61740ce1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:48864/0 (socket says 172.21.15.102:48864) 2024-04-17T22:31:14.993 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.005 7f61722b1700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f61740ce810 0x7f61741afee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:44092/0 (socket says 172.21.15.102:44092) 2024-04-17T22:31:14.993 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.005 7f61732b3700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f61740cec60 0x7f61741b0510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:58774/0 (socket says 172.21.15.102:58774) 2024-04-17T22:31:14.993 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.005 7f6172ab2700 1 -- 172.21.15.102:0/897180744 learned_addr learned my addr 172.21.15.102:0/897180744 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:31:14.994 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.005 7f6172ab2700 1 -- 172.21.15.102:0/897180744 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f61740cec60 msgr2=0x7f61741b0510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:14.994 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.005 7f6172ab2700 1 --2- 172.21.15.102:0/897180744 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f61740cec60 0x7f61741b0510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:14.994 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.005 7f6172ab2700 1 -- 172.21.15.102:0/897180744 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f61740ce810 msgr2=0x7f61741afee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:14.994 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.005 7f6172ab2700 1 --2- 172.21.15.102:0/897180744 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f61740ce810 0x7f61741afee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:14.994 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.005 7f6172ab2700 1 -- 172.21.15.102:0/897180744 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61741b0d20 con 0x7f61740d7160 2024-04-17T22:31:14.994 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.005 7f61722b1700 1 --2- 172.21.15.102:0/897180744 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f61740ce810 0x7f61741afee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:14.994 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.005 7f61732b3700 1 --2- 172.21.15.102:0/897180744 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f61740cec60 0x7f61741b0510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:14.994 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.005 7f6172ab2700 1 --2- 172.21.15.102:0/897180744 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f61740d7160 0x7f61740ce1e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 rx=0x7f61740c44a0 tx=0x7f61680068c0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:14.994 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.006 7f6163fff700 1 -- 172.21.15.102:0/897180744 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f61741b0d20 con 0x7f61740d7160 2024-04-17T22:31:14.994 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.006 7f6163fff700 1 -- 172.21.15.102:0/897180744 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6168011710 con 0x7f61740d7160 2024-04-17T22:31:14.994 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.006 7f6179d5d700 1 -- 172.21.15.102:0/897180744 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61741b7650 con 0x7f61740d7160 2024-04-17T22:31:14.994 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.006 7f6163fff700 1 -- 172.21.15.102:0/897180744 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f6168011ae0 con 0x7f61740d7160 2024-04-17T22:31:14.994 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.006 7f6179d5d700 1 -- 172.21.15.102:0/897180744 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61741b7a30 con 0x7f61740d7160 2024-04-17T22:31:14.995 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.006 7f6163fff700 1 -- 172.21.15.102:0/897180744 <== mon.2 v2:172.21.15.102:3302/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7f6168011ce0 con 0x7f61740d7160 2024-04-17T22:31:14.995 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.006 7f6163fff700 1 -- 172.21.15.102:0/897180744 <== mon.2 v2:172.21.15.102:3302/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3194+0+0 (secure 0 0 0) 0x7f6168001620 con 0x7f61740d7160 2024-04-17T22:31:14.995 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.007 7f6179d5d700 1 -- 172.21.15.102:0/897180744 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61740f0790 con 0x7f61740d7160 2024-04-17T22:31:14.996 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.010 7f6163fff700 1 -- 172.21.15.102:0/897180744 <== mon.2 v2:172.21.15.102:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+110455 (secure 0 0 0) 0x7f6168011500 con 0x7f61740d7160 2024-04-17T22:31:15.131 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.145 7f6179d5d700 1 -- 172.21.15.102:0/897180744 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f61741aa600 con 0x7f61740d7160 2024-04-17T22:31:15.132 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.146 7f6163fff700 1 -- 172.21.15.102:0/897180744 <== mon.2 v2:172.21.15.102:3302/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) v1 ==== 93+0+1704 (secure 0 0 0) 0x7f616800f300 con 0x7f61740d7160 2024-04-17T22:31:15.132 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-17T22:31:15.132 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":5,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[],"filesystems":[{"mdsmap":{"epoch":5,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17 22:29:07.787730","modified":"2024-04-17 22:29:09.795249","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"-1 (unspecified)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4177},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4177":{"gid":4177,"name":"a","rank":0,"incarnation":3,"state":"up:active","state_seq":4,"addr":"172.21.15.102:6817/1944542598","addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6817","nonce":1944542598}]},"export_targets":[],"features":4611087854035861503,"flags":0}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-04-17T22:31:15.133 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.147 7f6179d5d700 1 -- 172.21.15.102:0/897180744 >> v1:172.21.15.102:6816/42360 conn(0x7f61500145a0 legacy=0x7f6150016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:15.133 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.147 7f6179d5d700 1 -- 172.21.15.102:0/897180744 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f61740d7160 msgr2=0x7f61740ce1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:15.134 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.147 7f6179d5d700 1 --2- 172.21.15.102:0/897180744 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f61740d7160 0x7f61740ce1e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 rx=0x7f61740c44a0 tx=0x7f61680068c0).stop 2024-04-17T22:31:15.134 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.147 7f6179d5d700 1 -- 172.21.15.102:0/897180744 shutdown_connections 2024-04-17T22:31:15.134 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.147 7f6179d5d700 1 --2- 172.21.15.102:0/897180744 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f61740cec60 0x7f61741b0510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:15.134 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.147 7f6179d5d700 1 --2- 172.21.15.102:0/897180744 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f61740d7160 0x7f61740ce1e0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:15.134 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.147 7f6179d5d700 1 --2- 172.21.15.102:0/897180744 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f61740ce810 0x7f61741afee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:15.134 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.147 7f6179d5d700 1 -- 172.21.15.102:0/897180744 >> 172.21.15.102:0/897180744 conn(0x7f61740e0470 msgr2=0x7f61741a4260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:15.134 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.147 7f6179d5d700 1 -- 172.21.15.102:0/897180744 shutdown_connections 2024-04-17T22:31:15.134 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.147 7f6179d5d700 1 -- 172.21.15.102:0/897180744 wait complete. 2024-04-17T22:31:15.134 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 5 2024-04-17T22:31:15.144 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T22:31:15.270 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.282 7f6a4f828700 1 Processor -- start 2024-04-17T22:31:15.271 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.282 7f6a4f828700 1 -- start start 2024-04-17T22:31:15.271 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.283 7f6a4f828700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f6a480f56e0 0x7f6a4810c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:15.271 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.283 7f6a4f828700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6a4810c6b0 0x7f6a481a0b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:15.271 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.283 7f6a4f828700 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a481a19a0 con 0x7f6a4810cb00 2024-04-17T22:31:15.271 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.283 7f6a4f828700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a480f9350 con 0x7f6a4810c6b0 2024-04-17T22:31:15.271 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.283 7f6a4f828700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6a481a1bc0 con 0x7f6a480f56e0 2024-04-17T22:31:15.271 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.283 7f6a47fff700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f6a480f56e0 0x7f6a4810c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:15.271 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.283 7f6a3ffff700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6a4810c6b0 0x7f6a481a0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:15.271 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.283 7f6a3ffff700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6a4810c6b0 0x7f6a481a0b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:58782/0 (socket says 172.21.15.102:58782) 2024-04-17T22:31:15.271 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.283 7f6a47fff700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f6a480f56e0 0x7f6a4810c080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:48878/0 (socket says 172.21.15.102:48878) 2024-04-17T22:31:15.271 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.283 7f6a3ffff700 1 -- 172.21.15.102:0/2098402255 learned_addr learned my addr 172.21.15.102:0/2098402255 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:31:15.271 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.283 7f6a4ce24700 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f6a4810cb00 0x7f6a481a11a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:36376/0 (socket says 172.21.15.102:36376) 2024-04-17T22:31:15.271 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.283 7f6a3ffff700 1 -- 172.21.15.102:0/2098402255 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f6a480f56e0 msgr2=0x7f6a4810c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:31:15.271 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.283 7f6a3ffff700 1 --2- 172.21.15.102:0/2098402255 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f6a480f56e0 0x7f6a4810c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:15.272 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.283 7f6a3ffff700 1 -- 172.21.15.102:0/2098402255 >> v1:172.21.15.102:6789/0 conn(0x7f6a4810cb00 legacy=0x7f6a481a11a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:15.272 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.283 7f6a477fe700 1 -- 172.21.15.102:0/2098402255 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 559957049 0 0) 0x7f6a40001490 con 0x7f6a4810cb00 2024-04-17T22:31:15.272 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.283 7f6a3ffff700 1 -- 172.21.15.102:0/2098402255 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a481aa2a0 con 0x7f6a4810c6b0 2024-04-17T22:31:15.272 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.283 7f6a47fff700 1 --2- 172.21.15.102:0/2098402255 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f6a480f56e0 0x7f6a4810c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:15.272 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.283 7f6a3ffff700 1 --2- 172.21.15.102:0/2098402255 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6a4810c6b0 0x7f6a481a0b70 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 rx=0x7f6a480f9500 tx=0x7f6a30009ab0).ready entity=mon.1 client_cookie=949dccae355cd9ee server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:15.272 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.283 7f6a477fe700 1 -- 172.21.15.102:0/2098402255 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f6a481aa2a0 con 0x7f6a4810c6b0 2024-04-17T22:31:15.272 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.283 7f6a477fe700 1 -- 172.21.15.102:0/2098402255 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a3000f510 con 0x7f6a4810c6b0 2024-04-17T22:31:15.272 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.283 7f6a477fe700 1 -- 172.21.15.102:0/2098402255 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f6a3000f6f0 con 0x7f6a4810c6b0 2024-04-17T22:31:15.272 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.283 7f6a4f828700 1 -- 172.21.15.102:0/2098402255 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6a4810c6b0 msgr2=0x7f6a481a0b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:15.272 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.283 7f6a4f828700 1 --2- 172.21.15.102:0/2098402255 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6a4810c6b0 0x7f6a481a0b70 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 rx=0x7f6a480f9500 tx=0x7f6a30009ab0).stop 2024-04-17T22:31:15.272 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.284 7f6a4f828700 1 -- 172.21.15.102:0/2098402255 shutdown_connections 2024-04-17T22:31:15.272 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.284 7f6a4f828700 1 --2- 172.21.15.102:0/2098402255 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f6a480f56e0 0x7f6a4810c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:15.272 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.284 7f6a4f828700 1 --2- 172.21.15.102:0/2098402255 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6a4810c6b0 0x7f6a481a0b70 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:15.272 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.284 7f6a4f828700 1 -- 172.21.15.102:0/2098402255 >> 172.21.15.102:0/2098402255 conn(0x7f6a48113830 msgr2=0x7f6a480f4ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:15.273 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.284 7f6a4f828700 1 -- 172.21.15.102:0/2098402255 shutdown_connections 2024-04-17T22:31:15.273 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.284 7f6a4f828700 1 -- 172.21.15.102:0/2098402255 wait complete. 2024-04-17T22:31:15.295 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.307 7f6a4f828700 1 Processor -- start 2024-04-17T22:31:15.295 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.307 7f6a4f828700 1 -- start start 2024-04-17T22:31:15.295 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.307 7f6a4f828700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6a480dc020 0x7f6a480d8070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:15.295 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.307 7f6a4f828700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f6a480d86a0 0x7f6a481b3d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:15.295 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.307 7f6a4f828700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6a480d8af0 0x7f6a481b43c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:15.295 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.307 7f6a4f828700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a480f7730 con 0x7f6a480d8af0 2024-04-17T22:31:15.295 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.307 7f6a4f828700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a480cb7b0 con 0x7f6a480dc020 2024-04-17T22:31:15.295 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.307 7f6a4f828700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6a481b49f0 con 0x7f6a480d86a0 2024-04-17T22:31:15.295 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.307 7f6a47fff700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6a480dc020 0x7f6a480d8070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:15.295 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.307 7f6a4ce24700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6a480d8af0 0x7f6a481b43c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:15.295 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.307 7f6a47fff700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6a480dc020 0x7f6a480d8070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:58794/0 (socket says 172.21.15.102:58794) 2024-04-17T22:31:15.296 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.307 7f6a3ffff700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f6a480d86a0 0x7f6a481b3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:15.296 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.307 7f6a47fff700 1 -- 172.21.15.102:0/919136011 learned_addr learned my addr 172.21.15.102:0/919136011 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:31:15.296 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.307 7f6a3ffff700 1 -- 172.21.15.102:0/919136011 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6a480dc020 msgr2=0x7f6a480d8070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:15.296 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.307 7f6a3ffff700 1 --2- 172.21.15.102:0/919136011 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6a480dc020 0x7f6a480d8070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:15.296 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.307 7f6a3ffff700 1 -- 172.21.15.102:0/919136011 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6a480d8af0 msgr2=0x7f6a481b43c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:15.296 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.307 7f6a3ffff700 1 --2- 172.21.15.102:0/919136011 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6a480d8af0 0x7f6a481b43c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:15.296 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.307 7f6a3ffff700 1 -- 172.21.15.102:0/919136011 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a481b4bd0 con 0x7f6a480d86a0 2024-04-17T22:31:15.296 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.307 7f6a47fff700 1 --2- 172.21.15.102:0/919136011 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6a480dc020 0x7f6a480d8070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:15.296 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.307 7f6a4ce24700 1 --2- 172.21.15.102:0/919136011 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6a480d8af0 0x7f6a481b43c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:15.296 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.308 7f6a3ffff700 1 --2- 172.21.15.102:0/919136011 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f6a480d86a0 0x7f6a481b3d90 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0x7f6a480f7da0 tx=0x7f6a300068c0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:15.296 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.308 7f6a45ffb700 1 -- 172.21.15.102:0/919136011 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f6a481b4bd0 con 0x7f6a480d86a0 2024-04-17T22:31:15.296 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.308 7f6a45ffb700 1 -- 172.21.15.102:0/919136011 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a30011710 con 0x7f6a480d86a0 2024-04-17T22:31:15.296 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.308 7f6a4f828700 1 -- 172.21.15.102:0/919136011 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a480c6160 con 0x7f6a480d86a0 2024-04-17T22:31:15.296 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.308 7f6a45ffb700 1 -- 172.21.15.102:0/919136011 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f6a30021070 con 0x7f6a480d86a0 2024-04-17T22:31:15.296 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.308 7f6a4f828700 1 -- 172.21.15.102:0/919136011 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a480c6630 con 0x7f6a480d86a0 2024-04-17T22:31:15.296 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.308 7f6a45ffb700 1 -- 172.21.15.102:0/919136011 <== mon.2 v2:172.21.15.102:3302/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7f6a30019030 con 0x7f6a480d86a0 2024-04-17T22:31:15.297 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.308 7f6a45ffb700 1 -- 172.21.15.102:0/919136011 <== mon.2 v2:172.21.15.102:3302/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3194+0+0 (secure 0 0 0) 0x7f6a30018040 con 0x7f6a480d86a0 2024-04-17T22:31:15.297 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.309 7f6a4f828700 1 -- 172.21.15.102:0/919136011 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a480f0790 con 0x7f6a480d86a0 2024-04-17T22:31:15.297 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.313 7f6a45ffb700 1 -- 172.21.15.102:0/919136011 <== mon.2 v2:172.21.15.102:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+110455 (secure 0 0 0) 0x7f6a30011500 con 0x7f6a480d86a0 2024-04-17T22:31:15.402 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.416 7f6a4f828700 1 -- 172.21.15.102:0/919136011 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6a481ae4b0 con 0x7f6a480d86a0 2024-04-17T22:31:15.403 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.417 7f6a45ffb700 1 -- 172.21.15.102:0/919136011 <== mon.2 v2:172.21.15.102:3302/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) v1 ==== 74+0+8764 (secure 0 0 0) 0x7f6a3000f300 con 0x7f6a480d86a0 2024-04-17T22:31:15.403 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-17T22:31:15.403 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":17,"fsid":"cb63e789-091d-49b1-b120-55f9db526871","created":"2024-04-17 22:28:48.209635","modified":"2024-04-17 22:29:23.257387","last_up_change":"2024-04-17 22:28:54.236215","last_in_change":"2024-04-17 22:28:54.236215","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":4,"require_min_compat_client":"nautilus","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-04-17 22:28:57.478545","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-17 22:29:06.144708","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-17 22:29:06.748924","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3351ef85-ade4-4b90-a8f4-213f4e00ba01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6812","nonce":42955}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6813","nonce":42955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6815","nonce":42955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6814","nonce":42955}]},"public_addr":"172.21.15.102:6812/42955","cluster_addr":"172.21.15.102:6813/42955","heartbeat_back_addr":"172.21.15.102:6815/42955","heartbeat_front_addr":"172.21.15.102:6814/42955","state":["exists","up"]},{"osd":1,"uuid":"04f67eac-f36b-48f5-982d-4cfc776596d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6808","nonce":42958}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6809","nonce":42958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6811","nonce":42958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6810","nonce":42958}]},"public_addr":"172.21.15.102:6808/42958","cluster_addr":"172.21.15.102:6809/42958","heartbeat_back_addr":"172.21.15.102:6811/42958","heartbeat_front_addr":"172.21.15.102:6810/42958","state":["exists","up"]},{"osd":2,"uuid":"d524204b-70b2-40a8-8f73-f9e7d44318af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6804","nonce":42954}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6805","nonce":42954}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6807","nonce":42954}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6806","nonce":42954}]},"public_addr":"172.21.15.102:6804/42954","cluster_addr":"172.21.15.102:6805/42954","heartbeat_back_addr":"172.21.15.102:6807/42954","heartbeat_front_addr":"172.21.15.102:6806/42954","state":["exists","up"]},{"osd":3,"uuid":"4fc9d20c-1ad0-484f-aafa-2c9a7574245b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6800","nonce":42953}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6801","nonce":42953}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6803","nonce":42953}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6802","nonce":42953}]},"public_addr":"172.21.15.102:6800/42953","cluster_addr":"172.21.15.102:6801/42953","heartbeat_back_addr":"172.21.15.102:6803/42953","heartbeat_front_addr":"172.21.15.102:6802/42953","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2024-04-17T22:31:15.404 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.418 7f6a4f828700 1 -- 172.21.15.102:0/919136011 >> v1:172.21.15.102:6816/42360 conn(0x7f6a2c0145a0 legacy=0x7f6a2c016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:15.405 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.418 7f6a4f828700 1 -- 172.21.15.102:0/919136011 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f6a480d86a0 msgr2=0x7f6a481b3d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:15.405 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.418 7f6a4f828700 1 --2- 172.21.15.102:0/919136011 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f6a480d86a0 0x7f6a481b3d90 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0x7f6a480f7da0 tx=0x7f6a300068c0).stop 2024-04-17T22:31:15.405 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.418 7f6a4f828700 1 -- 172.21.15.102:0/919136011 shutdown_connections 2024-04-17T22:31:15.405 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.418 7f6a4f828700 1 --2- 172.21.15.102:0/919136011 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6a480d8af0 0x7f6a481b43c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:15.405 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.418 7f6a4f828700 1 --2- 172.21.15.102:0/919136011 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6a480dc020 0x7f6a480d8070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:15.405 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.418 7f6a4f828700 1 --2- 172.21.15.102:0/919136011 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f6a480d86a0 0x7f6a481b3d90 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:15.405 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.418 7f6a4f828700 1 -- 172.21.15.102:0/919136011 >> 172.21.15.102:0/919136011 conn(0x7f6a480d1560 msgr2=0x7f6a480f6200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:15.405 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.418 7f6a4f828700 1 -- 172.21.15.102:0/919136011 shutdown_connections 2024-04-17T22:31:15.405 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.418 7f6a4f828700 1 -- 172.21.15.102:0/919136011 wait complete. 2024-04-17T22:31:15.416 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-17T22:31:15.417 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T22:31:15.543 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.555 7f32e1aa3700 1 Processor -- start 2024-04-17T22:31:15.543 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.555 7f32e1aa3700 1 -- start start 2024-04-17T22:31:15.544 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.555 7f32e1aa3700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f32dc0f5770 0x7f32dc10c110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:15.544 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.555 7f32e1aa3700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f32dc10c740 0x7f32dc1a0c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:15.544 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.555 7f32e1aa3700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f32dc10cb90 0x7f32dc1a1230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:15.544 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.555 7f32e1aa3700 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32dc0f93e0 con 0x7f32dc10cb90 2024-04-17T22:31:15.544 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.555 7f32e1aa3700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32dc1a1860 con 0x7f32dc10c740 2024-04-17T22:31:15.544 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.555 7f32e1aa3700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f32dc1a1a90 con 0x7f32dc0f5770 2024-04-17T22:31:15.544 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.556 7f32da05c700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f32dc10c740 0x7f32dc1a0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:15.544 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.556 7f32da85d700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f32dc0f5770 0x7f32dc10c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:15.544 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.556 7f32db05e700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f32dc10cb90 0x7f32dc1a1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:15.544 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.556 7f32da05c700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f32dc10c740 0x7f32dc1a0c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:58804/0 (socket says 172.21.15.102:58804) 2024-04-17T22:31:15.544 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.556 7f32da85d700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f32dc0f5770 0x7f32dc10c110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:48894/0 (socket says 172.21.15.102:48894) 2024-04-17T22:31:15.544 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.556 7f32da05c700 1 -- 172.21.15.102:0/2852738159 learned_addr learned my addr 172.21.15.102:0/2852738159 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:31:15.544 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.556 7f32db05e700 1 --2- 172.21.15.102:0/2852738159 >> v2:172.21.15.102:3300/0 conn(0x7f32dc10cb90 0x7f32dc1a1230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:44098/0 (socket says 172.21.15.102:44098) 2024-04-17T22:31:15.545 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.556 7f32db05e700 1 -- 172.21.15.102:0/2852738159 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f32dc0f5770 msgr2=0x7f32dc10c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:31:15.545 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.556 7f32db05e700 1 --2- 172.21.15.102:0/2852738159 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f32dc0f5770 0x7f32dc10c110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:15.545 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.556 7f32db05e700 1 -- 172.21.15.102:0/2852738159 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f32dc10c740 msgr2=0x7f32dc1a0c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:31:15.545 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.556 7f32db05e700 1 --2- 172.21.15.102:0/2852738159 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f32dc10c740 0x7f32dc1a0c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:15.545 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.556 7f32db05e700 1 -- 172.21.15.102:0/2852738159 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32dc1a8310 con 0x7f32dc10cb90 2024-04-17T22:31:15.545 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.556 7f32da85d700 1 --2- 172.21.15.102:0/2852738159 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f32dc0f5770 0x7f32dc10c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:15.545 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.556 7f32db05e700 1 --2- 172.21.15.102:0/2852738159 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f32dc10cb90 0x7f32dc1a1230 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 rx=0x7f32d40032f0 tx=0x7f32d4009d10).ready entity=mon.0 client_cookie=f84806a7203d73bb server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:15.545 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.556 7f32d985b700 1 -- 172.21.15.102:0/2852738159 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f32dc1a8310 con 0x7f32dc10cb90 2024-04-17T22:31:15.545 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.556 7f32d985b700 1 -- 172.21.15.102:0/2852738159 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32d400b800 con 0x7f32dc10cb90 2024-04-17T22:31:15.545 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.556 7f32d985b700 1 -- 172.21.15.102:0/2852738159 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f32d400b9e0 con 0x7f32dc10cb90 2024-04-17T22:31:15.545 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.556 7f32e1aa3700 1 -- 172.21.15.102:0/2852738159 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f32dc10cb90 msgr2=0x7f32dc1a1230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:15.545 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.556 7f32e1aa3700 1 --2- 172.21.15.102:0/2852738159 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f32dc10cb90 0x7f32dc1a1230 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 rx=0x7f32d40032f0 tx=0x7f32d4009d10).stop 2024-04-17T22:31:15.545 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.557 7f32e1aa3700 1 -- 172.21.15.102:0/2852738159 shutdown_connections 2024-04-17T22:31:15.545 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.557 7f32e1aa3700 1 --2- 172.21.15.102:0/2852738159 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f32dc10cb90 0x7f32dc1a1230 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:15.546 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.557 7f32e1aa3700 1 --2- 172.21.15.102:0/2852738159 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f32dc0f5770 0x7f32dc10c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:15.546 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.557 7f32e1aa3700 1 --2- 172.21.15.102:0/2852738159 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f32dc10c740 0x7f32dc1a0c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:15.546 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.557 7f32e1aa3700 1 -- 172.21.15.102:0/2852738159 >> 172.21.15.102:0/2852738159 conn(0x7f32dc1138c0 msgr2=0x7f32dc0f4f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:15.546 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.557 7f32e1aa3700 1 -- 172.21.15.102:0/2852738159 shutdown_connections 2024-04-17T22:31:15.546 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.557 7f32e1aa3700 1 -- 172.21.15.102:0/2852738159 wait complete. 2024-04-17T22:31:15.567 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.579 7f32e1aa3700 1 Processor -- start 2024-04-17T22:31:15.568 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.579 7f32e1aa3700 1 -- start start 2024-04-17T22:31:15.568 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.580 7f32e1aa3700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f32dc0db160 0x7f32dc1a92d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:15.568 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.580 7f32e1aa3700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f32dc1a9900 0x7f32dc1afb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:15.568 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.580 7f32e1aa3700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f32dc1a9d50 0x7f32dc1b01b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:15.568 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.580 7f32e1aa3700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32dc0e2d30 con 0x7f32dc1a9d50 2024-04-17T22:31:15.568 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.580 7f32e1aa3700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32dc0d8000 con 0x7f32dc0db160 2024-04-17T22:31:15.568 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.580 7f32e1aa3700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f32dc1b07e0 con 0x7f32dc1a9900 2024-04-17T22:31:15.568 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.580 7f32da05c700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f32dc1a9900 0x7f32dc1afb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:15.568 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.580 7f32da85d700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f32dc0db160 0x7f32dc1a92d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:15.568 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.580 7f32db05e700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f32dc1a9d50 0x7f32dc1b01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:15.568 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.580 7f32da05c700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f32dc1a9900 0x7f32dc1afb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:48898/0 (socket says 172.21.15.102:48898) 2024-04-17T22:31:15.568 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.580 7f32da85d700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f32dc0db160 0x7f32dc1a92d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:58810/0 (socket says 172.21.15.102:58810) 2024-04-17T22:31:15.569 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.580 7f32da05c700 1 -- 172.21.15.102:0/3226475994 learned_addr learned my addr 172.21.15.102:0/3226475994 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:31:15.569 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.580 7f32da85d700 1 -- 172.21.15.102:0/3226475994 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f32dc1a9900 msgr2=0x7f32dc1afb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:15.569 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.580 7f32da85d700 1 --2- 172.21.15.102:0/3226475994 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f32dc1a9900 0x7f32dc1afb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:15.569 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.580 7f32da85d700 1 -- 172.21.15.102:0/3226475994 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f32dc1a9d50 msgr2=0x7f32dc1b01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:15.569 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.580 7f32da85d700 1 --2- 172.21.15.102:0/3226475994 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f32dc1a9d50 0x7f32dc1b01b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:15.569 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.580 7f32da85d700 1 -- 172.21.15.102:0/3226475994 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32dc1b0a10 con 0x7f32dc0db160 2024-04-17T22:31:15.569 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.580 7f32db05e700 1 --2- 172.21.15.102:0/3226475994 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f32dc1a9d50 0x7f32dc1b01b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:15.569 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.580 7f32da85d700 1 --2- 172.21.15.102:0/3226475994 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f32dc0db160 0x7f32dc1a92d0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 rx=0x7f32dc0e1340 tx=0x7f32d000eca0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:15.569 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.580 7f32cbfff700 1 -- 172.21.15.102:0/3226475994 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f32dc1b0a10 con 0x7f32dc0db160 2024-04-17T22:31:15.569 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.580 7f32cbfff700 1 -- 172.21.15.102:0/3226475994 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32d000c9a0 con 0x7f32dc0db160 2024-04-17T22:31:15.569 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.580 7f32e1aa3700 1 -- 172.21.15.102:0/3226475994 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32dc1b7380 con 0x7f32dc0db160 2024-04-17T22:31:15.569 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.580 7f32cbfff700 1 -- 172.21.15.102:0/3226475994 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f32d000f040 con 0x7f32dc0db160 2024-04-17T22:31:15.569 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.581 7f32e1aa3700 1 -- 172.21.15.102:0/3226475994 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32dc1b7760 con 0x7f32dc0db160 2024-04-17T22:31:15.569 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.581 7f32cbfff700 1 -- 172.21.15.102:0/3226475994 <== mon.1 v2:172.21.15.102:3301/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7f32d0011070 con 0x7f32dc0db160 2024-04-17T22:31:15.570 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.581 7f32cbfff700 1 -- 172.21.15.102:0/3226475994 <== mon.1 v2:172.21.15.102:3301/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3194+0+0 (secure 0 0 0) 0x7f32d0010040 con 0x7f32dc0db160 2024-04-17T22:31:15.570 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.582 7f32e1aa3700 1 -- 172.21.15.102:0/3226475994 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32dc0c2110 con 0x7f32dc0db160 2024-04-17T22:31:15.570 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.585 7f32cbfff700 1 -- 172.21.15.102:0/3226475994 <== mon.1 v2:172.21.15.102:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+110455 (secure 0 0 0) 0x7f32d000c5a0 con 0x7f32dc0db160 2024-04-17T22:31:15.705 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.719 7f32e1aa3700 1 -- 172.21.15.102:0/3226475994 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f32dc0f0820 con 0x7f32dc0db160 2024-04-17T22:31:15.705 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.719 7f32cbfff700 1 -- 172.21.15.102:0/3226475994 <== mon.1 v2:172.21.15.102:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) v1 ==== 93+0+1704 (secure 0 0 0) 0x7f32d0000c30 con 0x7f32dc0db160 2024-04-17T22:31:15.706 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-17T22:31:15.706 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":5,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[],"filesystems":[{"mdsmap":{"epoch":5,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17 22:29:07.787730","modified":"2024-04-17 22:29:09.795249","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"-1 (unspecified)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4177},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4177":{"gid":4177,"name":"a","rank":0,"incarnation":3,"state":"up:active","state_seq":4,"addr":"172.21.15.102:6817/1944542598","addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6817","nonce":1944542598}]},"export_targets":[],"features":4611087854035861503,"flags":0}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-04-17T22:31:15.707 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.720 7f32e1aa3700 1 -- 172.21.15.102:0/3226475994 >> v1:172.21.15.102:6816/42360 conn(0x7f32b80145a0 legacy=0x7f32b8016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:15.707 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.720 7f32e1aa3700 1 -- 172.21.15.102:0/3226475994 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f32dc0db160 msgr2=0x7f32dc1a92d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:15.707 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.720 7f32e1aa3700 1 --2- 172.21.15.102:0/3226475994 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f32dc0db160 0x7f32dc1a92d0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 rx=0x7f32dc0e1340 tx=0x7f32d000eca0).stop 2024-04-17T22:31:15.707 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.720 7f32e1aa3700 1 -- 172.21.15.102:0/3226475994 shutdown_connections 2024-04-17T22:31:15.707 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.720 7f32e1aa3700 1 --2- 172.21.15.102:0/3226475994 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f32dc1a9d50 0x7f32dc1b01b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:15.707 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.720 7f32e1aa3700 1 --2- 172.21.15.102:0/3226475994 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f32dc0db160 0x7f32dc1a92d0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:15.707 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.720 7f32e1aa3700 1 --2- 172.21.15.102:0/3226475994 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f32dc1a9900 0x7f32dc1afb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:15.707 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.720 7f32e1aa3700 1 -- 172.21.15.102:0/3226475994 >> 172.21.15.102:0/3226475994 conn(0x7f32dc0e0160 msgr2=0x7f32dc0de350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:15.707 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.720 7f32e1aa3700 1 -- 172.21.15.102:0/3226475994 shutdown_connections 2024-04-17T22:31:15.707 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.720 7f32e1aa3700 1 -- 172.21.15.102:0/3226475994 wait complete. 2024-04-17T22:31:15.708 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 5 2024-04-17T22:31:15.717 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T22:31:15.845 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.856 7f7f73b80700 1 Processor -- start 2024-04-17T22:31:15.845 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.856 7f7f73b80700 1 -- start start 2024-04-17T22:31:15.845 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.857 7f7f73b80700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f7f6c1a64d0 0x7f7f6c1a8930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:15.845 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.857 7f7f73b80700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f7f6c1a8f60 0x7f7f6c1ab3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:15.845 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.857 7f7f73b80700 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f6c10eeb0 con 0x7f7f6c10e260 2024-04-17T22:31:15.845 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.857 7f7f73b80700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f6c0f9f30 con 0x7f7f6c1a64d0 2024-04-17T22:31:15.845 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.857 7f7f73b80700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7f6c0f9050 con 0x7f7f6c1a8f60 2024-04-17T22:31:15.845 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.857 7f7f7097b700 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f7f6c10e260 0x7f7f6c10e6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:36388/0 (socket says 172.21.15.102:36388) 2024-04-17T22:31:15.845 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.857 7f7f7117c700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f7f6c1a8f60 0x7f7f6c1ab3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:15.846 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.857 7f7f7097b700 1 -- 172.21.15.102:0/786525415 learned_addr learned my addr 172.21.15.102:0/786525415 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-17T22:31:15.846 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.857 7f7f6bfff700 1 --2- 172.21.15.102:0/786525415 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f7f6c1a64d0 0x7f7f6c1a8930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:15.846 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.857 7f7f6b7fe700 1 -- 172.21.15.102:0/786525415 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 324777549 0 0) 0x7f7f60001110 con 0x7f7f6c10e260 2024-04-17T22:31:15.846 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.857 7f7f6b7fe700 1 -- 172.21.15.102:0/786525415 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7f50003270 con 0x7f7f6c10e260 2024-04-17T22:31:15.846 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.857 7f7f7117c700 1 -- 172.21.15.102:0/786525415 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f7f6c1a64d0 msgr2=0x7f7f6c1a8930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:31:15.846 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.857 7f7f7117c700 1 --2- 172.21.15.102:0/786525415 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f7f6c1a64d0 0x7f7f6c1a8930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:15.846 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.857 7f7f7117c700 1 -- 172.21.15.102:0/786525415 >> v1:172.21.15.102:6789/0 conn(0x7f7f6c10e260 legacy=0x7f7f6c10e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:15.846 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.857 7f7f7117c700 1 -- 172.21.15.102:0/786525415 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f6c1aca00 con 0x7f7f6c1a8f60 2024-04-17T22:31:15.846 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.857 7f7f6bfff700 1 --2- 172.21.15.102:0/786525415 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f7f6c1a64d0 0x7f7f6c1a8930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:15.846 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.857 7f7f7117c700 1 --2- 172.21.15.102:0/786525415 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f7f6c1a8f60 0x7f7f6c1ab3c0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 rx=0x7f7f6c0f95f0 tx=0x7f7f64009cf0).ready entity=mon.2 client_cookie=3b6a3368850215f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:15.846 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.858 7f7f6b7fe700 1 -- 172.21.15.102:0/786525415 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f7f6c1aca00 con 0x7f7f6c1a8f60 2024-04-17T22:31:15.846 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.858 7f7f6b7fe700 1 -- 172.21.15.102:0/786525415 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f6400b840 con 0x7f7f6c1a8f60 2024-04-17T22:31:15.847 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.858 7f7f6b7fe700 1 -- 172.21.15.102:0/786525415 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f7f6400ba20 con 0x7f7f6c1a8f60 2024-04-17T22:31:15.847 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.858 7f7f73b80700 1 -- 172.21.15.102:0/786525415 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f7f6c1a8f60 msgr2=0x7f7f6c1ab3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:15.847 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.858 7f7f73b80700 1 --2- 172.21.15.102:0/786525415 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f7f6c1a8f60 0x7f7f6c1ab3c0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 rx=0x7f7f6c0f95f0 tx=0x7f7f64009cf0).stop 2024-04-17T22:31:15.847 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.858 7f7f73b80700 1 -- 172.21.15.102:0/786525415 shutdown_connections 2024-04-17T22:31:15.847 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.858 7f7f73b80700 1 --2- 172.21.15.102:0/786525415 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f7f6c1a8f60 0x7f7f6c1ab3c0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:15.847 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.858 7f7f73b80700 1 --2- 172.21.15.102:0/786525415 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f7f6c1a64d0 0x7f7f6c1a8930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:15.847 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.858 7f7f73b80700 1 -- 172.21.15.102:0/786525415 >> 172.21.15.102:0/786525415 conn(0x7f7f6c1138c0 msgr2=0x7f7f6c0f4f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:15.847 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.858 7f7f73b80700 1 -- 172.21.15.102:0/786525415 shutdown_connections 2024-04-17T22:31:15.847 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.858 7f7f73b80700 1 -- 172.21.15.102:0/786525415 wait complete. 2024-04-17T22:31:15.867 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.881 7f7f73b80700 1 Processor -- start 2024-04-17T22:31:15.867 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.881 7f7f73b80700 1 -- start start 2024-04-17T22:31:15.869 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.881 7f7f73b80700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f7f6c0cded0 0x7f7f6c0d5ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:15.869 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.882 7f7f73b80700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f7f6c0d6620 0x7f7f6c1b3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:15.870 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.882 7f7f73b80700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7f6c0d6a70 0x7f7f6c1b4520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:15.870 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.882 7f7f73b80700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f6c0fa1d0 con 0x7f7f6c0d6a70 2024-04-17T22:31:15.870 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.882 7f7f73b80700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f6c0cd870 con 0x7f7f6c0cded0 2024-04-17T22:31:15.870 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.882 7f7f73b80700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7f6c0cdab0 con 0x7f7f6c0d6620 2024-04-17T22:31:15.870 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.882 7f7f7097b700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f7f6c0cded0 0x7f7f6c0d5ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:15.870 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.882 7f7f7097b700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f7f6c0cded0 0x7f7f6c0d5ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:58824/0 (socket says 172.21.15.102:58824) 2024-04-17T22:31:15.870 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.882 7f7f6bfff700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f7f6c0d6620 0x7f7f6c1b3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:15.870 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.882 7f7f7097b700 1 -- 172.21.15.102:0/3946281686 learned_addr learned my addr 172.21.15.102:0/3946281686 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:31:15.870 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.882 7f7f7117c700 1 --2- 172.21.15.102:0/3946281686 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7f6c0d6a70 0x7f7f6c1b4520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:15.870 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.882 7f7f7117c700 1 -- 172.21.15.102:0/3946281686 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f7f6c0d6620 msgr2=0x7f7f6c1b3ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:15.870 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.882 7f7f7117c700 1 --2- 172.21.15.102:0/3946281686 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f7f6c0d6620 0x7f7f6c1b3ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:15.873 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.882 7f7f7117c700 1 -- 172.21.15.102:0/3946281686 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f7f6c0cded0 msgr2=0x7f7f6c0d5ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:15.873 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.882 7f7f7117c700 1 --2- 172.21.15.102:0/3946281686 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f7f6c0cded0 0x7f7f6c0d5ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:15.873 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.882 7f7f7117c700 1 -- 172.21.15.102:0/3946281686 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f6c1b4b50 con 0x7f7f6c0d6a70 2024-04-17T22:31:15.873 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.882 7f7f6bfff700 1 --2- 172.21.15.102:0/3946281686 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f7f6c0d6620 0x7f7f6c1b3ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:15.873 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.882 7f7f7097b700 1 --2- 172.21.15.102:0/3946281686 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f7f6c0cded0 0x7f7f6c0d5ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:15.873 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.882 7f7f7117c700 1 --2- 172.21.15.102:0/3946281686 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7f6c0d6a70 0x7f7f6c1b4520 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 rx=0x7f7f64006fc0 tx=0x7f7f64006c40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:15.874 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.882 7f7f69ffb700 1 -- 172.21.15.102:0/3946281686 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f7f6c1b4b50 con 0x7f7f6c0d6a70 2024-04-17T22:31:15.874 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.882 7f7f69ffb700 1 -- 172.21.15.102:0/3946281686 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f64011a10 con 0x7f7f6c0d6a70 2024-04-17T22:31:15.874 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.882 7f7f73b80700 1 -- 172.21.15.102:0/3946281686 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f6c1b4e10 con 0x7f7f6c0d6a70 2024-04-17T22:31:15.874 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.882 7f7f69ffb700 1 -- 172.21.15.102:0/3946281686 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f7f64017040 con 0x7f7f6c0d6a70 2024-04-17T22:31:15.874 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.882 7f7f73b80700 1 -- 172.21.15.102:0/3946281686 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f6c1bb840 con 0x7f7f6c0d6a70 2024-04-17T22:31:15.874 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.883 7f7f69ffb700 1 -- 172.21.15.102:0/3946281686 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7f7f6401c470 con 0x7f7f6c0d6a70 2024-04-17T22:31:15.874 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.883 7f7f69ffb700 1 -- 172.21.15.102:0/3946281686 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3194+0+0 (secure 0 0 0) 0x7f7f64018030 con 0x7f7f6c0d6a70 2024-04-17T22:31:15.874 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.883 7f7f73b80700 1 -- 172.21.15.102:0/3946281686 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f6c0f0820 con 0x7f7f6c0d6a70 2024-04-17T22:31:15.874 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:15.887 7f7f69ffb700 1 -- 172.21.15.102:0/3946281686 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+110455 (secure 0 0 0) 0x7f7f6400b630 con 0x7f7f6c0d6a70 2024-04-17T22:31:16.007 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:16.021 7f7f73b80700 1 -- 172.21.15.102:0/3946281686 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7f6c0f5550 con 0x7f7f6c0d6a70 2024-04-17T22:31:16.007 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:16.021 7f7f69ffb700 1 -- 172.21.15.102:0/3946281686 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) v1 ==== 93+0+1704 (secure 0 0 0) 0x7f7f640356c0 con 0x7f7f6c0d6a70 2024-04-17T22:31:16.008 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-17T22:31:16.008 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":5,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[],"filesystems":[{"mdsmap":{"epoch":5,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17 22:29:07.787730","modified":"2024-04-17 22:29:09.795249","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"-1 (unspecified)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4177},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4177":{"gid":4177,"name":"a","rank":0,"incarnation":3,"state":"up:active","state_seq":4,"addr":"172.21.15.102:6817/1944542598","addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6817","nonce":1944542598}]},"export_targets":[],"features":4611087854035861503,"flags":0}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-04-17T22:31:16.009 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:16.023 7f7f73b80700 1 -- 172.21.15.102:0/3946281686 >> v1:172.21.15.102:6816/42360 conn(0x7f7f540145a0 legacy=0x7f7f54016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:16.009 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:16.023 7f7f73b80700 1 -- 172.21.15.102:0/3946281686 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7f6c0d6a70 msgr2=0x7f7f6c1b4520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:16.009 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:16.023 7f7f73b80700 1 --2- 172.21.15.102:0/3946281686 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7f6c0d6a70 0x7f7f6c1b4520 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 rx=0x7f7f64006fc0 tx=0x7f7f64006c40).stop 2024-04-17T22:31:16.009 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:16.023 7f7f73b80700 1 -- 172.21.15.102:0/3946281686 shutdown_connections 2024-04-17T22:31:16.009 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:16.023 7f7f73b80700 1 --2- 172.21.15.102:0/3946281686 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7f6c0d6a70 0x7f7f6c1b4520 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:16.009 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:16.023 7f7f73b80700 1 --2- 172.21.15.102:0/3946281686 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f7f6c0cded0 0x7f7f6c0d5ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:16.010 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:16.023 7f7f73b80700 1 --2- 172.21.15.102:0/3946281686 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f7f6c0d6620 0x7f7f6c1b3ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:16.010 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:16.023 7f7f73b80700 1 -- 172.21.15.102:0/3946281686 >> 172.21.15.102:0/3946281686 conn(0x7f7f6c0c3880 msgr2=0x7f7f6c0dae50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:16.010 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:16.023 7f7f73b80700 1 -- 172.21.15.102:0/3946281686 shutdown_connections 2024-04-17T22:31:16.010 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:16.023 7f7f73b80700 1 -- 172.21.15.102:0/3946281686 wait complete. 2024-04-17T22:31:16.010 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 5 2024-04-17T22:31:16.020 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T22:31:16.147 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:16.158 7fc28f8ce700 1 Processor -- start 2024-04-17T22:31:16.147 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:16.158 7fc28f8ce700 1 -- start start 2024-04-17T22:31:16.147 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:16.159 7fc28f8ce700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc28810e1d0 0x7fc28810e620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:16.147 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:16.159 7fc28f8ce700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fc2880f1ec0 0x7fc2881ab470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:16.147 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:16.159 7fc28f8ce700 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2880f2310 con 0x7fc28810ec50 2024-04-17T22:31:16.147 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:16.159 7fc28f8ce700 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2880f9ea0 con 0x7fc2880f1ec0 2024-04-17T22:31:16.147 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:16.159 7fc28f8ce700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2880f8fc0 con 0x7fc28810e1d0 2024-04-17T22:31:16.147 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:16.159 7fc28ceca700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fc2880f1ec0 0x7fc2881ab470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:16.147 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:16.159 7fc287fff700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc28810e1d0 0x7fc28810e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:16.147 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:16.159 7fc28ceca700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fc2880f1ec0 0x7fc2881ab470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:44114/0 (socket says 172.21.15.102:44114) 2024-04-17T22:31:16.148 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:16.159 7fc28ceca700 1 -- 172.21.15.102:0/3731310280 learned_addr learned my addr 172.21.15.102:0/3731310280 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:31:16.148 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:16.159 7fc287fff700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc28810e1d0 0x7fc28810e620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:58838/0 (socket says 172.21.15.102:58838) 2024-04-17T22:31:16.148 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:16.159 7fc27ffff700 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7fc28810ec50 0x7fc2880f16c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:36390/0 (socket says 172.21.15.102:36390) 2024-04-17T22:31:16.148 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:16.159 7fc28ceca700 1 -- 172.21.15.102:0/3731310280 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc28810e1d0 msgr2=0x7fc28810e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:31:16.148 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:16.159 7fc28ceca700 1 --2- 172.21.15.102:0/3731310280 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc28810e1d0 0x7fc28810e620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:16.148 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:16.159 7fc28ceca700 1 -- 172.21.15.102:0/3731310280 >> v1:172.21.15.102:6789/0 conn(0x7fc28810ec50 legacy=0x7fc2880f16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:16.148 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:16.159 7fc28ceca700 1 -- 172.21.15.102:0/3731310280 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2880f2530 con 0x7fc2880f1ec0 2024-04-17T22:31:16.148 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:16.159 7fc2877fe700 1 -- 172.21.15.102:0/3731310280 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2122896173 0 0) 0x7fc270001110 con 0x7fc28810ec50 2024-04-17T22:31:16.148 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:16.159 7fc287fff700 1 --2- 172.21.15.102:0/3731310280 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc28810e1d0 0x7fc28810e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:16.148 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:16.159 7fc28ceca700 1 --2- 172.21.15.102:0/3731310280 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc2880f1ec0 0x7fc2881ab470 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 rx=0x7fc288103510 tx=0x7fc280009e10).ready entity=mon.0 client_cookie=dc41097172a541b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:16.148 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:16.159 7fc2877fe700 1 -- 172.21.15.102:0/3731310280 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc2880f2530 con 0x7fc2880f1ec0 2024-04-17T22:31:16.148 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:16.159 7fc2877fe700 1 -- 172.21.15.102:0/3731310280 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc28000b890 con 0x7fc2880f1ec0 2024-04-17T22:31:16.148 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:16.159 7fc2877fe700 1 -- 172.21.15.102:0/3731310280 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc28000ba70 con 0x7fc2880f1ec0 2024-04-17T22:31:16.148 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:16.160 7fc28f8ce700 1 -- 172.21.15.102:0/3731310280 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc2880f1ec0 msgr2=0x7fc2881ab470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:16.149 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:16.160 7fc28f8ce700 1 --2- 172.21.15.102:0/3731310280 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc2880f1ec0 0x7fc2881ab470 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 rx=0x7fc288103510 tx=0x7fc280009e10).stop 2024-04-17T22:31:16.149 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:16.160 7fc28f8ce700 1 -- 172.21.15.102:0/3731310280 shutdown_connections 2024-04-17T22:31:16.149 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:16.160 7fc28f8ce700 1 --2- 172.21.15.102:0/3731310280 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc28810e1d0 0x7fc28810e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:16.149 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:16.160 7fc28f8ce700 1 --2- 172.21.15.102:0/3731310280 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc2880f1ec0 0x7fc2881ab470 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:16.149 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:16.160 7fc28f8ce700 1 -- 172.21.15.102:0/3731310280 >> 172.21.15.102:0/3731310280 conn(0x7fc288113830 msgr2=0x7fc2880f4ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:16.149 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:16.160 7fc28f8ce700 1 -- 172.21.15.102:0/3731310280 shutdown_connections 2024-04-17T22:31:16.149 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:16.160 7fc28f8ce700 1 -- 172.21.15.102:0/3731310280 wait complete. 2024-04-17T22:31:16.169 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:16.184 7fc28f8ce700 1 Processor -- start 2024-04-17T22:31:16.169 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:16.184 7fc28f8ce700 1 -- start start 2024-04-17T22:31:16.171 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:16.184 7fc28f8ce700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc2881ad970 0x7fc2881addc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:16.171 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:16.184 7fc28f8ce700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fc2881ae3f0 0x7fc2880dead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:16.172 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:16.184 7fc28f8ce700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc2880df100 0x7fc2880df550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:16.172 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:16.184 7fc28f8ce700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2880fa140 con 0x7fc2881ad970 2024-04-17T22:31:16.172 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:16.184 7fc28f8ce700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2880c49d0 con 0x7fc2880df100 2024-04-17T22:31:16.172 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:16.184 7fc28f8ce700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc2880dfe70 con 0x7fc2881ae3f0 2024-04-17T22:31:16.172 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:16.184 7fc287fff700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc2881ad970 0x7fc2881addc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:16.172 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:16.184 7fc287fff700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc2881ad970 0x7fc2881addc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:44118/0 (socket says 172.21.15.102:44118) 2024-04-17T22:31:16.172 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:16.184 7fc287fff700 1 -- 172.21.15.102:0/37325636 learned_addr learned my addr 172.21.15.102:0/37325636 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:31:16.172 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:16.184 7fc27ffff700 1 --2- 172.21.15.102:0/37325636 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fc2881ae3f0 0x7fc2880dead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:16.172 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:16.184 7fc28ceca700 1 --2- 172.21.15.102:0/37325636 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc2880df100 0x7fc2880df550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:16.172 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:16.184 7fc287fff700 1 -- 172.21.15.102:0/37325636 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fc2881ae3f0 msgr2=0x7fc2880dead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:16.172 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:16.184 7fc287fff700 1 --2- 172.21.15.102:0/37325636 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fc2881ae3f0 0x7fc2880dead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:16.172 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:16.184 7fc287fff700 1 -- 172.21.15.102:0/37325636 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc2880df100 msgr2=0x7fc2880df550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:16.172 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:16.184 7fc287fff700 1 --2- 172.21.15.102:0/37325636 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc2880df100 0x7fc2880df550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:16.172 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:16.184 7fc287fff700 1 -- 172.21.15.102:0/37325636 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2880c6c90 con 0x7fc2881ad970 2024-04-17T22:31:16.173 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:16.184 7fc287fff700 1 --2- 172.21.15.102:0/37325636 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc2881ad970 0x7fc2881addc0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 rx=0x7fc2880c52a0 tx=0x7fc27800bf70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:16.173 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:16.184 7fc285ffb700 1 -- 172.21.15.102:0/37325636 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc2880c6c90 con 0x7fc2881ad970 2024-04-17T22:31:16.173 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:16.184 7fc285ffb700 1 -- 172.21.15.102:0/37325636 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc27800c9a0 con 0x7fc2881ad970 2024-04-17T22:31:16.173 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:16.184 7fc285ffb700 1 -- 172.21.15.102:0/37325636 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc27800f040 con 0x7fc2881ad970 2024-04-17T22:31:16.173 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:16.184 7fc28f8ce700 1 -- 172.21.15.102:0/37325636 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2881bbb80 con 0x7fc2881ad970 2024-04-17T22:31:16.173 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:16.184 7fc28f8ce700 1 -- 172.21.15.102:0/37325636 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2881bbf60 con 0x7fc2881ad970 2024-04-17T22:31:16.173 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:16.184 7fc285ffb700 1 -- 172.21.15.102:0/37325636 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7fc278011070 con 0x7fc2881ad970 2024-04-17T22:31:16.173 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:16.185 7fc285ffb700 1 -- 172.21.15.102:0/37325636 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3194+0+0 (secure 0 0 0) 0x7fc278010040 con 0x7fc2881ad970 2024-04-17T22:31:16.173 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:16.185 7fc28f8ce700 1 -- 172.21.15.102:0/37325636 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2880f0790 con 0x7fc2881ad970 2024-04-17T22:31:16.174 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:16.188 7fc285ffb700 1 -- 172.21.15.102:0/37325636 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+110455 (secure 0 0 0) 0x7fc27800c5a0 con 0x7fc2881ad970 2024-04-17T22:31:16.279 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:16.293 7fc28f8ce700 1 -- 172.21.15.102:0/37325636 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc2880f54c0 con 0x7fc2881ad970 2024-04-17T22:31:16.280 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:16.294 7fc285ffb700 1 -- 172.21.15.102:0/37325636 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) v1 ==== 74+0+8764 (secure 0 0 0) 0x7fc278000c30 con 0x7fc2881ad970 2024-04-17T22:31:16.280 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-17T22:31:16.280 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":17,"fsid":"cb63e789-091d-49b1-b120-55f9db526871","created":"2024-04-17 22:28:48.209635","modified":"2024-04-17 22:29:23.257387","last_up_change":"2024-04-17 22:28:54.236215","last_in_change":"2024-04-17 22:28:54.236215","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":4,"require_min_compat_client":"nautilus","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-04-17 22:28:57.478545","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-17 22:29:06.144708","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-17 22:29:06.748924","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3351ef85-ade4-4b90-a8f4-213f4e00ba01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6812","nonce":42955}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6813","nonce":42955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6815","nonce":42955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6814","nonce":42955}]},"public_addr":"172.21.15.102:6812/42955","cluster_addr":"172.21.15.102:6813/42955","heartbeat_back_addr":"172.21.15.102:6815/42955","heartbeat_front_addr":"172.21.15.102:6814/42955","state":["exists","up"]},{"osd":1,"uuid":"04f67eac-f36b-48f5-982d-4cfc776596d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6808","nonce":42958}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6809","nonce":42958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6811","nonce":42958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6810","nonce":42958}]},"public_addr":"172.21.15.102:6808/42958","cluster_addr":"172.21.15.102:6809/42958","heartbeat_back_addr":"172.21.15.102:6811/42958","heartbeat_front_addr":"172.21.15.102:6810/42958","state":["exists","up"]},{"osd":2,"uuid":"d524204b-70b2-40a8-8f73-f9e7d44318af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6804","nonce":42954}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6805","nonce":42954}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6807","nonce":42954}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6806","nonce":42954}]},"public_addr":"172.21.15.102:6804/42954","cluster_addr":"172.21.15.102:6805/42954","heartbeat_back_addr":"172.21.15.102:6807/42954","heartbeat_front_addr":"172.21.15.102:6806/42954","state":["exists","up"]},{"osd":3,"uuid":"4fc9d20c-1ad0-484f-aafa-2c9a7574245b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6800","nonce":42953}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6801","nonce":42953}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6803","nonce":42953}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6802","nonce":42953}]},"public_addr":"172.21.15.102:6800/42953","cluster_addr":"172.21.15.102:6801/42953","heartbeat_back_addr":"172.21.15.102:6803/42953","heartbeat_front_addr":"172.21.15.102:6802/42953","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2024-04-17T22:31:16.281 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:16.295 7fc28f8ce700 1 -- 172.21.15.102:0/37325636 >> v1:172.21.15.102:6816/42360 conn(0x7fc26c0145a0 legacy=0x7fc26c016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:16.281 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:16.295 7fc28f8ce700 1 -- 172.21.15.102:0/37325636 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc2881ad970 msgr2=0x7fc2881addc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:16.281 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:16.295 7fc28f8ce700 1 --2- 172.21.15.102:0/37325636 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc2881ad970 0x7fc2881addc0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 rx=0x7fc2880c52a0 tx=0x7fc27800bf70).stop 2024-04-17T22:31:16.281 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:16.295 7fc28f8ce700 1 -- 172.21.15.102:0/37325636 shutdown_connections 2024-04-17T22:31:16.281 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:16.295 7fc28f8ce700 1 --2- 172.21.15.102:0/37325636 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc2880df100 0x7fc2880df550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:16.281 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:16.295 7fc28f8ce700 1 --2- 172.21.15.102:0/37325636 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc2881ad970 0x7fc2881addc0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:16.281 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:16.295 7fc28f8ce700 1 --2- 172.21.15.102:0/37325636 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fc2881ae3f0 0x7fc2880dead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:16.281 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:16.295 7fc28f8ce700 1 -- 172.21.15.102:0/37325636 >> 172.21.15.102:0/37325636 conn(0x7fc2880c58f0 msgr2=0x7fc2880c75b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:16.282 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:16.295 7fc28f8ce700 1 -- 172.21.15.102:0/37325636 shutdown_connections 2024-04-17T22:31:16.282 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:16.295 7fc28f8ce700 1 -- 172.21.15.102:0/37325636 wait complete. 2024-04-17T22:31:16.292 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-17T22:31:16.420 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:16.432 7f74cb7d8700 1 Processor -- start 2024-04-17T22:31:16.420 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:16.432 7f74cb7d8700 1 -- start start 2024-04-17T22:31:16.420 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:16.432 7f74cb7d8700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f74c410e240 0x7f74c410e690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:16.420 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:16.432 7f74cb7d8700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f74c410ecc0 0x7f74c40f1730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:16.421 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:16.432 7f74cb7d8700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f74c40f1d60 0x7f74c41aa4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:16.421 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:16.432 7f74cb7d8700 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74c40f9f10 con 0x7f74c40f1d60 2024-04-17T22:31:16.421 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:16.432 7f74cb7d8700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74c40f9030 con 0x7f74c410e240 2024-04-17T22:31:16.421 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:16.432 7f74cb7d8700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f74c40fa1b0 con 0x7f74c410ecc0 2024-04-17T22:31:16.421 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:16.432 7f74c3fff700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f74c410e240 0x7f74c410e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:16.421 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:16.432 7f74bbfff700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f74c410ecc0 0x7f74c40f1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:16.421 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:16.432 7f74c3fff700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f74c410e240 0x7f74c410e690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:58862/0 (socket says 172.21.15.102:58862) 2024-04-17T22:31:16.421 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:16.432 7f74bbfff700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f74c410ecc0 0x7f74c40f1730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:48940/0 (socket says 172.21.15.102:48940) 2024-04-17T22:31:16.421 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:16.432 7f74c3fff700 1 -- 172.21.15.102:0/4163670603 learned_addr learned my addr 172.21.15.102:0/4163670603 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:31:16.421 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:16.432 7f74c8dd4700 1 --2- 172.21.15.102:0/4163670603 >> v2:172.21.15.102:3300/0 conn(0x7f74c40f1d60 0x7f74c41aa4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:16.421 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:16.433 7f74bbfff700 1 -- 172.21.15.102:0/4163670603 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f74c410e240 msgr2=0x7f74c410e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:31:16.421 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:16.433 7f74bbfff700 1 --2- 172.21.15.102:0/4163670603 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f74c410e240 0x7f74c410e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:16.421 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:16.433 7f74bbfff700 1 -- 172.21.15.102:0/4163670603 >> v2:172.21.15.102:3300/0 conn(0x7f74c40f1d60 msgr2=0x7f74c41aa4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:31:16.422 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:16.433 7f74bbfff700 1 --2- 172.21.15.102:0/4163670603 >> v2:172.21.15.102:3300/0 conn(0x7f74c40f1d60 0x7f74c41aa4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:16.422 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:16.433 7f74bbfff700 1 -- 172.21.15.102:0/4163670603 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74c40f21b0 con 0x7f74c410ecc0 2024-04-17T22:31:16.422 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:16.433 7f74c3fff700 1 --2- 172.21.15.102:0/4163670603 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f74c410e240 0x7f74c410e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:16.422 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:16.433 7f74c8dd4700 1 --2- 172.21.15.102:0/4163670603 >> v2:172.21.15.102:3300/0 conn(0x7f74c40f1d60 0x7f74c41aa4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:16.422 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:16.433 7f74bbfff700 1 --2- 172.21.15.102:0/4163670603 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f74c410ecc0 0x7f74c40f1730 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 rx=0x7f74c4103580 tx=0x7f74ac00b9a0).ready entity=mon.2 client_cookie=56844e7db66a89c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:16.422 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:16.433 7f74c37fe700 1 -- 172.21.15.102:0/4163670603 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f74c40f21b0 con 0x7f74c410ecc0 2024-04-17T22:31:16.422 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:16.433 7f74c37fe700 1 -- 172.21.15.102:0/4163670603 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74ac00d570 con 0x7f74c410ecc0 2024-04-17T22:31:16.422 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:16.433 7f74c37fe700 1 -- 172.21.15.102:0/4163670603 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f74ac00d940 con 0x7f74c410ecc0 2024-04-17T22:31:16.422 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:16.433 7f74cb7d8700 1 -- 172.21.15.102:0/4163670603 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f74c410ecc0 msgr2=0x7f74c40f1730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:16.422 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:16.433 7f74cb7d8700 1 --2- 172.21.15.102:0/4163670603 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f74c410ecc0 0x7f74c40f1730 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 rx=0x7f74c4103580 tx=0x7f74ac00b9a0).stop 2024-04-17T22:31:16.422 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:16.434 7f74cb7d8700 1 -- 172.21.15.102:0/4163670603 shutdown_connections 2024-04-17T22:31:16.422 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:16.434 7f74cb7d8700 1 --2- 172.21.15.102:0/4163670603 >> v2:172.21.15.102:3300/0 conn(0x7f74c40f1d60 0x7f74c41aa4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:16.422 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:16.434 7f74cb7d8700 1 --2- 172.21.15.102:0/4163670603 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f74c410e240 0x7f74c410e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:16.422 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:16.434 7f74cb7d8700 1 --2- 172.21.15.102:0/4163670603 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f74c410ecc0 0x7f74c40f1730 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:16.423 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:16.434 7f74cb7d8700 1 -- 172.21.15.102:0/4163670603 >> 172.21.15.102:0/4163670603 conn(0x7f74c41138a0 msgr2=0x7f74c40f4f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:16.423 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:16.434 7f74cb7d8700 1 -- 172.21.15.102:0/4163670603 shutdown_connections 2024-04-17T22:31:16.423 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:16.434 7f74cb7d8700 1 -- 172.21.15.102:0/4163670603 wait complete. 2024-04-17T22:31:16.444 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:16.456 7f74cb7d8700 1 Processor -- start 2024-04-17T22:31:16.444 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:16.456 7f74cb7d8700 1 -- start start 2024-04-17T22:31:16.444 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:16.457 7f74cb7d8700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f74c40cd800 0x7f74c40d5e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:16.444 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:16.457 7f74cb7d8700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f74c40d64a0 0x7f74c41b3d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:16.444 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:16.457 7f74cb7d8700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f74c40d68f0 0x7f74c41b4390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:16.445 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:16.457 7f74cb7d8700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74c40a8660 con 0x7f74c40d68f0 2024-04-17T22:31:16.445 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:16.457 7f74cb7d8700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74c40cd1a0 con 0x7f74c40d64a0 2024-04-17T22:31:16.445 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:16.457 7f74cb7d8700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f74c40cdc50 con 0x7f74c40cd800 2024-04-17T22:31:16.445 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:16.457 7f74bbfff700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f74c40d64a0 0x7f74c41b3d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:16.445 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:16.457 7f74c3fff700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f74c40cd800 0x7f74c40d5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:16.445 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:16.457 7f74c3fff700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f74c40cd800 0x7f74c40d5e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:48948/0 (socket says 172.21.15.102:48948) 2024-04-17T22:31:16.445 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:16.457 7f74c3fff700 1 -- 172.21.15.102:0/2586785848 learned_addr learned my addr 172.21.15.102:0/2586785848 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:31:16.445 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:16.457 7f74c8dd4700 1 --2- 172.21.15.102:0/2586785848 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f74c40d68f0 0x7f74c41b4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:16.445 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:16.457 7f74bbfff700 1 -- 172.21.15.102:0/2586785848 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f74c40cd800 msgr2=0x7f74c40d5e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:16.445 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:16.457 7f74bbfff700 1 --2- 172.21.15.102:0/2586785848 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f74c40cd800 0x7f74c40d5e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:16.445 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:16.457 7f74bbfff700 1 -- 172.21.15.102:0/2586785848 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f74c40d68f0 msgr2=0x7f74c41b4390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:16.445 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:16.457 7f74bbfff700 1 --2- 172.21.15.102:0/2586785848 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f74c40d68f0 0x7f74c41b4390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:16.445 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:16.457 7f74bbfff700 1 -- 172.21.15.102:0/2586785848 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74c41b49c0 con 0x7f74c40d64a0 2024-04-17T22:31:16.445 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:16.457 7f74c3fff700 1 --2- 172.21.15.102:0/2586785848 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f74c40cd800 0x7f74c40d5e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:16.446 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:16.457 7f74c8dd4700 1 --2- 172.21.15.102:0/2586785848 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f74c40d68f0 0x7f74c41b4390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:16.446 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:16.457 7f74bbfff700 1 --2- 172.21.15.102:0/2586785848 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f74c40d64a0 0x7f74c41b3d60 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 rx=0x7f74c40c5490 tx=0x7f74ac0068c0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:16.446 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:16.457 7f74c1ffb700 1 -- 172.21.15.102:0/2586785848 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f74c41b49c0 con 0x7f74c40d64a0 2024-04-17T22:31:16.446 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:16.457 7f74c1ffb700 1 -- 172.21.15.102:0/2586785848 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74ac0119c0 con 0x7f74c40d64a0 2024-04-17T22:31:16.446 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:16.457 7f74cb7d8700 1 -- 172.21.15.102:0/2586785848 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74c41b4c80 con 0x7f74c40d64a0 2024-04-17T22:31:16.446 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:16.457 7f74c1ffb700 1 -- 172.21.15.102:0/2586785848 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f74ac017040 con 0x7f74c40d64a0 2024-04-17T22:31:16.446 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:16.457 7f74cb7d8700 1 -- 172.21.15.102:0/2586785848 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74c41bb6b0 con 0x7f74c40d64a0 2024-04-17T22:31:16.446 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:16.458 7f74c1ffb700 1 -- 172.21.15.102:0/2586785848 <== mon.1 v2:172.21.15.102:3301/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7f74ac0273b0 con 0x7f74c40d64a0 2024-04-17T22:31:16.446 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:16.458 7f74c1ffb700 1 -- 172.21.15.102:0/2586785848 <== mon.1 v2:172.21.15.102:3301/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3194+0+0 (secure 0 0 0) 0x7f74ac018070 con 0x7f74c40d64a0 2024-04-17T22:31:16.446 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:16.458 7f74cb7d8700 1 -- 172.21.15.102:0/2586785848 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74c40f0800 con 0x7f74c40d64a0 2024-04-17T22:31:16.447 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:16.462 7f74c1ffb700 1 -- 172.21.15.102:0/2586785848 <== mon.1 v2:172.21.15.102:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+110455 (secure 0 0 0) 0x7f74ac0117b0 con 0x7f74c40d64a0 2024-04-17T22:31:16.586 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:16.600 7f74cb7d8700 1 -- 172.21.15.102:0/2586785848 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f74c41ae4c0 con 0x7f74c40d64a0 2024-04-17T22:31:17.410 INFO:tasks.ceph.mon.a.smithi102.stderr:2024-04-17 22:31:17.425 7ff15d99f700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-17T22:31:17.415 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:17.430 7f74c1ffb700 1 -- 172.21.15.102:0/2586785848 <== mon.1 v2:172.21.15.102:3301/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v6) v1 ==== 161+0+0 (secure 0 0 0) 0x7f74ac00d360 con 0x7f74c40d64a0 2024-04-17T22:31:17.417 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:17.432 7f74cb7d8700 1 -- 172.21.15.102:0/2586785848 >> v1:172.21.15.102:6816/42360 conn(0x7f74a80145a0 legacy=0x7f74a8016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:17.418 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:17.432 7f74cb7d8700 1 -- 172.21.15.102:0/2586785848 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f74c40d64a0 msgr2=0x7f74c41b3d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:17.418 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:17.432 7f74cb7d8700 1 --2- 172.21.15.102:0/2586785848 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f74c40d64a0 0x7f74c41b3d60 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 rx=0x7f74c40c5490 tx=0x7f74ac0068c0).stop 2024-04-17T22:31:17.418 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:17.432 7f74cb7d8700 1 -- 172.21.15.102:0/2586785848 shutdown_connections 2024-04-17T22:31:17.418 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:17.432 7f74cb7d8700 1 --2- 172.21.15.102:0/2586785848 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f74c40d68f0 0x7f74c41b4390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:17.418 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:17.432 7f74cb7d8700 1 --2- 172.21.15.102:0/2586785848 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f74c40cd800 0x7f74c40d5e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:17.418 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:17.432 7f74cb7d8700 1 --2- 172.21.15.102:0/2586785848 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f74c40d64a0 0x7f74c41b3d60 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:17.418 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:17.432 7f74cb7d8700 1 -- 172.21.15.102:0/2586785848 >> 172.21.15.102:0/2586785848 conn(0x7f74c40c3750 msgr2=0x7f74c40dacf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:17.418 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:17.432 7f74cb7d8700 1 -- 172.21.15.102:0/2586785848 shutdown_connections 2024-04-17T22:31:17.418 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:17.432 7f74cb7d8700 1 -- 172.21.15.102:0/2586785848 wait complete. 2024-04-17T22:31:17.418 INFO:teuthology.orchestra.run.smithi102.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-17T22:31:17.431 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-17T22:31:17.436 INFO:tasks.ceph.mds.a.smithi102.stderr:warning: line 130: 'mon_down_mkfs_grace' in section 'mon' redefined 2024-04-17T22:31:17.487 INFO:tasks.ceph.mds.a.smithi102.stdout:starting mds.a at 2024-04-17T22:31:17.567 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:17.580 7f246335b700 1 Processor -- start 2024-04-17T22:31:17.567 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:17.580 7f246335b700 1 -- start start 2024-04-17T22:31:17.567 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:17.580 7f246335b700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f245c19fde0 0x7f245c1a0230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:17.567 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:17.580 7f246335b700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f245c1a0860 0x7f245c1030e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:17.567 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:17.580 7f246335b700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f245c103710 0x7f245c103b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:17.568 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:17.580 7f246335b700 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f245c1043d0 con 0x7f245c19fde0 2024-04-17T22:31:17.568 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:17.580 7f246335b700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f245c0f9c20 con 0x7f245c1a0860 2024-04-17T22:31:17.568 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:17.580 7f246335b700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f245c0f9870 con 0x7f245c103710 2024-04-17T22:31:17.568 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:17.580 7f2460957700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f245c103710 0x7f245c103b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:17.568 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:17.580 7f245bfff700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f245c19fde0 0x7f245c1a0230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:17.568 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:17.580 7f2453fff700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f245c1a0860 0x7f245c1030e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:17.568 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:17.580 7f2460957700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f245c103710 0x7f245c103b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:48952/0 (socket says 172.21.15.102:48952) 2024-04-17T22:31:17.568 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:17.580 7f245bfff700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f245c19fde0 0x7f245c1a0230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:44156/0 (socket says 172.21.15.102:44156) 2024-04-17T22:31:17.568 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:17.581 7f2460957700 1 -- 172.21.15.102:0/3042374724 learned_addr learned my addr 172.21.15.102:0/3042374724 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:31:17.568 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:17.581 7f245bfff700 1 -- 172.21.15.102:0/3042374724 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f245c103710 msgr2=0x7f245c103b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:31:17.568 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:17.581 7f245bfff700 1 --2- 172.21.15.102:0/3042374724 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f245c103710 0x7f245c103b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:17.568 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:17.581 7f245bfff700 1 -- 172.21.15.102:0/3042374724 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f245c1a0860 msgr2=0x7f245c1030e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:31:17.568 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:17.581 7f245bfff700 1 --2- 172.21.15.102:0/3042374724 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f245c1a0860 0x7f245c1030e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:17.569 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:17.581 7f245bfff700 1 -- 172.21.15.102:0/3042374724 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f245c1aa640 con 0x7f245c19fde0 2024-04-17T22:31:17.569 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:17.581 7f2460957700 1 --2- 172.21.15.102:0/3042374724 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f245c103710 0x7f245c103b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:17.569 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:17.581 7f2453fff700 1 --2- 172.21.15.102:0/3042374724 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f245c1a0860 0x7f245c1030e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:17.569 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:17.581 7f245bfff700 1 --2- 172.21.15.102:0/3042374724 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f245c19fde0 0x7f245c1a0230 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 rx=0x7f244c002f70 tx=0x7f244c00b9f0).ready entity=mon.0 client_cookie=889797318e326076 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:17.569 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:17.581 7f245b7fe700 1 -- 172.21.15.102:0/3042374724 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f245c1aa640 con 0x7f245c19fde0 2024-04-17T22:31:17.569 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:17.581 7f245b7fe700 1 -- 172.21.15.102:0/3042374724 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f244c00d570 con 0x7f245c19fde0 2024-04-17T22:31:17.569 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:17.581 7f245b7fe700 1 -- 172.21.15.102:0/3042374724 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f244c00d940 con 0x7f245c19fde0 2024-04-17T22:31:17.569 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:17.581 7f246335b700 1 -- 172.21.15.102:0/3042374724 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f245c19fde0 msgr2=0x7f245c1a0230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:17.569 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:17.581 7f246335b700 1 --2- 172.21.15.102:0/3042374724 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f245c19fde0 0x7f245c1a0230 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 rx=0x7f244c002f70 tx=0x7f244c00b9f0).stop 2024-04-17T22:31:17.569 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:17.582 7f246335b700 1 -- 172.21.15.102:0/3042374724 shutdown_connections 2024-04-17T22:31:17.569 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:17.582 7f246335b700 1 --2- 172.21.15.102:0/3042374724 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f245c103710 0x7f245c103b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:17.569 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:17.582 7f246335b700 1 --2- 172.21.15.102:0/3042374724 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f245c19fde0 0x7f245c1a0230 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:17.569 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:17.582 7f246335b700 1 --2- 172.21.15.102:0/3042374724 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f245c1a0860 0x7f245c1030e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:17.569 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:17.582 7f246335b700 1 -- 172.21.15.102:0/3042374724 >> 172.21.15.102:0/3042374724 conn(0x7f245c113000 msgr2=0x7f245c113450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:17.570 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:17.582 7f246335b700 1 -- 172.21.15.102:0/3042374724 shutdown_connections 2024-04-17T22:31:17.570 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:17.582 7f246335b700 1 -- 172.21.15.102:0/3042374724 wait complete. 2024-04-17T22:31:17.592 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:17.604 7f246335b700 1 Processor -- start 2024-04-17T22:31:17.592 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:17.604 7f246335b700 1 -- start start 2024-04-17T22:31:17.592 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:17.605 7f246335b700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f245c0d5300 0x7f245c0d5750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:17.592 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:17.605 7f246335b700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f245c0d5d80 0x7f245c1b3f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:17.592 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:17.605 7f246335b700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f245c1b4590 0x7f245c1b49e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:17.592 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:17.605 7f246335b700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f245c1b5010 con 0x7f245c0d5300 2024-04-17T22:31:17.592 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:17.605 7f246335b700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f245c0cc140 con 0x7f245c0d5d80 2024-04-17T22:31:17.592 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:17.605 7f246335b700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f245c0da810 con 0x7f245c1b4590 2024-04-17T22:31:17.592 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:17.605 7f245bfff700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f245c0d5300 0x7f245c0d5750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:17.592 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:17.605 7f2453fff700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f245c0d5d80 0x7f245c1b3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:17.592 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:17.605 7f245bfff700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f245c0d5300 0x7f245c0d5750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:44164/0 (socket says 172.21.15.102:44164) 2024-04-17T22:31:17.592 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:17.605 7f245bfff700 1 -- 172.21.15.102:0/1604217233 learned_addr learned my addr 172.21.15.102:0/1604217233 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:31:17.593 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:17.605 7f2460957700 1 --2- 172.21.15.102:0/1604217233 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f245c1b4590 0x7f245c1b49e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:17.593 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:17.605 7f245bfff700 1 -- 172.21.15.102:0/1604217233 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f245c1b4590 msgr2=0x7f245c1b49e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:17.593 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:17.605 7f245bfff700 1 --2- 172.21.15.102:0/1604217233 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f245c1b4590 0x7f245c1b49e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:17.593 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:17.605 7f245bfff700 1 -- 172.21.15.102:0/1604217233 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f245c0d5d80 msgr2=0x7f245c1b3f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:17.593 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:17.605 7f245bfff700 1 --2- 172.21.15.102:0/1604217233 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f245c0d5d80 0x7f245c1b3f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:17.593 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:17.605 7f245bfff700 1 -- 172.21.15.102:0/1604217233 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f245c1bb6d0 con 0x7f245c0d5300 2024-04-17T22:31:17.593 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:17.605 7f2453fff700 1 --2- 172.21.15.102:0/1604217233 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f245c0d5d80 0x7f245c1b3f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:17.593 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:17.605 7f2460957700 1 --2- 172.21.15.102:0/1604217233 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f245c1b4590 0x7f245c1b49e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:17.593 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:17.605 7f245bfff700 1 --2- 172.21.15.102:0/1604217233 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f245c0d5300 0x7f245c0d5750 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 rx=0x7f244c002f70 tx=0x7f244c0068c0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:17.593 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:17.606 7f2459ffb700 1 -- 172.21.15.102:0/1604217233 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f245c1bb6d0 con 0x7f245c0d5300 2024-04-17T22:31:17.593 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:17.606 7f2459ffb700 1 -- 172.21.15.102:0/1604217233 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f244c006ab0 con 0x7f245c0d5300 2024-04-17T22:31:17.593 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:17.606 7f246335b700 1 -- 172.21.15.102:0/1604217233 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f245c0fe970 con 0x7f245c0d5300 2024-04-17T22:31:17.593 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:17.606 7f2459ffb700 1 -- 172.21.15.102:0/1604217233 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f244c017040 con 0x7f245c0d5300 2024-04-17T22:31:17.593 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:17.606 7f246335b700 1 -- 172.21.15.102:0/1604217233 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f245c0fee10 con 0x7f245c0d5300 2024-04-17T22:31:17.594 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:17.606 7f2459ffb700 1 -- 172.21.15.102:0/1604217233 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7f244c011be0 con 0x7f245c0d5300 2024-04-17T22:31:17.594 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:17.606 7f2459ffb700 1 -- 172.21.15.102:0/1604217233 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 3237+0+0 (secure 0 0 0) 0x7f244c018070 con 0x7f245c0d5300 2024-04-17T22:31:17.594 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:17.607 7f246335b700 1 -- 172.21.15.102:0/1604217233 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f245c0f09f0 con 0x7f245c0d5300 2024-04-17T22:31:17.595 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:17.610 7f2459ffb700 1 -- 172.21.15.102:0/1604217233 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+110455 (secure 0 0 0) 0x7f244c0117e0 con 0x7f245c0d5300 2024-04-17T22:31:17.730 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:17.745 7f246335b700 1 -- 172.21.15.102:0/1604217233 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f245c1ae6a0 con 0x7f245c0d5300 2024-04-17T22:31:18.595 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:18.609 7f2459ffb700 1 -- 172.21.15.102:0/1604217233 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v7) v1 ==== 104+0+0 (secure 0 0 0) 0x7f244c00d360 con 0x7f245c0d5300 2024-04-17T22:31:18.597 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:18.611 7f246335b700 1 -- 172.21.15.102:0/1604217233 >> v1:172.21.15.102:6816/42360 conn(0x7f24400145a0 legacy=0x7f2440016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:18.597 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:18.611 7f246335b700 1 -- 172.21.15.102:0/1604217233 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f245c0d5300 msgr2=0x7f245c0d5750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:18.597 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:18.611 7f246335b700 1 --2- 172.21.15.102:0/1604217233 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f245c0d5300 0x7f245c0d5750 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 rx=0x7f244c002f70 tx=0x7f244c0068c0).stop 2024-04-17T22:31:18.597 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:18.611 7f246335b700 1 -- 172.21.15.102:0/1604217233 shutdown_connections 2024-04-17T22:31:18.597 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:18.611 7f246335b700 1 --2- 172.21.15.102:0/1604217233 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f245c1b4590 0x7f245c1b49e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:18.597 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:18.611 7f246335b700 1 --2- 172.21.15.102:0/1604217233 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f245c0d5300 0x7f245c0d5750 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:18.597 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:18.611 7f246335b700 1 --2- 172.21.15.102:0/1604217233 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f245c0d5d80 0x7f245c1b3f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:18.597 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:18.611 7f246335b700 1 -- 172.21.15.102:0/1604217233 >> 172.21.15.102:0/1604217233 conn(0x7f245c0c6c20 msgr2=0x7f245c0d63e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:18.597 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:18.612 7f246335b700 1 -- 172.21.15.102:0/1604217233 shutdown_connections 2024-04-17T22:31:18.597 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:18.612 7f246335b700 1 -- 172.21.15.102:0/1604217233 wait complete. 2024-04-17T22:31:18.610 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-17T22:31:18.788 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:18.800 7f734f165700 1 Processor -- start 2024-04-17T22:31:18.788 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:18.801 7f734f165700 1 -- start start 2024-04-17T22:31:18.788 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:18.801 7f734f165700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f734810eb50 0x7f73481a7ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:18.788 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:18.801 7f734f165700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f73481a8500 0x7f73481aa960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:18.788 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:18.801 7f734f165700 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7348103180 con 0x7f73480f56f0 2024-04-17T22:31:18.788 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:18.801 7f734f165700 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7348110960 con 0x7f73481a8500 2024-04-17T22:31:18.788 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:18.801 7f734f165700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f73481abfa0 con 0x7f734810eb50 2024-04-17T22:31:18.788 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:18.801 7f7347fff700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f73481a8500 0x7f73481aa960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:18.789 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:18.801 7f7346ffd700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f734810eb50 0x7f73481a7ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:18.789 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:18.801 7f7347fff700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f73481a8500 0x7f73481aa960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:44172/0 (socket says 172.21.15.102:44172) 2024-04-17T22:31:18.789 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:18.801 7f73477fe700 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f73480f56f0 0x7f734810e350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:36418/0 (socket says 172.21.15.102:36418) 2024-04-17T22:31:18.789 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:18.801 7f7347fff700 1 -- 172.21.15.102:0/2989273242 learned_addr learned my addr 172.21.15.102:0/2989273242 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:31:18.789 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:18.801 7f7347fff700 1 -- 172.21.15.102:0/2989273242 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f734810eb50 msgr2=0x7f73481a7ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:31:18.789 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:18.801 7f7347fff700 1 --2- 172.21.15.102:0/2989273242 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f734810eb50 0x7f73481a7ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:18.789 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:18.801 7f7347fff700 1 -- 172.21.15.102:0/2989273242 >> v1:172.21.15.102:6789/0 conn(0x7f73480f56f0 legacy=0x7f734810e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:18.789 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:18.801 7f7347fff700 1 -- 172.21.15.102:0/2989273242 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73481ac150 con 0x7f73481a8500 2024-04-17T22:31:18.789 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:18.801 7f73467fc700 1 -- 172.21.15.102:0/2989273242 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4278738352 0 0) 0x7f7340001110 con 0x7f73480f56f0 2024-04-17T22:31:18.789 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:18.801 7f7346ffd700 1 --2- 172.21.15.102:0/2989273242 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f734810eb50 0x7f73481a7ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:18.789 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:18.801 7f7347fff700 1 --2- 172.21.15.102:0/2989273242 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f73481a8500 0x7f73481aa960 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 rx=0x7f7338002f70 tx=0x7f7338009a50).ready entity=mon.0 client_cookie=b059a9fedbeb62cb server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:18.789 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:18.802 7f73467fc700 1 -- 172.21.15.102:0/2989273242 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f73481ac150 con 0x7f73481a8500 2024-04-17T22:31:18.789 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:18.802 7f73467fc700 1 -- 172.21.15.102:0/2989273242 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f733800f510 con 0x7f73481a8500 2024-04-17T22:31:18.790 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:18.802 7f73467fc700 1 -- 172.21.15.102:0/2989273242 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f733800f6f0 con 0x7f73481a8500 2024-04-17T22:31:18.790 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:18.802 7f734f165700 1 -- 172.21.15.102:0/2989273242 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f73481a8500 msgr2=0x7f73481aa960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:18.790 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:18.802 7f734f165700 1 --2- 172.21.15.102:0/2989273242 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f73481a8500 0x7f73481aa960 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 rx=0x7f7338002f70 tx=0x7f7338009a50).stop 2024-04-17T22:31:18.790 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:18.802 7f734f165700 1 -- 172.21.15.102:0/2989273242 shutdown_connections 2024-04-17T22:31:18.790 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:18.802 7f734f165700 1 --2- 172.21.15.102:0/2989273242 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f73481a8500 0x7f73481aa960 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:18.790 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:18.802 7f734f165700 1 --2- 172.21.15.102:0/2989273242 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f734810eb50 0x7f73481a7ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:18.790 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:18.802 7f734f165700 1 -- 172.21.15.102:0/2989273242 >> 172.21.15.102:0/2989273242 conn(0x7f73481135a0 msgr2=0x7f73480f4c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:18.790 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:18.802 7f734f165700 1 -- 172.21.15.102:0/2989273242 shutdown_connections 2024-04-17T22:31:18.790 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:18.802 7f734f165700 1 -- 172.21.15.102:0/2989273242 wait complete. 2024-04-17T22:31:18.810 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:18.825 7f734f165700 1 Processor -- start 2024-04-17T22:31:18.810 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:18.825 7f734f165700 1 -- start start 2024-04-17T22:31:18.813 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:18.825 7f734f165700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f73480d6e00 0x7f73480c2cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:18.813 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:18.825 7f734f165700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f73480c32f0 0x7f73481b3980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:18.813 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:18.825 7f734f165700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f73481b3fb0 0x7f73481b4400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:18.813 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:18.825 7f734f165700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73480f0220 con 0x7f73481b3fb0 2024-04-17T22:31:18.813 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:18.825 7f734f165700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73480c5b10 con 0x7f73480c32f0 2024-04-17T22:31:18.813 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:18.825 7f734f165700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f73480d7250 con 0x7f73480d6e00 2024-04-17T22:31:18.813 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:18.826 7f73477fe700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f73480d6e00 0x7f73480c2cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:18.813 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:18.826 7f7347fff700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f73481b3fb0 0x7f73481b4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:18.813 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:18.826 7f7346ffd700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f73480c32f0 0x7f73481b3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:18.814 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:18.826 7f7347fff700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f73481b3fb0 0x7f73481b4400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:44182/0 (socket says 172.21.15.102:44182) 2024-04-17T22:31:18.814 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:18.826 7f73477fe700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f73480d6e00 0x7f73480c2cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:48982/0 (socket says 172.21.15.102:48982) 2024-04-17T22:31:18.814 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:18.826 7f7347fff700 1 -- 172.21.15.102:0/2374169784 learned_addr learned my addr 172.21.15.102:0/2374169784 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:31:18.814 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:18.826 7f73477fe700 1 -- 172.21.15.102:0/2374169784 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f73480c32f0 msgr2=0x7f73481b3980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:18.814 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:18.826 7f73477fe700 1 --2- 172.21.15.102:0/2374169784 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f73480c32f0 0x7f73481b3980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:18.814 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:18.826 7f73477fe700 1 -- 172.21.15.102:0/2374169784 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f73481b3fb0 msgr2=0x7f73481b4400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:18.814 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:18.826 7f73477fe700 1 --2- 172.21.15.102:0/2374169784 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f73481b3fb0 0x7f73481b4400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:18.814 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:18.826 7f73477fe700 1 -- 172.21.15.102:0/2374169784 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73481bae60 con 0x7f73480d6e00 2024-04-17T22:31:18.814 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:18.826 7f7347fff700 1 --2- 172.21.15.102:0/2374169784 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f73481b3fb0 0x7f73481b4400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:18.814 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:18.826 7f7346ffd700 1 --2- 172.21.15.102:0/2374169784 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f73480c32f0 0x7f73481b3980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:18.814 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:18.826 7f73477fe700 1 --2- 172.21.15.102:0/2374169784 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f73480d6e00 0x7f73480c2cc0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 rx=0x7f73480e1ac0 tx=0x7f734000c7b0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:18.814 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:18.826 7f7344ff9700 1 -- 172.21.15.102:0/2374169784 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f73481bae60 con 0x7f73480d6e00 2024-04-17T22:31:18.815 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:18.826 7f7344ff9700 1 -- 172.21.15.102:0/2374169784 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73400070d0 con 0x7f73480d6e00 2024-04-17T22:31:18.815 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:18.826 7f7344ff9700 1 -- 172.21.15.102:0/2374169784 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f7340005560 con 0x7f73480d6e00 2024-04-17T22:31:18.815 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:18.826 7f734f165700 1 -- 172.21.15.102:0/2374169784 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73481bb060 con 0x7f73480d6e00 2024-04-17T22:31:18.815 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:18.826 7f734f165700 1 -- 172.21.15.102:0/2374169784 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73481bb260 con 0x7f73480d6e00 2024-04-17T22:31:18.815 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:18.827 7f7344ff9700 1 -- 172.21.15.102:0/2374169784 <== mon.2 v2:172.21.15.102:3302/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7f7340005970 con 0x7f73480d6e00 2024-04-17T22:31:18.815 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:18.827 7f7344ff9700 1 -- 172.21.15.102:0/2374169784 <== mon.2 v2:172.21.15.102:3302/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 3237+0+0 (secure 0 0 0) 0x7f73400147a0 con 0x7f73480d6e00 2024-04-17T22:31:18.815 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:18.827 7f734f165700 1 -- 172.21.15.102:0/2374169784 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73481ae170 con 0x7f73480d6e00 2024-04-17T22:31:18.815 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:18.830 7f7344ff9700 1 -- 172.21.15.102:0/2374169784 <== mon.2 v2:172.21.15.102:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+110455 (secure 0 0 0) 0x7f7340005760 con 0x7f73480d6e00 2024-04-17T22:31:18.919 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:18.934 7f734f165700 1 -- 172.21.15.102:0/2374169784 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f73480a6510 con 0x7f73480d6e00 2024-04-17T22:31:19.600 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:19.615 7f7344ff9700 1 -- 172.21.15.102:0/2374169784 <== mon.2 v2:172.21.15.102:3302/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v19) v1 ==== 181+0+0 (secure 0 0 0) 0x7f7340000c30 con 0x7f73480d6e00 2024-04-17T22:31:19.614 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:19.628 7f734f165700 1 -- 172.21.15.102:0/2374169784 >> v1:172.21.15.102:6816/42360 conn(0x7f73200145a0 legacy=0x7f7320016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:19.614 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:19.628 7f734f165700 1 -- 172.21.15.102:0/2374169784 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f73480d6e00 msgr2=0x7f73480c2cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:19.614 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:19.628 7f734f165700 1 --2- 172.21.15.102:0/2374169784 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f73480d6e00 0x7f73480c2cc0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 rx=0x7f73480e1ac0 tx=0x7f734000c7b0).stop 2024-04-17T22:31:19.614 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:19.628 7f734f165700 1 -- 172.21.15.102:0/2374169784 shutdown_connections 2024-04-17T22:31:19.614 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:19.628 7f734f165700 1 --2- 172.21.15.102:0/2374169784 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f73481b3fb0 0x7f73481b4400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:19.614 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:19.628 7f734f165700 1 --2- 172.21.15.102:0/2374169784 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f73480d6e00 0x7f73480c2cc0 secure :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 rx=0x7f73480e1ac0 tx=0x7f734000c7b0).stop 2024-04-17T22:31:19.614 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:19.628 7f734f165700 1 --2- 172.21.15.102:0/2374169784 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f73480c32f0 0x7f73481b3980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:19.614 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:19.628 7f734f165700 1 -- 172.21.15.102:0/2374169784 >> 172.21.15.102:0/2374169784 conn(0x7f73480dce40 msgr2=0x7f73480d8f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:19.614 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:19.629 7f734f165700 1 -- 172.21.15.102:0/2374169784 shutdown_connections 2024-04-17T22:31:19.615 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:19.629 7f734f165700 1 -- 172.21.15.102:0/2374169784 wait complete. 2024-04-17T22:31:19.620 INFO:teuthology.orchestra.run.smithi102.stderr:pool 'cephfs_metadata' removed 2024-04-17T22:31:19.631 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-17T22:31:19.764 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:19.777 7f8dc582c700 1 Processor -- start 2024-04-17T22:31:19.764 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:19.777 7f8dc582c700 1 -- start start 2024-04-17T22:31:19.764 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:19.777 7f8dc582c700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f8dc00f5450 0x7f8dc010be50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:19.764 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:19.777 7f8dc582c700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f8dc010c480 0x7f8dc00f1430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:19.764 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:19.777 7f8dc582c700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f8dc010c8d0 0x7f8dc00f1a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:19.765 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:19.777 7f8dc582c700 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8dc00f9000 con 0x7f8dc010c8d0 2024-04-17T22:31:19.765 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:19.777 7f8dc582c700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8dc00f8d60 con 0x7f8dc00f5450 2024-04-17T22:31:19.765 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:19.777 7f8dc582c700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8dc00f2090 con 0x7f8dc010c480 2024-04-17T22:31:19.765 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:19.777 7f8db7fff700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f8dc010c480 0x7f8dc00f1430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:19.765 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:19.777 7f8dbedc6700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f8dc010c8d0 0x7f8dc00f1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:19.765 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:19.777 7f8dbe5c5700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f8dc00f5450 0x7f8dc010be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:19.765 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:19.777 7f8db7fff700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f8dc010c480 0x7f8dc00f1430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:48998/0 (socket says 172.21.15.102:48998) 2024-04-17T22:31:19.765 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:19.777 7f8db7fff700 1 -- 172.21.15.102:0/3686185280 learned_addr learned my addr 172.21.15.102:0/3686185280 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:31:19.765 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:19.777 7f8dbedc6700 1 --2- 172.21.15.102:0/3686185280 >> v2:172.21.15.102:3300/0 conn(0x7f8dc010c8d0 0x7f8dc00f1a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:44186/0 (socket says 172.21.15.102:44186) 2024-04-17T22:31:19.765 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:19.777 7f8dbedc6700 1 -- 172.21.15.102:0/3686185280 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f8dc010c480 msgr2=0x7f8dc00f1430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:31:19.765 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:19.777 7f8dbedc6700 1 --2- 172.21.15.102:0/3686185280 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f8dc010c480 0x7f8dc00f1430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:19.766 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:19.777 7f8dbedc6700 1 -- 172.21.15.102:0/3686185280 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f8dc00f5450 msgr2=0x7f8dc010be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:31:19.766 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:19.777 7f8dbedc6700 1 --2- 172.21.15.102:0/3686185280 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f8dc00f5450 0x7f8dc010be50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:19.766 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:19.777 7f8dbedc6700 1 -- 172.21.15.102:0/3686185280 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8dc00f22c0 con 0x7f8dc010c8d0 2024-04-17T22:31:19.766 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:19.777 7f8dbe5c5700 1 --2- 172.21.15.102:0/3686185280 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f8dc00f5450 0x7f8dc010be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:19.766 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:19.778 7f8dbedc6700 1 --2- 172.21.15.102:0/3686185280 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8dc010c8d0 0x7f8dc00f1a60 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 rx=0x7f8db80032f0 tx=0x7f8db8009d10).ready entity=mon.0 client_cookie=11beaef1b4a47a29 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:19.766 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:19.778 7f8dbddc4700 1 -- 172.21.15.102:0/3686185280 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8dc00f22c0 con 0x7f8dc010c8d0 2024-04-17T22:31:19.766 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:19.778 7f8dbddc4700 1 -- 172.21.15.102:0/3686185280 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8db800b800 con 0x7f8dc010c8d0 2024-04-17T22:31:19.766 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:19.778 7f8dbddc4700 1 -- 172.21.15.102:0/3686185280 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8db800b9e0 con 0x7f8dc010c8d0 2024-04-17T22:31:19.766 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:19.778 7f8dc582c700 1 -- 172.21.15.102:0/3686185280 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8dc010c8d0 msgr2=0x7f8dc00f1a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:19.766 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:19.778 7f8dc582c700 1 --2- 172.21.15.102:0/3686185280 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8dc010c8d0 0x7f8dc00f1a60 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 rx=0x7f8db80032f0 tx=0x7f8db8009d10).stop 2024-04-17T22:31:19.767 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:19.778 7f8dc582c700 1 -- 172.21.15.102:0/3686185280 shutdown_connections 2024-04-17T22:31:19.767 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:19.778 7f8dc582c700 1 --2- 172.21.15.102:0/3686185280 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8dc010c8d0 0x7f8dc00f1a60 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:19.767 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:19.778 7f8dc582c700 1 --2- 172.21.15.102:0/3686185280 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f8dc00f5450 0x7f8dc010be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:19.767 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:19.778 7f8dc582c700 1 --2- 172.21.15.102:0/3686185280 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f8dc010c480 0x7f8dc00f1430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:19.767 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:19.778 7f8dc582c700 1 -- 172.21.15.102:0/3686185280 >> 172.21.15.102:0/3686185280 conn(0x7f8dc01135a0 msgr2=0x7f8dc00f4c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:19.767 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:19.778 7f8dc582c700 1 -- 172.21.15.102:0/3686185280 shutdown_connections 2024-04-17T22:31:19.767 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:19.778 7f8dc582c700 1 -- 172.21.15.102:0/3686185280 wait complete. 2024-04-17T22:31:19.788 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:19.801 7f8dc582c700 1 Processor -- start 2024-04-17T22:31:19.788 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:19.801 7f8dc582c700 1 -- start start 2024-04-17T22:31:19.788 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:19.801 7f8dc582c700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f8dc01a48d0 0x7f8dc00d17d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:19.788 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:19.801 7f8dc582c700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8dc00d1e00 0x7f8dc01b3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:19.789 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:19.801 7f8dc582c700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f8dc00d2250 0x7f8dc01b3cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:19.789 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:19.801 7f8dc582c700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dc00e61a0 con 0x7f8dc00d1e00 2024-04-17T22:31:19.789 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:19.801 7f8dc582c700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8dc01a4270 con 0x7f8dc00d2250 2024-04-17T22:31:19.789 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:19.801 7f8dc582c700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8dc01a4d20 con 0x7f8dc01a48d0 2024-04-17T22:31:19.789 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:19.801 7f8db7fff700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8dc00d1e00 0x7f8dc01b3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:19.789 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:19.801 7f8dbe5c5700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f8dc01a48d0 0x7f8dc00d17d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:19.789 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:19.801 7f8dbedc6700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f8dc00d2250 0x7f8dc01b3cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:19.789 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:19.801 7f8db7fff700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8dc00d1e00 0x7f8dc01b3680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:44202/0 (socket says 172.21.15.102:44202) 2024-04-17T22:31:19.789 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:19.802 7f8db7fff700 1 -- 172.21.15.102:0/2917423081 learned_addr learned my addr 172.21.15.102:0/2917423081 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:31:19.789 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:19.802 7f8dbedc6700 1 -- 172.21.15.102:0/2917423081 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f8dc01a48d0 msgr2=0x7f8dc00d17d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:19.789 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:19.802 7f8dbedc6700 1 --2- 172.21.15.102:0/2917423081 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f8dc01a48d0 0x7f8dc00d17d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:19.789 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:19.802 7f8dbedc6700 1 -- 172.21.15.102:0/2917423081 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8dc00d1e00 msgr2=0x7f8dc01b3680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:19.790 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:19.802 7f8dbedc6700 1 --2- 172.21.15.102:0/2917423081 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8dc00d1e00 0x7f8dc01b3680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:19.790 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:19.802 7f8dbedc6700 1 -- 172.21.15.102:0/2917423081 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8dc01b42e0 con 0x7f8dc00d2250 2024-04-17T22:31:19.790 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:19.802 7f8db7fff700 1 --2- 172.21.15.102:0/2917423081 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8dc00d1e00 0x7f8dc01b3680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:19.790 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:19.802 7f8dbe5c5700 1 --2- 172.21.15.102:0/2917423081 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f8dc01a48d0 0x7f8dc00d17d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:19.790 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:19.802 7f8dbedc6700 1 --2- 172.21.15.102:0/2917423081 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f8dc00d2250 0x7f8dc01b3cb0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 rx=0x7f8dc00f7c30 tx=0x7f8db8006c40).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:19.790 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:19.802 7f8db77fe700 1 -- 172.21.15.102:0/2917423081 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8dc01b42e0 con 0x7f8dc00d2250 2024-04-17T22:31:19.790 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:19.802 7f8db77fe700 1 -- 172.21.15.102:0/2917423081 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8db8011a50 con 0x7f8dc00d2250 2024-04-17T22:31:19.790 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:19.802 7f8dc582c700 1 -- 172.21.15.102:0/2917423081 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8dc01b45a0 con 0x7f8dc00d2250 2024-04-17T22:31:19.790 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:19.802 7f8db77fe700 1 -- 172.21.15.102:0/2917423081 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8db8017040 con 0x7f8dc00d2250 2024-04-17T22:31:19.790 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:19.802 7f8dc582c700 1 -- 172.21.15.102:0/2917423081 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8dc01bafd0 con 0x7f8dc00d2250 2024-04-17T22:31:19.790 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:19.803 7f8db77fe700 1 -- 172.21.15.102:0/2917423081 <== mon.1 v2:172.21.15.102:3301/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7f8db801c470 con 0x7f8dc00d2250 2024-04-17T22:31:19.791 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:19.803 7f8db77fe700 1 -- 172.21.15.102:0/2917423081 <== mon.1 v2:172.21.15.102:3301/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 2799+0+0 (secure 0 0 0) 0x7f8db8018030 con 0x7f8dc00d2250 2024-04-17T22:31:19.791 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:19.803 7f8dc582c700 1 -- 172.21.15.102:0/2917423081 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8dc00f0500 con 0x7f8dc00d2250 2024-04-17T22:31:19.791 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:19.806 7f8db77fe700 1 -- 172.21.15.102:0/2917423081 <== mon.1 v2:172.21.15.102:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+110455 (secure 0 0 0) 0x7f8db8011840 con 0x7f8dc00d2250 2024-04-17T22:31:19.897 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:19.912 7f8dc582c700 1 -- 172.21.15.102:0/2917423081 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f8dc00d9960 con 0x7f8dc00d2250 2024-04-17T22:31:20.611 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:20.626 7f8db77fe700 1 -- 172.21.15.102:0/2917423081 <== mon.1 v2:172.21.15.102:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v20) v1 ==== 169+0+0 (secure 0 0 0) 0x7f8db800b5f0 con 0x7f8dc00d2250 2024-04-17T22:31:20.615 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:20.630 7f8dc582c700 1 -- 172.21.15.102:0/2917423081 >> v1:172.21.15.102:6816/42360 conn(0x7f8da40145a0 legacy=0x7f8da4016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:20.615 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:20.630 7f8dc582c700 1 -- 172.21.15.102:0/2917423081 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f8dc00d2250 msgr2=0x7f8dc01b3cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:20.615 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:20.630 7f8dc582c700 1 --2- 172.21.15.102:0/2917423081 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f8dc00d2250 0x7f8dc01b3cb0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 rx=0x7f8dc00f7c30 tx=0x7f8db8006c40).stop 2024-04-17T22:31:20.615 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:20.630 7f8dc582c700 1 -- 172.21.15.102:0/2917423081 shutdown_connections 2024-04-17T22:31:20.615 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:20.630 7f8dc582c700 1 --2- 172.21.15.102:0/2917423081 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f8dc00d2250 0x7f8dc01b3cb0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:20.616 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:20.630 7f8dc582c700 1 --2- 172.21.15.102:0/2917423081 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f8dc01a48d0 0x7f8dc00d17d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:20.616 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:20.630 7f8dc582c700 1 --2- 172.21.15.102:0/2917423081 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8dc00d1e00 0x7f8dc01b3680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:20.616 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:20.630 7f8dc582c700 1 -- 172.21.15.102:0/2917423081 >> 172.21.15.102:0/2917423081 conn(0x7f8dc00e1c70 msgr2=0x7f8dc00f5f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:20.616 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:20.630 7f8dc582c700 1 -- 172.21.15.102:0/2917423081 shutdown_connections 2024-04-17T22:31:20.616 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:20.631 7f8dc582c700 1 -- 172.21.15.102:0/2917423081 wait complete. 2024-04-17T22:31:20.617 INFO:teuthology.orchestra.run.smithi102.stderr:pool 'cephfs_data' removed 2024-04-17T22:31:20.633 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-17T22:31:20.633 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-17T22:31:20.633 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-17T22:31:20.633 INFO:teuthology.orchestra.run:waiting for 300 2024-04-17T22:31:20.642 INFO:tasks.ceph.mds.a:Stopped 2024-04-17T22:31:20.643 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-17T22:31:20.685 INFO:tasks.ceph.mds.a:Started 2024-04-17T22:31:20.686 DEBUG:teuthology.parallel:result is None 2024-04-17T22:31:20.686 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-17T22:31:20.763 INFO:tasks.ceph.mds.a.smithi102.stderr:warning: line 130: 'mon_down_mkfs_grace' in section 'mon' redefined 2024-04-17T22:31:20.811 INFO:tasks.ceph.mds.a.smithi102.stdout:starting mds.a at 2024-04-17T22:31:20.822 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:20.837 7f5abe856700 1 Processor -- start 2024-04-17T22:31:20.822 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:20.837 7f5abe856700 1 -- start start 2024-04-17T22:31:20.824 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:20.837 7f5abe856700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f5ab810b810 0x7f5ab81a52c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:20.824 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:20.837 7f5abe856700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5ab810bc60 0x7f5ab81a7900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:20.825 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:20.837 7f5abe856700 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ab8103b40 con 0x7f5ab80f5840 2024-04-17T22:31:20.825 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:20.837 7f5abe856700 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ab80fa370 con 0x7f5ab810b810 2024-04-17T22:31:20.825 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:20.837 7f5abe856700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5ab81a0b00 con 0x7f5ab810bc60 2024-04-17T22:31:20.825 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:20.837 7f5ab75eb700 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f5ab80f5840 0x7f5ab810b010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:36442/0 (socket says 172.21.15.102:36442) 2024-04-17T22:31:20.825 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:20.837 7f5ab75eb700 1 -- 172.21.15.102:0/1127841276 learned_addr learned my addr 172.21.15.102:0/1127841276 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-17T22:31:20.825 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:20.837 7f5aaedea700 1 --2- 172.21.15.102:0/1127841276 >> v2:172.21.15.102:3300/0 conn(0x7f5ab810b810 0x7f5ab81a52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:20.825 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:20.838 7f5ab7dec700 1 --2- 172.21.15.102:0/1127841276 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5ab810bc60 0x7f5ab81a7900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:20.825 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:20.838 7f5ab6dea700 1 -- 172.21.15.102:0/1127841276 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4018135831 0 0) 0x7f5aa8001110 con 0x7f5ab80f5840 2024-04-17T22:31:20.825 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:20.838 7f5ab6dea700 1 -- 172.21.15.102:0/1127841276 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5a98003270 con 0x7f5ab80f5840 2024-04-17T22:31:20.825 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:20.838 7f5aaedea700 1 -- 172.21.15.102:0/1127841276 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5ab810bc60 msgr2=0x7f5ab81a7900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:31:20.825 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:20.838 7f5aaedea700 1 --2- 172.21.15.102:0/1127841276 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5ab810bc60 0x7f5ab81a7900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:20.825 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:20.838 7f5aaedea700 1 -- 172.21.15.102:0/1127841276 >> v1:172.21.15.102:6789/0 conn(0x7f5ab80f5840 legacy=0x7f5ab810b010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:20.825 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:20.838 7f5aaedea700 1 -- 172.21.15.102:0/1127841276 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ab81a8f40 con 0x7f5ab810b810 2024-04-17T22:31:20.825 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:20.838 7f5aaedea700 1 --2- 172.21.15.102:0/1127841276 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5ab810b810 0x7f5ab81a52c0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 rx=0x7f5aa0002f70 tx=0x7f5aa000b9f0).ready entity=mon.0 client_cookie=78f07eb9065f10c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:20.826 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:20.838 7f5ab6dea700 1 -- 172.21.15.102:0/1127841276 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f5ab81a8f40 con 0x7f5ab810b810 2024-04-17T22:31:20.826 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:20.838 7f5ab6dea700 1 -- 172.21.15.102:0/1127841276 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5aa000d5c0 con 0x7f5ab810b810 2024-04-17T22:31:20.826 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:20.838 7f5ab6dea700 1 -- 172.21.15.102:0/1127841276 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f5aa000d990 con 0x7f5ab810b810 2024-04-17T22:31:20.826 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:20.838 7f5abe856700 1 -- 172.21.15.102:0/1127841276 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5ab810b810 msgr2=0x7f5ab81a52c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:20.826 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:20.838 7f5abe856700 1 --2- 172.21.15.102:0/1127841276 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5ab810b810 0x7f5ab81a52c0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 rx=0x7f5aa0002f70 tx=0x7f5aa000b9f0).stop 2024-04-17T22:31:20.826 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:20.839 7f5abe856700 1 -- 172.21.15.102:0/1127841276 shutdown_connections 2024-04-17T22:31:20.826 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:20.839 7f5abe856700 1 --2- 172.21.15.102:0/1127841276 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5ab810bc60 0x7f5ab81a7900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:20.826 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:20.839 7f5abe856700 1 --2- 172.21.15.102:0/1127841276 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5ab810b810 0x7f5ab81a52c0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:20.826 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:20.839 7f5abe856700 1 -- 172.21.15.102:0/1127841276 >> 172.21.15.102:0/1127841276 conn(0x7f5ab81138a0 msgr2=0x7f5ab80f4f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:20.826 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:20.839 7f5abe856700 1 -- 172.21.15.102:0/1127841276 shutdown_connections 2024-04-17T22:31:20.826 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:20.839 7f5abe856700 1 -- 172.21.15.102:0/1127841276 wait complete. 2024-04-17T22:31:20.847 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:20.862 7f5abe856700 1 Processor -- start 2024-04-17T22:31:20.847 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:20.862 7f5abe856700 1 -- start start 2024-04-17T22:31:20.849 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:20.862 7f5abe856700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5ab80d7250 0x7f5ab80d76a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:20.849 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:20.862 7f5abe856700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5ab80d7cd0 0x7f5ab81b1e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:20.849 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:20.862 7f5abe856700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5ab81b24b0 0x7f5ab81b2900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:20.850 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:20.862 7f5abe856700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ab808a800 con 0x7f5ab80d7250 2024-04-17T22:31:20.850 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:20.862 7f5abe856700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ab80c5760 con 0x7f5ab80d7cd0 2024-04-17T22:31:20.850 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:20.862 7f5abe856700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5ab81b2f30 con 0x7f5ab81b24b0 2024-04-17T22:31:20.850 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:20.862 7f5ab75eb700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5ab80d7250 0x7f5ab80d76a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:20.850 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:20.862 7f5aaedea700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5ab80d7cd0 0x7f5ab81b1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:20.850 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:20.862 7f5ab75eb700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5ab80d7250 0x7f5ab80d76a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:44214/0 (socket says 172.21.15.102:44214) 2024-04-17T22:31:20.850 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:20.862 7f5ab75eb700 1 -- 172.21.15.102:0/2535714710 learned_addr learned my addr 172.21.15.102:0/2535714710 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:31:20.850 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:20.862 7f5ab7dec700 1 --2- 172.21.15.102:0/2535714710 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5ab81b24b0 0x7f5ab81b2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:20.850 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:20.863 7f5aaedea700 1 -- 172.21.15.102:0/2535714710 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5ab81b24b0 msgr2=0x7f5ab81b2900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:20.850 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:20.863 7f5aaedea700 1 --2- 172.21.15.102:0/2535714710 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5ab81b24b0 0x7f5ab81b2900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:20.850 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:20.863 7f5aaedea700 1 -- 172.21.15.102:0/2535714710 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5ab80d7250 msgr2=0x7f5ab80d76a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:20.850 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:20.863 7f5aaedea700 1 --2- 172.21.15.102:0/2535714710 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5ab80d7250 0x7f5ab80d76a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:20.850 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:20.863 7f5aaedea700 1 -- 172.21.15.102:0/2535714710 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ab80ceff0 con 0x7f5ab80d7cd0 2024-04-17T22:31:20.850 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:20.863 7f5ab7dec700 1 --2- 172.21.15.102:0/2535714710 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5ab81b24b0 0x7f5ab81b2900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:20.851 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:20.863 7f5ab75eb700 1 --2- 172.21.15.102:0/2535714710 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5ab80d7250 0x7f5ab80d76a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:20.851 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:20.863 7f5aaedea700 1 --2- 172.21.15.102:0/2535714710 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5ab80d7cd0 0x7f5ab81b1e80 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 rx=0x7f5ab81b30e0 tx=0x7f5aa00068c0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:20.851 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:20.863 7f5ab55e7700 1 -- 172.21.15.102:0/2535714710 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f5ab80ceff0 con 0x7f5ab80d7cd0 2024-04-17T22:31:20.851 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:20.863 7f5ab55e7700 1 -- 172.21.15.102:0/2535714710 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5aa0011610 con 0x7f5ab80d7cd0 2024-04-17T22:31:20.851 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:20.863 7f5abe856700 1 -- 172.21.15.102:0/2535714710 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ab80ce310 con 0x7f5ab80d7cd0 2024-04-17T22:31:20.851 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:20.863 7f5ab55e7700 1 -- 172.21.15.102:0/2535714710 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f5aa0017040 con 0x7f5ab80d7cd0 2024-04-17T22:31:20.851 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:20.863 7f5abe856700 1 -- 172.21.15.102:0/2535714710 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ab81b97d0 con 0x7f5ab80d7cd0 2024-04-17T22:31:20.851 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:20.863 7f5ab55e7700 1 -- 172.21.15.102:0/2535714710 <== mon.1 v2:172.21.15.102:3301/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7f5aa0027450 con 0x7f5ab80d7cd0 2024-04-17T22:31:20.851 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:20.863 7f5ab55e7700 1 -- 172.21.15.102:0/2535714710 <== mon.1 v2:172.21.15.102:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 2417+0+0 (secure 0 0 0) 0x7f5aa0018070 con 0x7f5ab80d7cd0 2024-04-17T22:31:20.851 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:20.864 7f5abe856700 1 -- 172.21.15.102:0/2535714710 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ab80f0800 con 0x7f5ab80d7cd0 2024-04-17T22:31:20.852 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:20.867 7f5ab55e7700 1 -- 172.21.15.102:0/2535714710 <== mon.1 v2:172.21.15.102:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+110455 (secure 0 0 0) 0x7f5aa0011400 con 0x7f5ab80d7cd0 2024-04-17T22:31:20.999 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.013 7f5abe856700 1 -- 172.21.15.102:0/2535714710 >> v1:172.21.15.102:6816/42360 conn(0x7f5a9c0145a0 legacy=0x7f5a9c016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:20.999 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.013 7f5abe856700 1 -- 172.21.15.102:0/2535714710 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5ab80d7cd0 msgr2=0x7f5ab81b1e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:20.999 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.013 7f5abe856700 1 --2- 172.21.15.102:0/2535714710 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5ab80d7cd0 0x7f5ab81b1e80 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 rx=0x7f5ab81b30e0 tx=0x7f5aa00068c0).stop 2024-04-17T22:31:20.999 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.013 7f5abe856700 1 -- 172.21.15.102:0/2535714710 shutdown_connections 2024-04-17T22:31:20.999 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.013 7f5abe856700 1 --2- 172.21.15.102:0/2535714710 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5ab81b24b0 0x7f5ab81b2900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:20.999 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.013 7f5abe856700 1 --2- 172.21.15.102:0/2535714710 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5ab80d7250 0x7f5ab80d76a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:20.999 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.013 7f5abe856700 1 --2- 172.21.15.102:0/2535714710 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5ab80d7cd0 0x7f5ab81b1e80 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:20.999 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.013 7f5abe856700 1 -- 172.21.15.102:0/2535714710 >> 172.21.15.102:0/2535714710 conn(0x7f5ab80c5240 msgr2=0x7f5ab80c6fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:20.999 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.014 7f5abe856700 1 -- 172.21.15.102:0/2535714710 shutdown_connections 2024-04-17T22:31:21.000 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.014 7f5abe856700 1 -- 172.21.15.102:0/2535714710 wait complete. 2024-04-17T22:31:21.000 INFO:teuthology.orchestra.run.smithi102.stderr:no valid command found; 10 closest matches: 2024-04-17T22:31:21.000 INFO:teuthology.orchestra.run.smithi102.stderr:osd perf 2024-04-17T22:31:21.000 INFO:teuthology.orchestra.run.smithi102.stderr:osd df {plain|tree} {class|name} {} 2024-04-17T22:31:21.000 INFO:teuthology.orchestra.run.smithi102.stderr:osd blocked-by 2024-04-17T22:31:21.000 INFO:teuthology.orchestra.run.smithi102.stderr:osd pool stats {} 2024-04-17T22:31:21.000 INFO:teuthology.orchestra.run.smithi102.stderr:osd pool scrub [...] 2024-04-17T22:31:21.001 INFO:teuthology.orchestra.run.smithi102.stderr:osd pool deep-scrub [...] 2024-04-17T22:31:21.001 INFO:teuthology.orchestra.run.smithi102.stderr:osd pool repair [...] 2024-04-17T22:31:21.001 INFO:teuthology.orchestra.run.smithi102.stderr:osd pool force-recovery [...] 2024-04-17T22:31:21.001 INFO:teuthology.orchestra.run.smithi102.stderr:osd pool force-backfill [...] 2024-04-17T22:31:21.001 INFO:teuthology.orchestra.run.smithi102.stderr:osd pool cancel-force-recovery [...] 2024-04-17T22:31:21.001 INFO:teuthology.orchestra.run.smithi102.stderr:Error EINVAL: invalid command 2024-04-17T22:31:21.009 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-17T22:31:21.009 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2024-04-17T22:31:21.135 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.148 7f3cedb48700 1 Processor -- start 2024-04-17T22:31:21.135 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.148 7f3cedb48700 1 -- start start 2024-04-17T22:31:21.135 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.148 7f3cedb48700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f3ce810b7a0 0x7f3ce81a5250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:21.135 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.148 7f3cedb48700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f3ce810bbf0 0x7f3ce81a7890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:21.135 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.148 7f3cedb48700 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ce8103ad0 con 0x7f3ce80f57d0 2024-04-17T22:31:21.135 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.148 7f3cedb48700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ce80fa300 con 0x7f3ce810bbf0 2024-04-17T22:31:21.136 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.148 7f3cedb48700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3ce81a0a90 con 0x7f3ce810b7a0 2024-04-17T22:31:21.136 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.148 7f3cdffff700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f3ce810b7a0 0x7f3ce81a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:21.136 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.148 7f3cdffff700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f3ce810b7a0 0x7f3ce81a5250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:49050/0 (socket says 172.21.15.102:49050) 2024-04-17T22:31:21.136 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.148 7f3cdffff700 1 -- 172.21.15.102:0/2122469782 learned_addr learned my addr 172.21.15.102:0/2122469782 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:31:21.136 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.148 7f3ce705e700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f3ce810bbf0 0x7f3ce81a7890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:21.136 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.149 7f3cdffff700 1 -- 172.21.15.102:0/2122469782 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f3ce810bbf0 msgr2=0x7f3ce81a7890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:31:21.136 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.149 7f3cdffff700 1 --2- 172.21.15.102:0/2122469782 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f3ce810bbf0 0x7f3ce81a7890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:21.136 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.149 7f3cdffff700 1 -- 172.21.15.102:0/2122469782 >> v1:172.21.15.102:6789/0 conn(0x7f3ce80f57d0 legacy=0x7f3ce810afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:21.136 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.149 7f3ce605c700 1 -- 172.21.15.102:0/2122469782 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3349240278 0 0) 0x7f3cd8001110 con 0x7f3ce80f57d0 2024-04-17T22:31:21.136 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.149 7f3cdffff700 1 -- 172.21.15.102:0/2122469782 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ce81a8ed0 con 0x7f3ce810b7a0 2024-04-17T22:31:21.136 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.149 7f3ce705e700 1 --2- 172.21.15.102:0/2122469782 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f3ce810bbf0 0x7f3ce81a7890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:21.136 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.149 7f3cdffff700 1 --2- 172.21.15.102:0/2122469782 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f3ce810b7a0 0x7f3ce81a5250 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 rx=0x7f3ce80f4630 tx=0x7f3cd0009a30).ready entity=mon.2 client_cookie=296559815419900c server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:21.136 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.149 7f3ce605c700 1 -- 172.21.15.102:0/2122469782 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f3ce81a8ed0 con 0x7f3ce810b7a0 2024-04-17T22:31:21.136 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.149 7f3ce605c700 1 -- 172.21.15.102:0/2122469782 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cd000f510 con 0x7f3ce810b7a0 2024-04-17T22:31:21.137 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.149 7f3ce605c700 1 -- 172.21.15.102:0/2122469782 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f3cd000f6f0 con 0x7f3ce810b7a0 2024-04-17T22:31:21.137 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.149 7f3cedb48700 1 -- 172.21.15.102:0/2122469782 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f3ce810b7a0 msgr2=0x7f3ce81a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:21.137 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.149 7f3cedb48700 1 --2- 172.21.15.102:0/2122469782 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f3ce810b7a0 0x7f3ce81a5250 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 rx=0x7f3ce80f4630 tx=0x7f3cd0009a30).stop 2024-04-17T22:31:21.137 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.150 7f3cedb48700 1 -- 172.21.15.102:0/2122469782 shutdown_connections 2024-04-17T22:31:21.137 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.150 7f3cedb48700 1 --2- 172.21.15.102:0/2122469782 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f3ce810bbf0 0x7f3ce81a7890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:21.137 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.150 7f3cedb48700 1 --2- 172.21.15.102:0/2122469782 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f3ce810b7a0 0x7f3ce81a5250 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:21.137 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.150 7f3cedb48700 1 -- 172.21.15.102:0/2122469782 >> 172.21.15.102:0/2122469782 conn(0x7f3ce8113830 msgr2=0x7f3ce80f4ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:21.137 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.150 7f3cedb48700 1 -- 172.21.15.102:0/2122469782 shutdown_connections 2024-04-17T22:31:21.137 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.150 7f3cedb48700 1 -- 172.21.15.102:0/2122469782 wait complete. 2024-04-17T22:31:21.159 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.172 7f3cedb48700 1 Processor -- start 2024-04-17T22:31:21.159 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.172 7f3cedb48700 1 -- start start 2024-04-17T22:31:21.159 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.173 7f3cedb48700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3ce80f1740 0x7f3ce80f1b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:21.159 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.173 7f3cedb48700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f3ce80f21c0 0x7f3ce81ad780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:21.159 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.173 7f3cedb48700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f3ce81addb0 0x7f3ce81bba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:21.160 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.173 7f3cedb48700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ce80d4570 con 0x7f3ce80f1740 2024-04-17T22:31:21.160 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.173 7f3cedb48700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ce80d48c0 con 0x7f3ce80f21c0 2024-04-17T22:31:21.160 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.173 7f3cedb48700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3ce80c4750 con 0x7f3ce81addb0 2024-04-17T22:31:21.160 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.173 7f3cdffff700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f3ce80f21c0 0x7f3ce81ad780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:21.160 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.173 7f3ce705e700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f3ce81addb0 0x7f3ce81bba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:21.160 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.173 7f3cdffff700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f3ce80f21c0 0x7f3ce81ad780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:58960/0 (socket says 172.21.15.102:58960) 2024-04-17T22:31:21.160 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.173 7f3cdffff700 1 -- 172.21.15.102:0/1692663216 learned_addr learned my addr 172.21.15.102:0/1692663216 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:31:21.160 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.173 7f3ce685d700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3ce80f1740 0x7f3ce80f1b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:21.160 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.173 7f3cdffff700 1 -- 172.21.15.102:0/1692663216 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f3ce81addb0 msgr2=0x7f3ce81bba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:21.160 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.173 7f3cdffff700 1 --2- 172.21.15.102:0/1692663216 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f3ce81addb0 0x7f3ce81bba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:21.160 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.173 7f3cdffff700 1 -- 172.21.15.102:0/1692663216 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3ce80f1740 msgr2=0x7f3ce80f1b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:21.160 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.173 7f3cdffff700 1 --2- 172.21.15.102:0/1692663216 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3ce80f1740 0x7f3ce80f1b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:21.160 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.173 7f3cdffff700 1 -- 172.21.15.102:0/1692663216 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ce80d5480 con 0x7f3ce80f21c0 2024-04-17T22:31:21.160 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.173 7f3cdffff700 1 --2- 172.21.15.102:0/1692663216 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f3ce80f21c0 0x7f3ce81ad780 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 rx=0x7f3ce80c8410 tx=0x7f3cd00068c0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:21.161 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.173 7f3ce4859700 1 -- 172.21.15.102:0/1692663216 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f3ce80d5480 con 0x7f3ce80f21c0 2024-04-17T22:31:21.161 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.173 7f3ce4859700 1 -- 172.21.15.102:0/1692663216 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cd0011690 con 0x7f3ce80f21c0 2024-04-17T22:31:21.161 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.173 7f3cedb48700 1 -- 172.21.15.102:0/1692663216 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ce80d50d0 con 0x7f3ce80f21c0 2024-04-17T22:31:21.161 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.173 7f3ce4859700 1 -- 172.21.15.102:0/1692663216 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f3cd0011a60 con 0x7f3ce80f21c0 2024-04-17T22:31:21.161 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.173 7f3cedb48700 1 -- 172.21.15.102:0/1692663216 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ce81ae3e0 con 0x7f3ce80f21c0 2024-04-17T22:31:21.161 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.174 7f3ce4859700 1 -- 172.21.15.102:0/1692663216 <== mon.1 v2:172.21.15.102:3301/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7f3cd0019030 con 0x7f3ce80f21c0 2024-04-17T22:31:21.161 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.174 7f3ce4859700 1 -- 172.21.15.102:0/1692663216 <== mon.1 v2:172.21.15.102:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 2417+0+0 (secure 0 0 0) 0x7f3cd0018040 con 0x7f3ce80f21c0 2024-04-17T22:31:21.161 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.174 7f3cedb48700 1 -- 172.21.15.102:0/1692663216 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ce80f0790 con 0x7f3ce80f21c0 2024-04-17T22:31:21.162 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.177 7f3ce4859700 1 -- 172.21.15.102:0/1692663216 <== mon.1 v2:172.21.15.102:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+110455 (secure 0 0 0) 0x7f3cd0011480 con 0x7f3ce80f21c0 2024-04-17T22:31:21.270 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.285 7f3cedb48700 1 -- 172.21.15.102:0/1692663216 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f3ce80f2610 con 0x7f3ce80f21c0 2024-04-17T22:31:21.271 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.286 7f3ce4859700 1 -- 172.21.15.102:0/1692663216 <== mon.1 v2:172.21.15.102:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v20) v1 ==== 81+0+11848 (secure 0 0 0) 0x7f3cd000f300 con 0x7f3ce80f21c0 2024-04-17T22:31:21.271 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-17T22:31:21.271 INFO:teuthology.orchestra.run.smithi102.stdout:{ 2024-04-17T22:31:21.271 INFO:teuthology.orchestra.run.smithi102.stdout: "epoch": 20, 2024-04-17T22:31:21.271 INFO:teuthology.orchestra.run.smithi102.stdout: "fsid": "cb63e789-091d-49b1-b120-55f9db526871", 2024-04-17T22:31:21.271 INFO:teuthology.orchestra.run.smithi102.stdout: "created": "2024-04-17 22:28:48.209635", 2024-04-17T22:31:21.271 INFO:teuthology.orchestra.run.smithi102.stdout: "modified": "2024-04-17 22:31:20.613707", 2024-04-17T22:31:21.271 INFO:teuthology.orchestra.run.smithi102.stdout: "last_up_change": "2024-04-17 22:28:54.236215", 2024-04-17T22:31:21.272 INFO:teuthology.orchestra.run.smithi102.stdout: "last_in_change": "2024-04-17 22:28:54.236215", 2024-04-17T22:31:21.272 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-04-17T22:31:21.272 INFO:teuthology.orchestra.run.smithi102.stdout: "flags_num": 5799936, 2024-04-17T22:31:21.272 INFO:teuthology.orchestra.run.smithi102.stdout: "flags_set": [ 2024-04-17T22:31:21.272 INFO:teuthology.orchestra.run.smithi102.stdout: "pglog_hardlimit", 2024-04-17T22:31:21.272 INFO:teuthology.orchestra.run.smithi102.stdout: "purged_snapdirs", 2024-04-17T22:31:21.272 INFO:teuthology.orchestra.run.smithi102.stdout: "recovery_deletes", 2024-04-17T22:31:21.272 INFO:teuthology.orchestra.run.smithi102.stdout: "sortbitwise" 2024-04-17T22:31:21.272 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-17T22:31:21.272 INFO:teuthology.orchestra.run.smithi102.stdout: "crush_version": 4, 2024-04-17T22:31:21.272 INFO:teuthology.orchestra.run.smithi102.stdout: "full_ratio": 0.94999998807907104, 2024-04-17T22:31:21.272 INFO:teuthology.orchestra.run.smithi102.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-04-17T22:31:21.272 INFO:teuthology.orchestra.run.smithi102.stdout: "nearfull_ratio": 0.85000002384185791, 2024-04-17T22:31:21.272 INFO:teuthology.orchestra.run.smithi102.stdout: "cluster_snapshot": "", 2024-04-17T22:31:21.273 INFO:teuthology.orchestra.run.smithi102.stdout: "pool_max": 3, 2024-04-17T22:31:21.273 INFO:teuthology.orchestra.run.smithi102.stdout: "max_osd": 4, 2024-04-17T22:31:21.273 INFO:teuthology.orchestra.run.smithi102.stdout: "require_min_compat_client": "nautilus", 2024-04-17T22:31:21.273 INFO:teuthology.orchestra.run.smithi102.stdout: "min_compat_client": "jewel", 2024-04-17T22:31:21.273 INFO:teuthology.orchestra.run.smithi102.stdout: "require_osd_release": "nautilus", 2024-04-17T22:31:21.273 INFO:teuthology.orchestra.run.smithi102.stdout: "pools": [ 2024-04-17T22:31:21.273 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-04-17T22:31:21.273 INFO:teuthology.orchestra.run.smithi102.stdout: "pool": 1, 2024-04-17T22:31:21.273 INFO:teuthology.orchestra.run.smithi102.stdout: "pool_name": "rbd", 2024-04-17T22:31:21.273 INFO:teuthology.orchestra.run.smithi102.stdout: "create_time": "2024-04-17 22:28:57.478545", 2024-04-17T22:31:21.273 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-17T22:31:21.273 INFO:teuthology.orchestra.run.smithi102.stdout: "flags_names": "hashpspool", 2024-04-17T22:31:21.273 INFO:teuthology.orchestra.run.smithi102.stdout: "type": 1, 2024-04-17T22:31:21.274 INFO:teuthology.orchestra.run.smithi102.stdout: "size": 2, 2024-04-17T22:31:21.274 INFO:teuthology.orchestra.run.smithi102.stdout: "min_size": 1, 2024-04-17T22:31:21.274 INFO:teuthology.orchestra.run.smithi102.stdout: "crush_rule": 0, 2024-04-17T22:31:21.274 INFO:teuthology.orchestra.run.smithi102.stdout: "object_hash": 2, 2024-04-17T22:31:21.274 INFO:teuthology.orchestra.run.smithi102.stdout: "pg_autoscale_mode": "off", 2024-04-17T22:31:21.274 INFO:teuthology.orchestra.run.smithi102.stdout: "pg_num": 8, 2024-04-17T22:31:21.274 INFO:teuthology.orchestra.run.smithi102.stdout: "pg_placement_num": 8, 2024-04-17T22:31:21.274 INFO:teuthology.orchestra.run.smithi102.stdout: "pg_placement_num_target": 8, 2024-04-17T22:31:21.274 INFO:teuthology.orchestra.run.smithi102.stdout: "pg_num_target": 8, 2024-04-17T22:31:21.274 INFO:teuthology.orchestra.run.smithi102.stdout: "pg_num_pending": 8, 2024-04-17T22:31:21.274 INFO:teuthology.orchestra.run.smithi102.stdout: "last_pg_merge_meta": { 2024-04-17T22:31:21.274 INFO:teuthology.orchestra.run.smithi102.stdout: "source_pgid": "0.0", 2024-04-17T22:31:21.274 INFO:teuthology.orchestra.run.smithi102.stdout: "ready_epoch": 0, 2024-04-17T22:31:21.275 INFO:teuthology.orchestra.run.smithi102.stdout: "last_epoch_started": 0, 2024-04-17T22:31:21.275 INFO:teuthology.orchestra.run.smithi102.stdout: "last_epoch_clean": 0, 2024-04-17T22:31:21.275 INFO:teuthology.orchestra.run.smithi102.stdout: "source_version": "0'0", 2024-04-17T22:31:21.275 INFO:teuthology.orchestra.run.smithi102.stdout: "target_version": "0'0" 2024-04-17T22:31:21.275 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-17T22:31:21.275 INFO:teuthology.orchestra.run.smithi102.stdout: "last_change": "12", 2024-04-17T22:31:21.275 INFO:teuthology.orchestra.run.smithi102.stdout: "last_force_op_resend": "0", 2024-04-17T22:31:21.275 INFO:teuthology.orchestra.run.smithi102.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-17T22:31:21.275 INFO:teuthology.orchestra.run.smithi102.stdout: "last_force_op_resend_preluminous": "0", 2024-04-17T22:31:21.275 INFO:teuthology.orchestra.run.smithi102.stdout: "auid": 0, 2024-04-17T22:31:21.275 INFO:teuthology.orchestra.run.smithi102.stdout: "snap_mode": "selfmanaged", 2024-04-17T22:31:21.275 INFO:teuthology.orchestra.run.smithi102.stdout: "snap_seq": 0, 2024-04-17T22:31:21.275 INFO:teuthology.orchestra.run.smithi102.stdout: "snap_epoch": 0, 2024-04-17T22:31:21.275 INFO:teuthology.orchestra.run.smithi102.stdout: "pool_snaps": [], 2024-04-17T22:31:21.275 INFO:teuthology.orchestra.run.smithi102.stdout: "removed_snaps": "[]", 2024-04-17T22:31:21.276 INFO:teuthology.orchestra.run.smithi102.stdout: "quota_max_bytes": 0, 2024-04-17T22:31:21.276 INFO:teuthology.orchestra.run.smithi102.stdout: "quota_max_objects": 0, 2024-04-17T22:31:21.276 INFO:teuthology.orchestra.run.smithi102.stdout: "tiers": [], 2024-04-17T22:31:21.276 INFO:teuthology.orchestra.run.smithi102.stdout: "tier_of": -1, 2024-04-17T22:31:21.276 INFO:teuthology.orchestra.run.smithi102.stdout: "read_tier": -1, 2024-04-17T22:31:21.276 INFO:teuthology.orchestra.run.smithi102.stdout: "write_tier": -1, 2024-04-17T22:31:21.276 INFO:teuthology.orchestra.run.smithi102.stdout: "cache_mode": "none", 2024-04-17T22:31:21.276 INFO:teuthology.orchestra.run.smithi102.stdout: "target_max_bytes": 0, 2024-04-17T22:31:21.276 INFO:teuthology.orchestra.run.smithi102.stdout: "target_max_objects": 0, 2024-04-17T22:31:21.276 INFO:teuthology.orchestra.run.smithi102.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-17T22:31:21.276 INFO:teuthology.orchestra.run.smithi102.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-17T22:31:21.276 INFO:teuthology.orchestra.run.smithi102.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-17T22:31:21.276 INFO:teuthology.orchestra.run.smithi102.stdout: "cache_min_flush_age": 0, 2024-04-17T22:31:21.276 INFO:teuthology.orchestra.run.smithi102.stdout: "cache_min_evict_age": 0, 2024-04-17T22:31:21.277 INFO:teuthology.orchestra.run.smithi102.stdout: "erasure_code_profile": "", 2024-04-17T22:31:21.277 INFO:teuthology.orchestra.run.smithi102.stdout: "hit_set_params": { 2024-04-17T22:31:21.277 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "none" 2024-04-17T22:31:21.277 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-17T22:31:21.277 INFO:teuthology.orchestra.run.smithi102.stdout: "hit_set_period": 0, 2024-04-17T22:31:21.277 INFO:teuthology.orchestra.run.smithi102.stdout: "hit_set_count": 0, 2024-04-17T22:31:21.277 INFO:teuthology.orchestra.run.smithi102.stdout: "use_gmt_hitset": true, 2024-04-17T22:31:21.277 INFO:teuthology.orchestra.run.smithi102.stdout: "min_read_recency_for_promote": 0, 2024-04-17T22:31:21.277 INFO:teuthology.orchestra.run.smithi102.stdout: "min_write_recency_for_promote": 0, 2024-04-17T22:31:21.277 INFO:teuthology.orchestra.run.smithi102.stdout: "hit_set_grade_decay_rate": 0, 2024-04-17T22:31:21.277 INFO:teuthology.orchestra.run.smithi102.stdout: "hit_set_search_last_n": 0, 2024-04-17T22:31:21.277 INFO:teuthology.orchestra.run.smithi102.stdout: "grade_table": [], 2024-04-17T22:31:21.277 INFO:teuthology.orchestra.run.smithi102.stdout: "stripe_width": 0, 2024-04-17T22:31:21.278 INFO:teuthology.orchestra.run.smithi102.stdout: "expected_num_objects": 0, 2024-04-17T22:31:21.278 INFO:teuthology.orchestra.run.smithi102.stdout: "fast_read": false, 2024-04-17T22:31:21.278 INFO:teuthology.orchestra.run.smithi102.stdout: "options": {}, 2024-04-17T22:31:21.278 INFO:teuthology.orchestra.run.smithi102.stdout: "application_metadata": { 2024-04-17T22:31:21.278 INFO:teuthology.orchestra.run.smithi102.stdout: "rbd": {} 2024-04-17T22:31:21.278 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-17T22:31:21.278 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-17T22:31:21.278 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-17T22:31:21.278 INFO:teuthology.orchestra.run.smithi102.stdout: "osds": [ 2024-04-17T22:31:21.278 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-04-17T22:31:21.278 INFO:teuthology.orchestra.run.smithi102.stdout: "osd": 0, 2024-04-17T22:31:21.278 INFO:teuthology.orchestra.run.smithi102.stdout: "uuid": "3351ef85-ade4-4b90-a8f4-213f4e00ba01", 2024-04-17T22:31:21.278 INFO:teuthology.orchestra.run.smithi102.stdout: "up": 1, 2024-04-17T22:31:21.278 INFO:teuthology.orchestra.run.smithi102.stdout: "in": 1, 2024-04-17T22:31:21.279 INFO:teuthology.orchestra.run.smithi102.stdout: "weight": 1, 2024-04-17T22:31:21.279 INFO:teuthology.orchestra.run.smithi102.stdout: "primary_affinity": 1, 2024-04-17T22:31:21.279 INFO:teuthology.orchestra.run.smithi102.stdout: "last_clean_begin": 0, 2024-04-17T22:31:21.279 INFO:teuthology.orchestra.run.smithi102.stdout: "last_clean_end": 0, 2024-04-17T22:31:21.279 INFO:teuthology.orchestra.run.smithi102.stdout: "up_from": 9, 2024-04-17T22:31:21.279 INFO:teuthology.orchestra.run.smithi102.stdout: "up_thru": 14, 2024-04-17T22:31:21.279 INFO:teuthology.orchestra.run.smithi102.stdout: "down_at": 0, 2024-04-17T22:31:21.279 INFO:teuthology.orchestra.run.smithi102.stdout: "lost_at": 0, 2024-04-17T22:31:21.279 INFO:teuthology.orchestra.run.smithi102.stdout: "public_addrs": { 2024-04-17T22:31:21.279 INFO:teuthology.orchestra.run.smithi102.stdout: "addrvec": [ 2024-04-17T22:31:21.279 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-04-17T22:31:21.279 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "v1", 2024-04-17T22:31:21.279 INFO:teuthology.orchestra.run.smithi102.stdout: "addr": "172.21.15.102:6812", 2024-04-17T22:31:21.279 INFO:teuthology.orchestra.run.smithi102.stdout: "nonce": 42955 2024-04-17T22:31:21.280 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-17T22:31:21.280 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-04-17T22:31:21.280 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-17T22:31:21.280 INFO:teuthology.orchestra.run.smithi102.stdout: "cluster_addrs": { 2024-04-17T22:31:21.280 INFO:teuthology.orchestra.run.smithi102.stdout: "addrvec": [ 2024-04-17T22:31:21.280 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-04-17T22:31:21.280 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "v1", 2024-04-17T22:31:21.280 INFO:teuthology.orchestra.run.smithi102.stdout: "addr": "172.21.15.102:6813", 2024-04-17T22:31:21.280 INFO:teuthology.orchestra.run.smithi102.stdout: "nonce": 42955 2024-04-17T22:31:21.280 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-17T22:31:21.280 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-04-17T22:31:21.280 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-17T22:31:21.280 INFO:teuthology.orchestra.run.smithi102.stdout: "heartbeat_back_addrs": { 2024-04-17T22:31:21.280 INFO:teuthology.orchestra.run.smithi102.stdout: "addrvec": [ 2024-04-17T22:31:21.281 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-04-17T22:31:21.281 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "v1", 2024-04-17T22:31:21.281 INFO:teuthology.orchestra.run.smithi102.stdout: "addr": "172.21.15.102:6815", 2024-04-17T22:31:21.281 INFO:teuthology.orchestra.run.smithi102.stdout: "nonce": 42955 2024-04-17T22:31:21.281 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-17T22:31:21.281 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-04-17T22:31:21.281 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-17T22:31:21.281 INFO:teuthology.orchestra.run.smithi102.stdout: "heartbeat_front_addrs": { 2024-04-17T22:31:21.281 INFO:teuthology.orchestra.run.smithi102.stdout: "addrvec": [ 2024-04-17T22:31:21.281 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-04-17T22:31:21.281 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "v1", 2024-04-17T22:31:21.281 INFO:teuthology.orchestra.run.smithi102.stdout: "addr": "172.21.15.102:6814", 2024-04-17T22:31:21.281 INFO:teuthology.orchestra.run.smithi102.stdout: "nonce": 42955 2024-04-17T22:31:21.282 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-17T22:31:21.282 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-04-17T22:31:21.282 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-17T22:31:21.282 INFO:teuthology.orchestra.run.smithi102.stdout: "public_addr": "172.21.15.102:6812/42955", 2024-04-17T22:31:21.282 INFO:teuthology.orchestra.run.smithi102.stdout: "cluster_addr": "172.21.15.102:6813/42955", 2024-04-17T22:31:21.282 INFO:teuthology.orchestra.run.smithi102.stdout: "heartbeat_back_addr": "172.21.15.102:6815/42955", 2024-04-17T22:31:21.282 INFO:teuthology.orchestra.run.smithi102.stdout: "heartbeat_front_addr": "172.21.15.102:6814/42955", 2024-04-17T22:31:21.282 INFO:teuthology.orchestra.run.smithi102.stdout: "state": [ 2024-04-17T22:31:21.282 INFO:teuthology.orchestra.run.smithi102.stdout: "exists", 2024-04-17T22:31:21.282 INFO:teuthology.orchestra.run.smithi102.stdout: "up" 2024-04-17T22:31:21.282 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-04-17T22:31:21.282 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-17T22:31:21.282 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-04-17T22:31:21.282 INFO:teuthology.orchestra.run.smithi102.stdout: "osd": 1, 2024-04-17T22:31:21.283 INFO:teuthology.orchestra.run.smithi102.stdout: "uuid": "04f67eac-f36b-48f5-982d-4cfc776596d0", 2024-04-17T22:31:21.283 INFO:teuthology.orchestra.run.smithi102.stdout: "up": 1, 2024-04-17T22:31:21.283 INFO:teuthology.orchestra.run.smithi102.stdout: "in": 1, 2024-04-17T22:31:21.283 INFO:teuthology.orchestra.run.smithi102.stdout: "weight": 1, 2024-04-17T22:31:21.283 INFO:teuthology.orchestra.run.smithi102.stdout: "primary_affinity": 1, 2024-04-17T22:31:21.283 INFO:teuthology.orchestra.run.smithi102.stdout: "last_clean_begin": 0, 2024-04-17T22:31:21.283 INFO:teuthology.orchestra.run.smithi102.stdout: "last_clean_end": 0, 2024-04-17T22:31:21.283 INFO:teuthology.orchestra.run.smithi102.stdout: "up_from": 9, 2024-04-17T22:31:21.283 INFO:teuthology.orchestra.run.smithi102.stdout: "up_thru": 14, 2024-04-17T22:31:21.283 INFO:teuthology.orchestra.run.smithi102.stdout: "down_at": 0, 2024-04-17T22:31:21.283 INFO:teuthology.orchestra.run.smithi102.stdout: "lost_at": 0, 2024-04-17T22:31:21.283 INFO:teuthology.orchestra.run.smithi102.stdout: "public_addrs": { 2024-04-17T22:31:21.283 INFO:teuthology.orchestra.run.smithi102.stdout: "addrvec": [ 2024-04-17T22:31:21.283 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-04-17T22:31:21.284 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "v1", 2024-04-17T22:31:21.284 INFO:teuthology.orchestra.run.smithi102.stdout: "addr": "172.21.15.102:6808", 2024-04-17T22:31:21.284 INFO:teuthology.orchestra.run.smithi102.stdout: "nonce": 42958 2024-04-17T22:31:21.284 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-17T22:31:21.284 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-04-17T22:31:21.284 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-17T22:31:21.284 INFO:teuthology.orchestra.run.smithi102.stdout: "cluster_addrs": { 2024-04-17T22:31:21.284 INFO:teuthology.orchestra.run.smithi102.stdout: "addrvec": [ 2024-04-17T22:31:21.284 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-04-17T22:31:21.284 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "v1", 2024-04-17T22:31:21.284 INFO:teuthology.orchestra.run.smithi102.stdout: "addr": "172.21.15.102:6809", 2024-04-17T22:31:21.284 INFO:teuthology.orchestra.run.smithi102.stdout: "nonce": 42958 2024-04-17T22:31:21.284 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-17T22:31:21.284 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-04-17T22:31:21.285 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-17T22:31:21.285 INFO:teuthology.orchestra.run.smithi102.stdout: "heartbeat_back_addrs": { 2024-04-17T22:31:21.285 INFO:teuthology.orchestra.run.smithi102.stdout: "addrvec": [ 2024-04-17T22:31:21.285 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-04-17T22:31:21.285 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "v1", 2024-04-17T22:31:21.285 INFO:teuthology.orchestra.run.smithi102.stdout: "addr": "172.21.15.102:6811", 2024-04-17T22:31:21.285 INFO:teuthology.orchestra.run.smithi102.stdout: "nonce": 42958 2024-04-17T22:31:21.285 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-17T22:31:21.285 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-04-17T22:31:21.285 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-17T22:31:21.285 INFO:teuthology.orchestra.run.smithi102.stdout: "heartbeat_front_addrs": { 2024-04-17T22:31:21.285 INFO:teuthology.orchestra.run.smithi102.stdout: "addrvec": [ 2024-04-17T22:31:21.285 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-04-17T22:31:21.285 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "v1", 2024-04-17T22:31:21.286 INFO:teuthology.orchestra.run.smithi102.stdout: "addr": "172.21.15.102:6810", 2024-04-17T22:31:21.286 INFO:teuthology.orchestra.run.smithi102.stdout: "nonce": 42958 2024-04-17T22:31:21.286 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-17T22:31:21.286 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-04-17T22:31:21.286 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-17T22:31:21.286 INFO:teuthology.orchestra.run.smithi102.stdout: "public_addr": "172.21.15.102:6808/42958", 2024-04-17T22:31:21.286 INFO:teuthology.orchestra.run.smithi102.stdout: "cluster_addr": "172.21.15.102:6809/42958", 2024-04-17T22:31:21.286 INFO:teuthology.orchestra.run.smithi102.stdout: "heartbeat_back_addr": "172.21.15.102:6811/42958", 2024-04-17T22:31:21.286 INFO:teuthology.orchestra.run.smithi102.stdout: "heartbeat_front_addr": "172.21.15.102:6810/42958", 2024-04-17T22:31:21.286 INFO:teuthology.orchestra.run.smithi102.stdout: "state": [ 2024-04-17T22:31:21.286 INFO:teuthology.orchestra.run.smithi102.stdout: "exists", 2024-04-17T22:31:21.286 INFO:teuthology.orchestra.run.smithi102.stdout: "up" 2024-04-17T22:31:21.286 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-04-17T22:31:21.287 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-17T22:31:21.287 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-04-17T22:31:21.287 INFO:teuthology.orchestra.run.smithi102.stdout: "osd": 2, 2024-04-17T22:31:21.287 INFO:teuthology.orchestra.run.smithi102.stdout: "uuid": "d524204b-70b2-40a8-8f73-f9e7d44318af", 2024-04-17T22:31:21.287 INFO:teuthology.orchestra.run.smithi102.stdout: "up": 1, 2024-04-17T22:31:21.287 INFO:teuthology.orchestra.run.smithi102.stdout: "in": 1, 2024-04-17T22:31:21.287 INFO:teuthology.orchestra.run.smithi102.stdout: "weight": 1, 2024-04-17T22:31:21.287 INFO:teuthology.orchestra.run.smithi102.stdout: "primary_affinity": 1, 2024-04-17T22:31:21.287 INFO:teuthology.orchestra.run.smithi102.stdout: "last_clean_begin": 0, 2024-04-17T22:31:21.287 INFO:teuthology.orchestra.run.smithi102.stdout: "last_clean_end": 0, 2024-04-17T22:31:21.287 INFO:teuthology.orchestra.run.smithi102.stdout: "up_from": 9, 2024-04-17T22:31:21.287 INFO:teuthology.orchestra.run.smithi102.stdout: "up_thru": 14, 2024-04-17T22:31:21.287 INFO:teuthology.orchestra.run.smithi102.stdout: "down_at": 0, 2024-04-17T22:31:21.288 INFO:teuthology.orchestra.run.smithi102.stdout: "lost_at": 0, 2024-04-17T22:31:21.288 INFO:teuthology.orchestra.run.smithi102.stdout: "public_addrs": { 2024-04-17T22:31:21.288 INFO:teuthology.orchestra.run.smithi102.stdout: "addrvec": [ 2024-04-17T22:31:21.288 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-04-17T22:31:21.288 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "v1", 2024-04-17T22:31:21.288 INFO:teuthology.orchestra.run.smithi102.stdout: "addr": "172.21.15.102:6804", 2024-04-17T22:31:21.288 INFO:teuthology.orchestra.run.smithi102.stdout: "nonce": 42954 2024-04-17T22:31:21.288 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-17T22:31:21.288 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-04-17T22:31:21.288 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-17T22:31:21.288 INFO:teuthology.orchestra.run.smithi102.stdout: "cluster_addrs": { 2024-04-17T22:31:21.288 INFO:teuthology.orchestra.run.smithi102.stdout: "addrvec": [ 2024-04-17T22:31:21.288 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-04-17T22:31:21.288 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "v1", 2024-04-17T22:31:21.288 INFO:teuthology.orchestra.run.smithi102.stdout: "addr": "172.21.15.102:6805", 2024-04-17T22:31:21.289 INFO:teuthology.orchestra.run.smithi102.stdout: "nonce": 42954 2024-04-17T22:31:21.289 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-17T22:31:21.289 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-04-17T22:31:21.289 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-17T22:31:21.289 INFO:teuthology.orchestra.run.smithi102.stdout: "heartbeat_back_addrs": { 2024-04-17T22:31:21.289 INFO:teuthology.orchestra.run.smithi102.stdout: "addrvec": [ 2024-04-17T22:31:21.289 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-04-17T22:31:21.289 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "v1", 2024-04-17T22:31:21.289 INFO:teuthology.orchestra.run.smithi102.stdout: "addr": "172.21.15.102:6807", 2024-04-17T22:31:21.289 INFO:teuthology.orchestra.run.smithi102.stdout: "nonce": 42954 2024-04-17T22:31:21.289 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-17T22:31:21.289 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-04-17T22:31:21.289 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-17T22:31:21.289 INFO:teuthology.orchestra.run.smithi102.stdout: "heartbeat_front_addrs": { 2024-04-17T22:31:21.290 INFO:teuthology.orchestra.run.smithi102.stdout: "addrvec": [ 2024-04-17T22:31:21.290 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-04-17T22:31:21.290 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "v1", 2024-04-17T22:31:21.290 INFO:teuthology.orchestra.run.smithi102.stdout: "addr": "172.21.15.102:6806", 2024-04-17T22:31:21.290 INFO:teuthology.orchestra.run.smithi102.stdout: "nonce": 42954 2024-04-17T22:31:21.290 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-17T22:31:21.290 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-04-17T22:31:21.290 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-17T22:31:21.290 INFO:teuthology.orchestra.run.smithi102.stdout: "public_addr": "172.21.15.102:6804/42954", 2024-04-17T22:31:21.290 INFO:teuthology.orchestra.run.smithi102.stdout: "cluster_addr": "172.21.15.102:6805/42954", 2024-04-17T22:31:21.290 INFO:teuthology.orchestra.run.smithi102.stdout: "heartbeat_back_addr": "172.21.15.102:6807/42954", 2024-04-17T22:31:21.290 INFO:teuthology.orchestra.run.smithi102.stdout: "heartbeat_front_addr": "172.21.15.102:6806/42954", 2024-04-17T22:31:21.290 INFO:teuthology.orchestra.run.smithi102.stdout: "state": [ 2024-04-17T22:31:21.290 INFO:teuthology.orchestra.run.smithi102.stdout: "exists", 2024-04-17T22:31:21.291 INFO:teuthology.orchestra.run.smithi102.stdout: "up" 2024-04-17T22:31:21.291 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-04-17T22:31:21.291 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-17T22:31:21.291 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-04-17T22:31:21.291 INFO:teuthology.orchestra.run.smithi102.stdout: "osd": 3, 2024-04-17T22:31:21.291 INFO:teuthology.orchestra.run.smithi102.stdout: "uuid": "4fc9d20c-1ad0-484f-aafa-2c9a7574245b", 2024-04-17T22:31:21.291 INFO:teuthology.orchestra.run.smithi102.stdout: "up": 1, 2024-04-17T22:31:21.291 INFO:teuthology.orchestra.run.smithi102.stdout: "in": 1, 2024-04-17T22:31:21.291 INFO:teuthology.orchestra.run.smithi102.stdout: "weight": 1, 2024-04-17T22:31:21.291 INFO:teuthology.orchestra.run.smithi102.stdout: "primary_affinity": 1, 2024-04-17T22:31:21.291 INFO:teuthology.orchestra.run.smithi102.stdout: "last_clean_begin": 0, 2024-04-17T22:31:21.291 INFO:teuthology.orchestra.run.smithi102.stdout: "last_clean_end": 0, 2024-04-17T22:31:21.291 INFO:teuthology.orchestra.run.smithi102.stdout: "up_from": 9, 2024-04-17T22:31:21.291 INFO:teuthology.orchestra.run.smithi102.stdout: "up_thru": 14, 2024-04-17T22:31:21.292 INFO:teuthology.orchestra.run.smithi102.stdout: "down_at": 0, 2024-04-17T22:31:21.292 INFO:teuthology.orchestra.run.smithi102.stdout: "lost_at": 0, 2024-04-17T22:31:21.292 INFO:teuthology.orchestra.run.smithi102.stdout: "public_addrs": { 2024-04-17T22:31:21.292 INFO:teuthology.orchestra.run.smithi102.stdout: "addrvec": [ 2024-04-17T22:31:21.292 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-04-17T22:31:21.292 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "v1", 2024-04-17T22:31:21.292 INFO:teuthology.orchestra.run.smithi102.stdout: "addr": "172.21.15.102:6800", 2024-04-17T22:31:21.292 INFO:teuthology.orchestra.run.smithi102.stdout: "nonce": 42953 2024-04-17T22:31:21.292 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-17T22:31:21.292 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-04-17T22:31:21.292 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-17T22:31:21.292 INFO:teuthology.orchestra.run.smithi102.stdout: "cluster_addrs": { 2024-04-17T22:31:21.292 INFO:teuthology.orchestra.run.smithi102.stdout: "addrvec": [ 2024-04-17T22:31:21.293 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-04-17T22:31:21.293 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "v1", 2024-04-17T22:31:21.293 INFO:teuthology.orchestra.run.smithi102.stdout: "addr": "172.21.15.102:6801", 2024-04-17T22:31:21.293 INFO:teuthology.orchestra.run.smithi102.stdout: "nonce": 42953 2024-04-17T22:31:21.293 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-17T22:31:21.293 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-04-17T22:31:21.293 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-17T22:31:21.293 INFO:teuthology.orchestra.run.smithi102.stdout: "heartbeat_back_addrs": { 2024-04-17T22:31:21.293 INFO:teuthology.orchestra.run.smithi102.stdout: "addrvec": [ 2024-04-17T22:31:21.293 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-04-17T22:31:21.293 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "v1", 2024-04-17T22:31:21.293 INFO:teuthology.orchestra.run.smithi102.stdout: "addr": "172.21.15.102:6803", 2024-04-17T22:31:21.293 INFO:teuthology.orchestra.run.smithi102.stdout: "nonce": 42953 2024-04-17T22:31:21.293 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-17T22:31:21.294 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-04-17T22:31:21.294 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-17T22:31:21.294 INFO:teuthology.orchestra.run.smithi102.stdout: "heartbeat_front_addrs": { 2024-04-17T22:31:21.294 INFO:teuthology.orchestra.run.smithi102.stdout: "addrvec": [ 2024-04-17T22:31:21.294 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-04-17T22:31:21.294 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "v1", 2024-04-17T22:31:21.294 INFO:teuthology.orchestra.run.smithi102.stdout: "addr": "172.21.15.102:6802", 2024-04-17T22:31:21.294 INFO:teuthology.orchestra.run.smithi102.stdout: "nonce": 42953 2024-04-17T22:31:21.294 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-17T22:31:21.294 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-04-17T22:31:21.294 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-17T22:31:21.294 INFO:teuthology.orchestra.run.smithi102.stdout: "public_addr": "172.21.15.102:6800/42953", 2024-04-17T22:31:21.294 INFO:teuthology.orchestra.run.smithi102.stdout: "cluster_addr": "172.21.15.102:6801/42953", 2024-04-17T22:31:21.294 INFO:teuthology.orchestra.run.smithi102.stdout: "heartbeat_back_addr": "172.21.15.102:6803/42953", 2024-04-17T22:31:21.295 INFO:teuthology.orchestra.run.smithi102.stdout: "heartbeat_front_addr": "172.21.15.102:6802/42953", 2024-04-17T22:31:21.295 INFO:teuthology.orchestra.run.smithi102.stdout: "state": [ 2024-04-17T22:31:21.295 INFO:teuthology.orchestra.run.smithi102.stdout: "exists", 2024-04-17T22:31:21.295 INFO:teuthology.orchestra.run.smithi102.stdout: "up" 2024-04-17T22:31:21.295 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-04-17T22:31:21.295 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-17T22:31:21.295 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-17T22:31:21.295 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_xinfo": [ 2024-04-17T22:31:21.295 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-04-17T22:31:21.295 INFO:teuthology.orchestra.run.smithi102.stdout: "osd": 0, 2024-04-17T22:31:21.295 INFO:teuthology.orchestra.run.smithi102.stdout: "down_stamp": "0.000000", 2024-04-17T22:31:21.295 INFO:teuthology.orchestra.run.smithi102.stdout: "laggy_probability": 0, 2024-04-17T22:31:21.295 INFO:teuthology.orchestra.run.smithi102.stdout: "laggy_interval": 0, 2024-04-17T22:31:21.295 INFO:teuthology.orchestra.run.smithi102.stdout: "features": 4611087854035861503, 2024-04-17T22:31:21.295 INFO:teuthology.orchestra.run.smithi102.stdout: "old_weight": 0 2024-04-17T22:31:21.296 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-17T22:31:21.296 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-04-17T22:31:21.296 INFO:teuthology.orchestra.run.smithi102.stdout: "osd": 1, 2024-04-17T22:31:21.296 INFO:teuthology.orchestra.run.smithi102.stdout: "down_stamp": "0.000000", 2024-04-17T22:31:21.296 INFO:teuthology.orchestra.run.smithi102.stdout: "laggy_probability": 0, 2024-04-17T22:31:21.296 INFO:teuthology.orchestra.run.smithi102.stdout: "laggy_interval": 0, 2024-04-17T22:31:21.296 INFO:teuthology.orchestra.run.smithi102.stdout: "features": 4611087854035861503, 2024-04-17T22:31:21.296 INFO:teuthology.orchestra.run.smithi102.stdout: "old_weight": 0 2024-04-17T22:31:21.296 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-17T22:31:21.296 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-04-17T22:31:21.296 INFO:teuthology.orchestra.run.smithi102.stdout: "osd": 2, 2024-04-17T22:31:21.296 INFO:teuthology.orchestra.run.smithi102.stdout: "down_stamp": "0.000000", 2024-04-17T22:31:21.296 INFO:teuthology.orchestra.run.smithi102.stdout: "laggy_probability": 0, 2024-04-17T22:31:21.296 INFO:teuthology.orchestra.run.smithi102.stdout: "laggy_interval": 0, 2024-04-17T22:31:21.297 INFO:teuthology.orchestra.run.smithi102.stdout: "features": 4611087854035861503, 2024-04-17T22:31:21.297 INFO:teuthology.orchestra.run.smithi102.stdout: "old_weight": 0 2024-04-17T22:31:21.297 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-17T22:31:21.297 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-04-17T22:31:21.297 INFO:teuthology.orchestra.run.smithi102.stdout: "osd": 3, 2024-04-17T22:31:21.297 INFO:teuthology.orchestra.run.smithi102.stdout: "down_stamp": "0.000000", 2024-04-17T22:31:21.297 INFO:teuthology.orchestra.run.smithi102.stdout: "laggy_probability": 0, 2024-04-17T22:31:21.297 INFO:teuthology.orchestra.run.smithi102.stdout: "laggy_interval": 0, 2024-04-17T22:31:21.297 INFO:teuthology.orchestra.run.smithi102.stdout: "features": 4611087854035861503, 2024-04-17T22:31:21.297 INFO:teuthology.orchestra.run.smithi102.stdout: "old_weight": 0 2024-04-17T22:31:21.297 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-17T22:31:21.297 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-17T22:31:21.297 INFO:teuthology.orchestra.run.smithi102.stdout: "pg_upmap": [], 2024-04-17T22:31:21.297 INFO:teuthology.orchestra.run.smithi102.stdout: "pg_upmap_items": [], 2024-04-17T22:31:21.298 INFO:teuthology.orchestra.run.smithi102.stdout: "pg_temp": [], 2024-04-17T22:31:21.298 INFO:teuthology.orchestra.run.smithi102.stdout: "primary_temp": [], 2024-04-17T22:31:21.298 INFO:teuthology.orchestra.run.smithi102.stdout: "blacklist": { 2024-04-17T22:31:21.298 INFO:teuthology.orchestra.run.smithi102.stdout: "172.21.15.102:6817/1944542598": "2024-04-18 22:31:16.601994" 2024-04-17T22:31:21.298 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-17T22:31:21.298 INFO:teuthology.orchestra.run.smithi102.stdout: "erasure_code_profiles": { 2024-04-17T22:31:21.298 INFO:teuthology.orchestra.run.smithi102.stdout: "default": { 2024-04-17T22:31:21.298 INFO:teuthology.orchestra.run.smithi102.stdout: "crush-failure-domain": "osd", 2024-04-17T22:31:21.298 INFO:teuthology.orchestra.run.smithi102.stdout: "k": "2", 2024-04-17T22:31:21.298 INFO:teuthology.orchestra.run.smithi102.stdout: "m": "1", 2024-04-17T22:31:21.298 INFO:teuthology.orchestra.run.smithi102.stdout: "plugin": "jerasure", 2024-04-17T22:31:21.298 INFO:teuthology.orchestra.run.smithi102.stdout: "technique": "reed_sol_van" 2024-04-17T22:31:21.298 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-17T22:31:21.298 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-17T22:31:21.299 INFO:teuthology.orchestra.run.smithi102.stdout: "removed_snaps_queue": [], 2024-04-17T22:31:21.299 INFO:teuthology.orchestra.run.smithi102.stdout: "new_removed_snaps": [], 2024-04-17T22:31:21.299 INFO:teuthology.orchestra.run.smithi102.stdout: "new_purged_snaps": [], 2024-04-17T22:31:21.299 INFO:teuthology.orchestra.run.smithi102.stdout: "crush_node_flags": {}, 2024-04-17T22:31:21.299 INFO:teuthology.orchestra.run.smithi102.stdout: "device_class_flags": {} 2024-04-17T22:31:21.299 INFO:teuthology.orchestra.run.smithi102.stdout:} 2024-04-17T22:31:21.300 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.287 7f3cedb48700 1 -- 172.21.15.102:0/1692663216 >> v1:172.21.15.102:6816/42360 conn(0x7f3ccc0145a0 legacy=0x7f3ccc016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:21.300 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.287 7f3cedb48700 1 -- 172.21.15.102:0/1692663216 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f3ce80f21c0 msgr2=0x7f3ce81ad780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:21.300 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.287 7f3cedb48700 1 --2- 172.21.15.102:0/1692663216 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f3ce80f21c0 0x7f3ce81ad780 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 rx=0x7f3ce80c8410 tx=0x7f3cd00068c0).stop 2024-04-17T22:31:21.301 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.287 7f3cedb48700 1 -- 172.21.15.102:0/1692663216 shutdown_connections 2024-04-17T22:31:21.301 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.287 7f3cedb48700 1 --2- 172.21.15.102:0/1692663216 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f3ce81addb0 0x7f3ce81bba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:21.301 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.287 7f3cedb48700 1 --2- 172.21.15.102:0/1692663216 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3ce80f1740 0x7f3ce80f1b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:21.301 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.287 7f3cedb48700 1 --2- 172.21.15.102:0/1692663216 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f3ce80f21c0 0x7f3ce81ad780 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:21.301 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.287 7f3cedb48700 1 -- 172.21.15.102:0/1692663216 >> 172.21.15.102:0/1692663216 conn(0x7f3ce80c5850 msgr2=0x7f3ce80cedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:21.301 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.287 7f3cedb48700 1 -- 172.21.15.102:0/1692663216 shutdown_connections 2024-04-17T22:31:21.301 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.287 7f3cedb48700 1 -- 172.21.15.102:0/1692663216 wait complete. 2024-04-17T22:31:21.301 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2024-04-17T22:31:21.428 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.441 7f564b9b1700 1 Processor -- start 2024-04-17T22:31:21.428 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.441 7f564b9b1700 1 -- start start 2024-04-17T22:31:21.428 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.441 7f564b9b1700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f564410e260 0x7f564410e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:21.428 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.441 7f564b9b1700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f564410ece0 0x7f56440f1750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:21.428 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.441 7f564b9b1700 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56440f21d0 con 0x7f56440f1d80 2024-04-17T22:31:21.428 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.441 7f564b9b1700 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56440f9f30 con 0x7f564410ece0 2024-04-17T22:31:21.429 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.441 7f564b9b1700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56440f9050 con 0x7f564410e260 2024-04-17T22:31:21.429 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.441 7f563bfff700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f564410ece0 0x7f56440f1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:21.429 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.441 7f5643fff700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f564410e260 0x7f564410e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:21.429 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.441 7f5643fff700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f564410e260 0x7f564410e6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:58964/0 (socket says 172.21.15.102:58964) 2024-04-17T22:31:21.429 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.441 7f563bfff700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f564410ece0 0x7f56440f1750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:44232/0 (socket says 172.21.15.102:44232) 2024-04-17T22:31:21.429 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.441 7f5643fff700 1 -- 172.21.15.102:0/1971488795 learned_addr learned my addr 172.21.15.102:0/1971488795 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:31:21.429 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.441 7f5648fad700 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f56440f1d80 0x7f56441aa4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:36460/0 (socket says 172.21.15.102:36460) 2024-04-17T22:31:21.429 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.442 7f563bfff700 1 -- 172.21.15.102:0/1971488795 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f564410e260 msgr2=0x7f564410e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:31:21.429 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.442 7f563bfff700 1 --2- 172.21.15.102:0/1971488795 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f564410e260 0x7f564410e6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:21.429 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.442 7f563bfff700 1 -- 172.21.15.102:0/1971488795 >> v1:172.21.15.102:6789/0 conn(0x7f56440f1d80 legacy=0x7f56441aa4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:21.429 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.442 7f56437fe700 1 -- 172.21.15.102:0/1971488795 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3648075529 0 0) 0x7f563c001490 con 0x7f56440f1d80 2024-04-17T22:31:21.429 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.442 7f563bfff700 1 -- 172.21.15.102:0/1971488795 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56440f23f0 con 0x7f564410ece0 2024-04-17T22:31:21.429 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.442 7f5643fff700 1 --2- 172.21.15.102:0/1971488795 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f564410e260 0x7f564410e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:21.430 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.442 7f563bfff700 1 --2- 172.21.15.102:0/1971488795 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f564410ece0 0x7f56440f1750 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 rx=0x7f56441035a0 tx=0x7f562c00ba20).ready entity=mon.0 client_cookie=49b45808972f1e28 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:21.430 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.442 7f56437fe700 1 -- 172.21.15.102:0/1971488795 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f56440f23f0 con 0x7f564410ece0 2024-04-17T22:31:21.430 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.442 7f56437fe700 1 -- 172.21.15.102:0/1971488795 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f562c00d5c0 con 0x7f564410ece0 2024-04-17T22:31:21.430 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.442 7f56437fe700 1 -- 172.21.15.102:0/1971488795 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f562c00d9e0 con 0x7f564410ece0 2024-04-17T22:31:21.430 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.442 7f564b9b1700 1 -- 172.21.15.102:0/1971488795 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f564410ece0 msgr2=0x7f56440f1750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:21.430 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.442 7f564b9b1700 1 --2- 172.21.15.102:0/1971488795 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f564410ece0 0x7f56440f1750 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 rx=0x7f56441035a0 tx=0x7f562c00ba20).stop 2024-04-17T22:31:21.430 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.442 7f564b9b1700 1 -- 172.21.15.102:0/1971488795 shutdown_connections 2024-04-17T22:31:21.430 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.442 7f564b9b1700 1 --2- 172.21.15.102:0/1971488795 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f564410e260 0x7f564410e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:21.430 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.442 7f564b9b1700 1 --2- 172.21.15.102:0/1971488795 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f564410ece0 0x7f56440f1750 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:21.430 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.442 7f564b9b1700 1 -- 172.21.15.102:0/1971488795 >> 172.21.15.102:0/1971488795 conn(0x7f56441138c0 msgr2=0x7f56440f4f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:21.430 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.443 7f564b9b1700 1 -- 172.21.15.102:0/1971488795 shutdown_connections 2024-04-17T22:31:21.430 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.443 7f564b9b1700 1 -- 172.21.15.102:0/1971488795 wait complete. 2024-04-17T22:31:21.450 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.465 7f564b9b1700 1 Processor -- start 2024-04-17T22:31:21.450 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.466 7f564b9b1700 1 -- start start 2024-04-17T22:31:21.453 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.466 7f564b9b1700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f56440cf4c0 0x7f56440d5f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:21.453 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.466 7f564b9b1700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f56440d6570 0x7f56441b3eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:21.453 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.466 7f564b9b1700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f56440d69c0 0x7f56441b44e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:21.453 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.466 7f564b9b1700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56440fa1d0 con 0x7f56440cf4c0 2024-04-17T22:31:21.453 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.466 7f564b9b1700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56440cd3f0 con 0x7f56440d6570 2024-04-17T22:31:21.453 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.466 7f564b9b1700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f56440cf910 con 0x7f56440d69c0 2024-04-17T22:31:21.453 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.466 7f5643fff700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f56440cf4c0 0x7f56440d5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:21.453 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.466 7f563bfff700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f56440d6570 0x7f56441b3eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:21.453 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.466 7f5643fff700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f56440cf4c0 0x7f56440d5f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:44234/0 (socket says 172.21.15.102:44234) 2024-04-17T22:31:21.453 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.466 7f5643fff700 1 -- 172.21.15.102:0/750384294 learned_addr learned my addr 172.21.15.102:0/750384294 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:31:21.453 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.466 7f5648fad700 1 --2- 172.21.15.102:0/750384294 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f56440d69c0 0x7f56441b44e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:21.454 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.466 7f563bfff700 1 -- 172.21.15.102:0/750384294 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f56440d69c0 msgr2=0x7f56441b44e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:21.454 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.466 7f563bfff700 1 --2- 172.21.15.102:0/750384294 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f56440d69c0 0x7f56441b44e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:21.454 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.466 7f563bfff700 1 -- 172.21.15.102:0/750384294 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f56440cf4c0 msgr2=0x7f56440d5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:21.454 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.466 7f563bfff700 1 --2- 172.21.15.102:0/750384294 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f56440cf4c0 0x7f56440d5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:21.454 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.466 7f563bfff700 1 -- 172.21.15.102:0/750384294 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56441b4b10 con 0x7f56440d6570 2024-04-17T22:31:21.454 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.466 7f5643fff700 1 --2- 172.21.15.102:0/750384294 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f56440cf4c0 0x7f56440d5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:21.454 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.466 7f5648fad700 1 --2- 172.21.15.102:0/750384294 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f56440d69c0 0x7f56441b44e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:21.454 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.467 7f563bfff700 1 --2- 172.21.15.102:0/750384294 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f56440d6570 0x7f56441b3eb0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 rx=0x7f56440d73e0 tx=0x7f562c0068c0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:21.454 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.467 7f5641ffb700 1 -- 172.21.15.102:0/750384294 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f56441b4b10 con 0x7f56440d6570 2024-04-17T22:31:21.454 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.467 7f5641ffb700 1 -- 172.21.15.102:0/750384294 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f562c011940 con 0x7f56440d6570 2024-04-17T22:31:21.454 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.467 7f564b9b1700 1 -- 172.21.15.102:0/750384294 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56441b4dd0 con 0x7f56440d6570 2024-04-17T22:31:21.454 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.467 7f5641ffb700 1 -- 172.21.15.102:0/750384294 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f562c017040 con 0x7f56440d6570 2024-04-17T22:31:21.454 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.467 7f564b9b1700 1 -- 172.21.15.102:0/750384294 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56441bb800 con 0x7f56440d6570 2024-04-17T22:31:21.454 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.467 7f5641ffb700 1 -- 172.21.15.102:0/750384294 <== mon.1 v2:172.21.15.102:3301/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7f562c011d10 con 0x7f56440d6570 2024-04-17T22:31:21.455 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.467 7f5641ffb700 1 -- 172.21.15.102:0/750384294 <== mon.1 v2:172.21.15.102:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 2417+0+0 (secure 0 0 0) 0x7f562c018070 con 0x7f56440d6570 2024-04-17T22:31:21.455 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.468 7f564b9b1700 1 -- 172.21.15.102:0/750384294 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56440f0820 con 0x7f56440d6570 2024-04-17T22:31:21.456 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.471 7f5641ffb700 1 -- 172.21.15.102:0/750384294 <== mon.1 v2:172.21.15.102:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+110455 (secure 0 0 0) 0x7f562c011730 con 0x7f56440d6570 2024-04-17T22:31:21.561 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.576 7f564b9b1700 1 -- 172.21.15.102:0/750384294 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f56441ae5f0 con 0x7f56440d6570 2024-04-17T22:31:21.561 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.576 7f5641ffb700 1 -- 172.21.15.102:0/750384294 <== mon.1 v2:172.21.15.102:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v20) v1 ==== 81+0+11848 (secure 0 0 0) 0x7f562c00d3b0 con 0x7f56440d6570 2024-04-17T22:31:21.562 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-17T22:31:21.562 INFO:teuthology.orchestra.run.smithi102.stdout:{ 2024-04-17T22:31:21.562 INFO:teuthology.orchestra.run.smithi102.stdout: "epoch": 20, 2024-04-17T22:31:21.562 INFO:teuthology.orchestra.run.smithi102.stdout: "fsid": "cb63e789-091d-49b1-b120-55f9db526871", 2024-04-17T22:31:21.562 INFO:teuthology.orchestra.run.smithi102.stdout: "created": "2024-04-17 22:28:48.209635", 2024-04-17T22:31:21.562 INFO:teuthology.orchestra.run.smithi102.stdout: "modified": "2024-04-17 22:31:20.613707", 2024-04-17T22:31:21.562 INFO:teuthology.orchestra.run.smithi102.stdout: "last_up_change": "2024-04-17 22:28:54.236215", 2024-04-17T22:31:21.562 INFO:teuthology.orchestra.run.smithi102.stdout: "last_in_change": "2024-04-17 22:28:54.236215", 2024-04-17T22:31:21.562 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-04-17T22:31:21.562 INFO:teuthology.orchestra.run.smithi102.stdout: "flags_num": 5799936, 2024-04-17T22:31:21.562 INFO:teuthology.orchestra.run.smithi102.stdout: "flags_set": [ 2024-04-17T22:31:21.562 INFO:teuthology.orchestra.run.smithi102.stdout: "pglog_hardlimit", 2024-04-17T22:31:21.563 INFO:teuthology.orchestra.run.smithi102.stdout: "purged_snapdirs", 2024-04-17T22:31:21.563 INFO:teuthology.orchestra.run.smithi102.stdout: "recovery_deletes", 2024-04-17T22:31:21.563 INFO:teuthology.orchestra.run.smithi102.stdout: "sortbitwise" 2024-04-17T22:31:21.563 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-17T22:31:21.563 INFO:teuthology.orchestra.run.smithi102.stdout: "crush_version": 4, 2024-04-17T22:31:21.563 INFO:teuthology.orchestra.run.smithi102.stdout: "full_ratio": 0.94999998807907104, 2024-04-17T22:31:21.563 INFO:teuthology.orchestra.run.smithi102.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-04-17T22:31:21.563 INFO:teuthology.orchestra.run.smithi102.stdout: "nearfull_ratio": 0.85000002384185791, 2024-04-17T22:31:21.563 INFO:teuthology.orchestra.run.smithi102.stdout: "cluster_snapshot": "", 2024-04-17T22:31:21.563 INFO:teuthology.orchestra.run.smithi102.stdout: "pool_max": 3, 2024-04-17T22:31:21.563 INFO:teuthology.orchestra.run.smithi102.stdout: "max_osd": 4, 2024-04-17T22:31:21.563 INFO:teuthology.orchestra.run.smithi102.stdout: "require_min_compat_client": "nautilus", 2024-04-17T22:31:21.563 INFO:teuthology.orchestra.run.smithi102.stdout: "min_compat_client": "jewel", 2024-04-17T22:31:21.563 INFO:teuthology.orchestra.run.smithi102.stdout: "require_osd_release": "nautilus", 2024-04-17T22:31:21.563 INFO:teuthology.orchestra.run.smithi102.stdout: "pools": [ 2024-04-17T22:31:21.564 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-04-17T22:31:21.564 INFO:teuthology.orchestra.run.smithi102.stdout: "pool": 1, 2024-04-17T22:31:21.564 INFO:teuthology.orchestra.run.smithi102.stdout: "pool_name": "rbd", 2024-04-17T22:31:21.564 INFO:teuthology.orchestra.run.smithi102.stdout: "create_time": "2024-04-17 22:28:57.478545", 2024-04-17T22:31:21.564 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-17T22:31:21.564 INFO:teuthology.orchestra.run.smithi102.stdout: "flags_names": "hashpspool", 2024-04-17T22:31:21.564 INFO:teuthology.orchestra.run.smithi102.stdout: "type": 1, 2024-04-17T22:31:21.564 INFO:teuthology.orchestra.run.smithi102.stdout: "size": 2, 2024-04-17T22:31:21.564 INFO:teuthology.orchestra.run.smithi102.stdout: "min_size": 1, 2024-04-17T22:31:21.564 INFO:teuthology.orchestra.run.smithi102.stdout: "crush_rule": 0, 2024-04-17T22:31:21.564 INFO:teuthology.orchestra.run.smithi102.stdout: "object_hash": 2, 2024-04-17T22:31:21.564 INFO:teuthology.orchestra.run.smithi102.stdout: "pg_autoscale_mode": "off", 2024-04-17T22:31:21.564 INFO:teuthology.orchestra.run.smithi102.stdout: "pg_num": 8, 2024-04-17T22:31:21.564 INFO:teuthology.orchestra.run.smithi102.stdout: "pg_placement_num": 8, 2024-04-17T22:31:21.565 INFO:teuthology.orchestra.run.smithi102.stdout: "pg_placement_num_target": 8, 2024-04-17T22:31:21.565 INFO:teuthology.orchestra.run.smithi102.stdout: "pg_num_target": 8, 2024-04-17T22:31:21.565 INFO:teuthology.orchestra.run.smithi102.stdout: "pg_num_pending": 8, 2024-04-17T22:31:21.565 INFO:teuthology.orchestra.run.smithi102.stdout: "last_pg_merge_meta": { 2024-04-17T22:31:21.565 INFO:teuthology.orchestra.run.smithi102.stdout: "source_pgid": "0.0", 2024-04-17T22:31:21.565 INFO:teuthology.orchestra.run.smithi102.stdout: "ready_epoch": 0, 2024-04-17T22:31:21.565 INFO:teuthology.orchestra.run.smithi102.stdout: "last_epoch_started": 0, 2024-04-17T22:31:21.565 INFO:teuthology.orchestra.run.smithi102.stdout: "last_epoch_clean": 0, 2024-04-17T22:31:21.565 INFO:teuthology.orchestra.run.smithi102.stdout: "source_version": "0'0", 2024-04-17T22:31:21.565 INFO:teuthology.orchestra.run.smithi102.stdout: "target_version": "0'0" 2024-04-17T22:31:21.565 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-17T22:31:21.565 INFO:teuthology.orchestra.run.smithi102.stdout: "last_change": "12", 2024-04-17T22:31:21.565 INFO:teuthology.orchestra.run.smithi102.stdout: "last_force_op_resend": "0", 2024-04-17T22:31:21.565 INFO:teuthology.orchestra.run.smithi102.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-17T22:31:21.565 INFO:teuthology.orchestra.run.smithi102.stdout: "last_force_op_resend_preluminous": "0", 2024-04-17T22:31:21.566 INFO:teuthology.orchestra.run.smithi102.stdout: "auid": 0, 2024-04-17T22:31:21.566 INFO:teuthology.orchestra.run.smithi102.stdout: "snap_mode": "selfmanaged", 2024-04-17T22:31:21.566 INFO:teuthology.orchestra.run.smithi102.stdout: "snap_seq": 0, 2024-04-17T22:31:21.566 INFO:teuthology.orchestra.run.smithi102.stdout: "snap_epoch": 0, 2024-04-17T22:31:21.566 INFO:teuthology.orchestra.run.smithi102.stdout: "pool_snaps": [], 2024-04-17T22:31:21.566 INFO:teuthology.orchestra.run.smithi102.stdout: "removed_snaps": "[]", 2024-04-17T22:31:21.566 INFO:teuthology.orchestra.run.smithi102.stdout: "quota_max_bytes": 0, 2024-04-17T22:31:21.566 INFO:teuthology.orchestra.run.smithi102.stdout: "quota_max_objects": 0, 2024-04-17T22:31:21.566 INFO:teuthology.orchestra.run.smithi102.stdout: "tiers": [], 2024-04-17T22:31:21.566 INFO:teuthology.orchestra.run.smithi102.stdout: "tier_of": -1, 2024-04-17T22:31:21.566 INFO:teuthology.orchestra.run.smithi102.stdout: "read_tier": -1, 2024-04-17T22:31:21.566 INFO:teuthology.orchestra.run.smithi102.stdout: "write_tier": -1, 2024-04-17T22:31:21.566 INFO:teuthology.orchestra.run.smithi102.stdout: "cache_mode": "none", 2024-04-17T22:31:21.566 INFO:teuthology.orchestra.run.smithi102.stdout: "target_max_bytes": 0, 2024-04-17T22:31:21.567 INFO:teuthology.orchestra.run.smithi102.stdout: "target_max_objects": 0, 2024-04-17T22:31:21.567 INFO:teuthology.orchestra.run.smithi102.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-17T22:31:21.567 INFO:teuthology.orchestra.run.smithi102.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-17T22:31:21.567 INFO:teuthology.orchestra.run.smithi102.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-17T22:31:21.567 INFO:teuthology.orchestra.run.smithi102.stdout: "cache_min_flush_age": 0, 2024-04-17T22:31:21.567 INFO:teuthology.orchestra.run.smithi102.stdout: "cache_min_evict_age": 0, 2024-04-17T22:31:21.567 INFO:teuthology.orchestra.run.smithi102.stdout: "erasure_code_profile": "", 2024-04-17T22:31:21.567 INFO:teuthology.orchestra.run.smithi102.stdout: "hit_set_params": { 2024-04-17T22:31:21.567 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "none" 2024-04-17T22:31:21.567 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-17T22:31:21.567 INFO:teuthology.orchestra.run.smithi102.stdout: "hit_set_period": 0, 2024-04-17T22:31:21.567 INFO:teuthology.orchestra.run.smithi102.stdout: "hit_set_count": 0, 2024-04-17T22:31:21.567 INFO:teuthology.orchestra.run.smithi102.stdout: "use_gmt_hitset": true, 2024-04-17T22:31:21.567 INFO:teuthology.orchestra.run.smithi102.stdout: "min_read_recency_for_promote": 0, 2024-04-17T22:31:21.567 INFO:teuthology.orchestra.run.smithi102.stdout: "min_write_recency_for_promote": 0, 2024-04-17T22:31:21.567 INFO:teuthology.orchestra.run.smithi102.stdout: "hit_set_grade_decay_rate": 0, 2024-04-17T22:31:21.568 INFO:teuthology.orchestra.run.smithi102.stdout: "hit_set_search_last_n": 0, 2024-04-17T22:31:21.568 INFO:teuthology.orchestra.run.smithi102.stdout: "grade_table": [], 2024-04-17T22:31:21.568 INFO:teuthology.orchestra.run.smithi102.stdout: "stripe_width": 0, 2024-04-17T22:31:21.568 INFO:teuthology.orchestra.run.smithi102.stdout: "expected_num_objects": 0, 2024-04-17T22:31:21.568 INFO:teuthology.orchestra.run.smithi102.stdout: "fast_read": false, 2024-04-17T22:31:21.568 INFO:teuthology.orchestra.run.smithi102.stdout: "options": {}, 2024-04-17T22:31:21.568 INFO:teuthology.orchestra.run.smithi102.stdout: "application_metadata": { 2024-04-17T22:31:21.568 INFO:teuthology.orchestra.run.smithi102.stdout: "rbd": {} 2024-04-17T22:31:21.568 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-17T22:31:21.568 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-17T22:31:21.568 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-17T22:31:21.568 INFO:teuthology.orchestra.run.smithi102.stdout: "osds": [ 2024-04-17T22:31:21.568 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-04-17T22:31:21.568 INFO:teuthology.orchestra.run.smithi102.stdout: "osd": 0, 2024-04-17T22:31:21.568 INFO:teuthology.orchestra.run.smithi102.stdout: "uuid": "3351ef85-ade4-4b90-a8f4-213f4e00ba01", 2024-04-17T22:31:21.569 INFO:teuthology.orchestra.run.smithi102.stdout: "up": 1, 2024-04-17T22:31:21.569 INFO:teuthology.orchestra.run.smithi102.stdout: "in": 1, 2024-04-17T22:31:21.569 INFO:teuthology.orchestra.run.smithi102.stdout: "weight": 1, 2024-04-17T22:31:21.569 INFO:teuthology.orchestra.run.smithi102.stdout: "primary_affinity": 1, 2024-04-17T22:31:21.569 INFO:teuthology.orchestra.run.smithi102.stdout: "last_clean_begin": 0, 2024-04-17T22:31:21.569 INFO:teuthology.orchestra.run.smithi102.stdout: "last_clean_end": 0, 2024-04-17T22:31:21.569 INFO:teuthology.orchestra.run.smithi102.stdout: "up_from": 9, 2024-04-17T22:31:21.569 INFO:teuthology.orchestra.run.smithi102.stdout: "up_thru": 14, 2024-04-17T22:31:21.569 INFO:teuthology.orchestra.run.smithi102.stdout: "down_at": 0, 2024-04-17T22:31:21.569 INFO:teuthology.orchestra.run.smithi102.stdout: "lost_at": 0, 2024-04-17T22:31:21.569 INFO:teuthology.orchestra.run.smithi102.stdout: "public_addrs": { 2024-04-17T22:31:21.569 INFO:teuthology.orchestra.run.smithi102.stdout: "addrvec": [ 2024-04-17T22:31:21.569 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-04-17T22:31:21.569 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "v1", 2024-04-17T22:31:21.569 INFO:teuthology.orchestra.run.smithi102.stdout: "addr": "172.21.15.102:6812", 2024-04-17T22:31:21.570 INFO:teuthology.orchestra.run.smithi102.stdout: "nonce": 42955 2024-04-17T22:31:21.570 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-17T22:31:21.570 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-04-17T22:31:21.570 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-17T22:31:21.570 INFO:teuthology.orchestra.run.smithi102.stdout: "cluster_addrs": { 2024-04-17T22:31:21.570 INFO:teuthology.orchestra.run.smithi102.stdout: "addrvec": [ 2024-04-17T22:31:21.570 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-04-17T22:31:21.570 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "v1", 2024-04-17T22:31:21.570 INFO:teuthology.orchestra.run.smithi102.stdout: "addr": "172.21.15.102:6813", 2024-04-17T22:31:21.570 INFO:teuthology.orchestra.run.smithi102.stdout: "nonce": 42955 2024-04-17T22:31:21.570 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-17T22:31:21.570 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-04-17T22:31:21.570 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-17T22:31:21.570 INFO:teuthology.orchestra.run.smithi102.stdout: "heartbeat_back_addrs": { 2024-04-17T22:31:21.571 INFO:teuthology.orchestra.run.smithi102.stdout: "addrvec": [ 2024-04-17T22:31:21.571 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-04-17T22:31:21.571 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "v1", 2024-04-17T22:31:21.571 INFO:teuthology.orchestra.run.smithi102.stdout: "addr": "172.21.15.102:6815", 2024-04-17T22:31:21.571 INFO:teuthology.orchestra.run.smithi102.stdout: "nonce": 42955 2024-04-17T22:31:21.571 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-17T22:31:21.571 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-04-17T22:31:21.571 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-17T22:31:21.571 INFO:teuthology.orchestra.run.smithi102.stdout: "heartbeat_front_addrs": { 2024-04-17T22:31:21.571 INFO:teuthology.orchestra.run.smithi102.stdout: "addrvec": [ 2024-04-17T22:31:21.571 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-04-17T22:31:21.571 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "v1", 2024-04-17T22:31:21.571 INFO:teuthology.orchestra.run.smithi102.stdout: "addr": "172.21.15.102:6814", 2024-04-17T22:31:21.571 INFO:teuthology.orchestra.run.smithi102.stdout: "nonce": 42955 2024-04-17T22:31:21.571 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-17T22:31:21.572 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-04-17T22:31:21.572 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-17T22:31:21.572 INFO:teuthology.orchestra.run.smithi102.stdout: "public_addr": "172.21.15.102:6812/42955", 2024-04-17T22:31:21.572 INFO:teuthology.orchestra.run.smithi102.stdout: "cluster_addr": "172.21.15.102:6813/42955", 2024-04-17T22:31:21.572 INFO:teuthology.orchestra.run.smithi102.stdout: "heartbeat_back_addr": "172.21.15.102:6815/42955", 2024-04-17T22:31:21.572 INFO:teuthology.orchestra.run.smithi102.stdout: "heartbeat_front_addr": "172.21.15.102:6814/42955", 2024-04-17T22:31:21.572 INFO:teuthology.orchestra.run.smithi102.stdout: "state": [ 2024-04-17T22:31:21.572 INFO:teuthology.orchestra.run.smithi102.stdout: "exists", 2024-04-17T22:31:21.572 INFO:teuthology.orchestra.run.smithi102.stdout: "up" 2024-04-17T22:31:21.572 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-04-17T22:31:21.572 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-17T22:31:21.572 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-04-17T22:31:21.572 INFO:teuthology.orchestra.run.smithi102.stdout: "osd": 1, 2024-04-17T22:31:21.572 INFO:teuthology.orchestra.run.smithi102.stdout: "uuid": "04f67eac-f36b-48f5-982d-4cfc776596d0", 2024-04-17T22:31:21.572 INFO:teuthology.orchestra.run.smithi102.stdout: "up": 1, 2024-04-17T22:31:21.573 INFO:teuthology.orchestra.run.smithi102.stdout: "in": 1, 2024-04-17T22:31:21.573 INFO:teuthology.orchestra.run.smithi102.stdout: "weight": 1, 2024-04-17T22:31:21.573 INFO:teuthology.orchestra.run.smithi102.stdout: "primary_affinity": 1, 2024-04-17T22:31:21.573 INFO:teuthology.orchestra.run.smithi102.stdout: "last_clean_begin": 0, 2024-04-17T22:31:21.573 INFO:teuthology.orchestra.run.smithi102.stdout: "last_clean_end": 0, 2024-04-17T22:31:21.573 INFO:teuthology.orchestra.run.smithi102.stdout: "up_from": 9, 2024-04-17T22:31:21.573 INFO:teuthology.orchestra.run.smithi102.stdout: "up_thru": 14, 2024-04-17T22:31:21.573 INFO:teuthology.orchestra.run.smithi102.stdout: "down_at": 0, 2024-04-17T22:31:21.573 INFO:teuthology.orchestra.run.smithi102.stdout: "lost_at": 0, 2024-04-17T22:31:21.573 INFO:teuthology.orchestra.run.smithi102.stdout: "public_addrs": { 2024-04-17T22:31:21.573 INFO:teuthology.orchestra.run.smithi102.stdout: "addrvec": [ 2024-04-17T22:31:21.573 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-04-17T22:31:21.573 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "v1", 2024-04-17T22:31:21.573 INFO:teuthology.orchestra.run.smithi102.stdout: "addr": "172.21.15.102:6808", 2024-04-17T22:31:21.573 INFO:teuthology.orchestra.run.smithi102.stdout: "nonce": 42958 2024-04-17T22:31:21.574 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-17T22:31:21.574 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-04-17T22:31:21.574 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-17T22:31:21.574 INFO:teuthology.orchestra.run.smithi102.stdout: "cluster_addrs": { 2024-04-17T22:31:21.574 INFO:teuthology.orchestra.run.smithi102.stdout: "addrvec": [ 2024-04-17T22:31:21.574 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-04-17T22:31:21.574 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "v1", 2024-04-17T22:31:21.574 INFO:teuthology.orchestra.run.smithi102.stdout: "addr": "172.21.15.102:6809", 2024-04-17T22:31:21.574 INFO:teuthology.orchestra.run.smithi102.stdout: "nonce": 42958 2024-04-17T22:31:21.574 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-17T22:31:21.574 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-04-17T22:31:21.574 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-17T22:31:21.574 INFO:teuthology.orchestra.run.smithi102.stdout: "heartbeat_back_addrs": { 2024-04-17T22:31:21.574 INFO:teuthology.orchestra.run.smithi102.stdout: "addrvec": [ 2024-04-17T22:31:21.574 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-04-17T22:31:21.575 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "v1", 2024-04-17T22:31:21.575 INFO:teuthology.orchestra.run.smithi102.stdout: "addr": "172.21.15.102:6811", 2024-04-17T22:31:21.575 INFO:teuthology.orchestra.run.smithi102.stdout: "nonce": 42958 2024-04-17T22:31:21.575 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-17T22:31:21.575 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-04-17T22:31:21.575 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-17T22:31:21.575 INFO:teuthology.orchestra.run.smithi102.stdout: "heartbeat_front_addrs": { 2024-04-17T22:31:21.575 INFO:teuthology.orchestra.run.smithi102.stdout: "addrvec": [ 2024-04-17T22:31:21.575 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-04-17T22:31:21.575 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "v1", 2024-04-17T22:31:21.575 INFO:teuthology.orchestra.run.smithi102.stdout: "addr": "172.21.15.102:6810", 2024-04-17T22:31:21.575 INFO:teuthology.orchestra.run.smithi102.stdout: "nonce": 42958 2024-04-17T22:31:21.575 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-17T22:31:21.575 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-04-17T22:31:21.575 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-17T22:31:21.576 INFO:teuthology.orchestra.run.smithi102.stdout: "public_addr": "172.21.15.102:6808/42958", 2024-04-17T22:31:21.576 INFO:teuthology.orchestra.run.smithi102.stdout: "cluster_addr": "172.21.15.102:6809/42958", 2024-04-17T22:31:21.576 INFO:teuthology.orchestra.run.smithi102.stdout: "heartbeat_back_addr": "172.21.15.102:6811/42958", 2024-04-17T22:31:21.576 INFO:teuthology.orchestra.run.smithi102.stdout: "heartbeat_front_addr": "172.21.15.102:6810/42958", 2024-04-17T22:31:21.576 INFO:teuthology.orchestra.run.smithi102.stdout: "state": [ 2024-04-17T22:31:21.576 INFO:teuthology.orchestra.run.smithi102.stdout: "exists", 2024-04-17T22:31:21.576 INFO:teuthology.orchestra.run.smithi102.stdout: "up" 2024-04-17T22:31:21.576 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-04-17T22:31:21.576 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-17T22:31:21.576 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-04-17T22:31:21.576 INFO:teuthology.orchestra.run.smithi102.stdout: "osd": 2, 2024-04-17T22:31:21.576 INFO:teuthology.orchestra.run.smithi102.stdout: "uuid": "d524204b-70b2-40a8-8f73-f9e7d44318af", 2024-04-17T22:31:21.576 INFO:teuthology.orchestra.run.smithi102.stdout: "up": 1, 2024-04-17T22:31:21.576 INFO:teuthology.orchestra.run.smithi102.stdout: "in": 1, 2024-04-17T22:31:21.576 INFO:teuthology.orchestra.run.smithi102.stdout: "weight": 1, 2024-04-17T22:31:21.577 INFO:teuthology.orchestra.run.smithi102.stdout: "primary_affinity": 1, 2024-04-17T22:31:21.577 INFO:teuthology.orchestra.run.smithi102.stdout: "last_clean_begin": 0, 2024-04-17T22:31:21.577 INFO:teuthology.orchestra.run.smithi102.stdout: "last_clean_end": 0, 2024-04-17T22:31:21.577 INFO:teuthology.orchestra.run.smithi102.stdout: "up_from": 9, 2024-04-17T22:31:21.577 INFO:teuthology.orchestra.run.smithi102.stdout: "up_thru": 14, 2024-04-17T22:31:21.577 INFO:teuthology.orchestra.run.smithi102.stdout: "down_at": 0, 2024-04-17T22:31:21.577 INFO:teuthology.orchestra.run.smithi102.stdout: "lost_at": 0, 2024-04-17T22:31:21.577 INFO:teuthology.orchestra.run.smithi102.stdout: "public_addrs": { 2024-04-17T22:31:21.577 INFO:teuthology.orchestra.run.smithi102.stdout: "addrvec": [ 2024-04-17T22:31:21.577 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-04-17T22:31:21.577 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "v1", 2024-04-17T22:31:21.577 INFO:teuthology.orchestra.run.smithi102.stdout: "addr": "172.21.15.102:6804", 2024-04-17T22:31:21.577 INFO:teuthology.orchestra.run.smithi102.stdout: "nonce": 42954 2024-04-17T22:31:21.577 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-17T22:31:21.578 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-04-17T22:31:21.578 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-17T22:31:21.578 INFO:teuthology.orchestra.run.smithi102.stdout: "cluster_addrs": { 2024-04-17T22:31:21.578 INFO:teuthology.orchestra.run.smithi102.stdout: "addrvec": [ 2024-04-17T22:31:21.578 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-04-17T22:31:21.578 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "v1", 2024-04-17T22:31:21.578 INFO:teuthology.orchestra.run.smithi102.stdout: "addr": "172.21.15.102:6805", 2024-04-17T22:31:21.578 INFO:teuthology.orchestra.run.smithi102.stdout: "nonce": 42954 2024-04-17T22:31:21.578 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-17T22:31:21.578 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-04-17T22:31:21.578 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-17T22:31:21.578 INFO:teuthology.orchestra.run.smithi102.stdout: "heartbeat_back_addrs": { 2024-04-17T22:31:21.578 INFO:teuthology.orchestra.run.smithi102.stdout: "addrvec": [ 2024-04-17T22:31:21.578 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-04-17T22:31:21.579 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "v1", 2024-04-17T22:31:21.579 INFO:teuthology.orchestra.run.smithi102.stdout: "addr": "172.21.15.102:6807", 2024-04-17T22:31:21.579 INFO:teuthology.orchestra.run.smithi102.stdout: "nonce": 42954 2024-04-17T22:31:21.579 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-17T22:31:21.579 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-04-17T22:31:21.579 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-17T22:31:21.579 INFO:teuthology.orchestra.run.smithi102.stdout: "heartbeat_front_addrs": { 2024-04-17T22:31:21.579 INFO:teuthology.orchestra.run.smithi102.stdout: "addrvec": [ 2024-04-17T22:31:21.579 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-04-17T22:31:21.579 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "v1", 2024-04-17T22:31:21.579 INFO:teuthology.orchestra.run.smithi102.stdout: "addr": "172.21.15.102:6806", 2024-04-17T22:31:21.579 INFO:teuthology.orchestra.run.smithi102.stdout: "nonce": 42954 2024-04-17T22:31:21.579 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-17T22:31:21.579 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-04-17T22:31:21.580 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-17T22:31:21.580 INFO:teuthology.orchestra.run.smithi102.stdout: "public_addr": "172.21.15.102:6804/42954", 2024-04-17T22:31:21.580 INFO:teuthology.orchestra.run.smithi102.stdout: "cluster_addr": "172.21.15.102:6805/42954", 2024-04-17T22:31:21.580 INFO:teuthology.orchestra.run.smithi102.stdout: "heartbeat_back_addr": "172.21.15.102:6807/42954", 2024-04-17T22:31:21.580 INFO:teuthology.orchestra.run.smithi102.stdout: "heartbeat_front_addr": "172.21.15.102:6806/42954", 2024-04-17T22:31:21.580 INFO:teuthology.orchestra.run.smithi102.stdout: "state": [ 2024-04-17T22:31:21.580 INFO:teuthology.orchestra.run.smithi102.stdout: "exists", 2024-04-17T22:31:21.580 INFO:teuthology.orchestra.run.smithi102.stdout: "up" 2024-04-17T22:31:21.580 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-04-17T22:31:21.580 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-17T22:31:21.580 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-04-17T22:31:21.580 INFO:teuthology.orchestra.run.smithi102.stdout: "osd": 3, 2024-04-17T22:31:21.580 INFO:teuthology.orchestra.run.smithi102.stdout: "uuid": "4fc9d20c-1ad0-484f-aafa-2c9a7574245b", 2024-04-17T22:31:21.580 INFO:teuthology.orchestra.run.smithi102.stdout: "up": 1, 2024-04-17T22:31:21.580 INFO:teuthology.orchestra.run.smithi102.stdout: "in": 1, 2024-04-17T22:31:21.581 INFO:teuthology.orchestra.run.smithi102.stdout: "weight": 1, 2024-04-17T22:31:21.581 INFO:teuthology.orchestra.run.smithi102.stdout: "primary_affinity": 1, 2024-04-17T22:31:21.581 INFO:teuthology.orchestra.run.smithi102.stdout: "last_clean_begin": 0, 2024-04-17T22:31:21.581 INFO:teuthology.orchestra.run.smithi102.stdout: "last_clean_end": 0, 2024-04-17T22:31:21.581 INFO:teuthology.orchestra.run.smithi102.stdout: "up_from": 9, 2024-04-17T22:31:21.581 INFO:teuthology.orchestra.run.smithi102.stdout: "up_thru": 14, 2024-04-17T22:31:21.581 INFO:teuthology.orchestra.run.smithi102.stdout: "down_at": 0, 2024-04-17T22:31:21.581 INFO:teuthology.orchestra.run.smithi102.stdout: "lost_at": 0, 2024-04-17T22:31:21.581 INFO:teuthology.orchestra.run.smithi102.stdout: "public_addrs": { 2024-04-17T22:31:21.581 INFO:teuthology.orchestra.run.smithi102.stdout: "addrvec": [ 2024-04-17T22:31:21.581 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-04-17T22:31:21.581 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "v1", 2024-04-17T22:31:21.581 INFO:teuthology.orchestra.run.smithi102.stdout: "addr": "172.21.15.102:6800", 2024-04-17T22:31:21.581 INFO:teuthology.orchestra.run.smithi102.stdout: "nonce": 42953 2024-04-17T22:31:21.581 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-17T22:31:21.582 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-04-17T22:31:21.582 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-17T22:31:21.582 INFO:teuthology.orchestra.run.smithi102.stdout: "cluster_addrs": { 2024-04-17T22:31:21.582 INFO:teuthology.orchestra.run.smithi102.stdout: "addrvec": [ 2024-04-17T22:31:21.582 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-04-17T22:31:21.582 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "v1", 2024-04-17T22:31:21.582 INFO:teuthology.orchestra.run.smithi102.stdout: "addr": "172.21.15.102:6801", 2024-04-17T22:31:21.582 INFO:teuthology.orchestra.run.smithi102.stdout: "nonce": 42953 2024-04-17T22:31:21.582 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-17T22:31:21.582 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-04-17T22:31:21.582 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-17T22:31:21.582 INFO:teuthology.orchestra.run.smithi102.stdout: "heartbeat_back_addrs": { 2024-04-17T22:31:21.582 INFO:teuthology.orchestra.run.smithi102.stdout: "addrvec": [ 2024-04-17T22:31:21.582 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-04-17T22:31:21.583 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "v1", 2024-04-17T22:31:21.583 INFO:teuthology.orchestra.run.smithi102.stdout: "addr": "172.21.15.102:6803", 2024-04-17T22:31:21.583 INFO:teuthology.orchestra.run.smithi102.stdout: "nonce": 42953 2024-04-17T22:31:21.583 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-17T22:31:21.583 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-04-17T22:31:21.583 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-17T22:31:21.583 INFO:teuthology.orchestra.run.smithi102.stdout: "heartbeat_front_addrs": { 2024-04-17T22:31:21.583 INFO:teuthology.orchestra.run.smithi102.stdout: "addrvec": [ 2024-04-17T22:31:21.583 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-04-17T22:31:21.583 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "v1", 2024-04-17T22:31:21.583 INFO:teuthology.orchestra.run.smithi102.stdout: "addr": "172.21.15.102:6802", 2024-04-17T22:31:21.583 INFO:teuthology.orchestra.run.smithi102.stdout: "nonce": 42953 2024-04-17T22:31:21.583 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-17T22:31:21.583 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-04-17T22:31:21.583 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-17T22:31:21.584 INFO:teuthology.orchestra.run.smithi102.stdout: "public_addr": "172.21.15.102:6800/42953", 2024-04-17T22:31:21.584 INFO:teuthology.orchestra.run.smithi102.stdout: "cluster_addr": "172.21.15.102:6801/42953", 2024-04-17T22:31:21.584 INFO:teuthology.orchestra.run.smithi102.stdout: "heartbeat_back_addr": "172.21.15.102:6803/42953", 2024-04-17T22:31:21.584 INFO:teuthology.orchestra.run.smithi102.stdout: "heartbeat_front_addr": "172.21.15.102:6802/42953", 2024-04-17T22:31:21.584 INFO:teuthology.orchestra.run.smithi102.stdout: "state": [ 2024-04-17T22:31:21.584 INFO:teuthology.orchestra.run.smithi102.stdout: "exists", 2024-04-17T22:31:21.584 INFO:teuthology.orchestra.run.smithi102.stdout: "up" 2024-04-17T22:31:21.584 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-04-17T22:31:21.584 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-17T22:31:21.584 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-17T22:31:21.584 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_xinfo": [ 2024-04-17T22:31:21.584 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-04-17T22:31:21.584 INFO:teuthology.orchestra.run.smithi102.stdout: "osd": 0, 2024-04-17T22:31:21.584 INFO:teuthology.orchestra.run.smithi102.stdout: "down_stamp": "0.000000", 2024-04-17T22:31:21.585 INFO:teuthology.orchestra.run.smithi102.stdout: "laggy_probability": 0, 2024-04-17T22:31:21.585 INFO:teuthology.orchestra.run.smithi102.stdout: "laggy_interval": 0, 2024-04-17T22:31:21.585 INFO:teuthology.orchestra.run.smithi102.stdout: "features": 4611087854035861503, 2024-04-17T22:31:21.585 INFO:teuthology.orchestra.run.smithi102.stdout: "old_weight": 0 2024-04-17T22:31:21.585 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-17T22:31:21.585 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-04-17T22:31:21.585 INFO:teuthology.orchestra.run.smithi102.stdout: "osd": 1, 2024-04-17T22:31:21.585 INFO:teuthology.orchestra.run.smithi102.stdout: "down_stamp": "0.000000", 2024-04-17T22:31:21.585 INFO:teuthology.orchestra.run.smithi102.stdout: "laggy_probability": 0, 2024-04-17T22:31:21.585 INFO:teuthology.orchestra.run.smithi102.stdout: "laggy_interval": 0, 2024-04-17T22:31:21.585 INFO:teuthology.orchestra.run.smithi102.stdout: "features": 4611087854035861503, 2024-04-17T22:31:21.585 INFO:teuthology.orchestra.run.smithi102.stdout: "old_weight": 0 2024-04-17T22:31:21.585 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-17T22:31:21.585 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-04-17T22:31:21.585 INFO:teuthology.orchestra.run.smithi102.stdout: "osd": 2, 2024-04-17T22:31:21.585 INFO:teuthology.orchestra.run.smithi102.stdout: "down_stamp": "0.000000", 2024-04-17T22:31:21.586 INFO:teuthology.orchestra.run.smithi102.stdout: "laggy_probability": 0, 2024-04-17T22:31:21.586 INFO:teuthology.orchestra.run.smithi102.stdout: "laggy_interval": 0, 2024-04-17T22:31:21.586 INFO:teuthology.orchestra.run.smithi102.stdout: "features": 4611087854035861503, 2024-04-17T22:31:21.586 INFO:teuthology.orchestra.run.smithi102.stdout: "old_weight": 0 2024-04-17T22:31:21.586 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-17T22:31:21.586 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-04-17T22:31:21.586 INFO:teuthology.orchestra.run.smithi102.stdout: "osd": 3, 2024-04-17T22:31:21.586 INFO:teuthology.orchestra.run.smithi102.stdout: "down_stamp": "0.000000", 2024-04-17T22:31:21.586 INFO:teuthology.orchestra.run.smithi102.stdout: "laggy_probability": 0, 2024-04-17T22:31:21.586 INFO:teuthology.orchestra.run.smithi102.stdout: "laggy_interval": 0, 2024-04-17T22:31:21.586 INFO:teuthology.orchestra.run.smithi102.stdout: "features": 4611087854035861503, 2024-04-17T22:31:21.586 INFO:teuthology.orchestra.run.smithi102.stdout: "old_weight": 0 2024-04-17T22:31:21.586 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-17T22:31:21.586 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-17T22:31:21.587 INFO:teuthology.orchestra.run.smithi102.stdout: "pg_upmap": [], 2024-04-17T22:31:21.587 INFO:teuthology.orchestra.run.smithi102.stdout: "pg_upmap_items": [], 2024-04-17T22:31:21.587 INFO:teuthology.orchestra.run.smithi102.stdout: "pg_temp": [], 2024-04-17T22:31:21.587 INFO:teuthology.orchestra.run.smithi102.stdout: "primary_temp": [], 2024-04-17T22:31:21.587 INFO:teuthology.orchestra.run.smithi102.stdout: "blacklist": { 2024-04-17T22:31:21.587 INFO:teuthology.orchestra.run.smithi102.stdout: "172.21.15.102:6817/1944542598": "2024-04-18 22:31:16.601994" 2024-04-17T22:31:21.587 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-17T22:31:21.587 INFO:teuthology.orchestra.run.smithi102.stdout: "erasure_code_profiles": { 2024-04-17T22:31:21.587 INFO:teuthology.orchestra.run.smithi102.stdout: "default": { 2024-04-17T22:31:21.587 INFO:teuthology.orchestra.run.smithi102.stdout: "crush-failure-domain": "osd", 2024-04-17T22:31:21.587 INFO:teuthology.orchestra.run.smithi102.stdout: "k": "2", 2024-04-17T22:31:21.587 INFO:teuthology.orchestra.run.smithi102.stdout: "m": "1", 2024-04-17T22:31:21.587 INFO:teuthology.orchestra.run.smithi102.stdout: "plugin": "jerasure", 2024-04-17T22:31:21.587 INFO:teuthology.orchestra.run.smithi102.stdout: "technique": "reed_sol_van" 2024-04-17T22:31:21.587 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-17T22:31:21.588 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-17T22:31:21.588 INFO:teuthology.orchestra.run.smithi102.stdout: "removed_snaps_queue": [], 2024-04-17T22:31:21.588 INFO:teuthology.orchestra.run.smithi102.stdout: "new_removed_snaps": [], 2024-04-17T22:31:21.588 INFO:teuthology.orchestra.run.smithi102.stdout: "new_purged_snaps": [], 2024-04-17T22:31:21.588 INFO:teuthology.orchestra.run.smithi102.stdout: "crush_node_flags": {}, 2024-04-17T22:31:21.588 INFO:teuthology.orchestra.run.smithi102.stdout: "device_class_flags": {} 2024-04-17T22:31:21.588 INFO:teuthology.orchestra.run.smithi102.stdout:} 2024-04-17T22:31:21.589 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.578 7f564b9b1700 1 -- 172.21.15.102:0/750384294 >> v1:172.21.15.102:6816/42360 conn(0x7f56280145a0 legacy=0x7f5628016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:21.589 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.578 7f564b9b1700 1 -- 172.21.15.102:0/750384294 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f56440d6570 msgr2=0x7f56441b3eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:21.590 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.578 7f564b9b1700 1 --2- 172.21.15.102:0/750384294 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f56440d6570 0x7f56441b3eb0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 rx=0x7f56440d73e0 tx=0x7f562c0068c0).stop 2024-04-17T22:31:21.590 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.578 7f564b9b1700 1 -- 172.21.15.102:0/750384294 shutdown_connections 2024-04-17T22:31:21.590 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.578 7f564b9b1700 1 --2- 172.21.15.102:0/750384294 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f56440d69c0 0x7f56441b44e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:21.590 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.578 7f564b9b1700 1 --2- 172.21.15.102:0/750384294 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f56440cf4c0 0x7f56440d5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:21.590 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.578 7f564b9b1700 1 --2- 172.21.15.102:0/750384294 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f56440d6570 0x7f56441b3eb0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:21.590 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.578 7f564b9b1700 1 -- 172.21.15.102:0/750384294 >> 172.21.15.102:0/750384294 conn(0x7f56440c3830 msgr2=0x7f56440d76c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:21.590 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.578 7f564b9b1700 1 -- 172.21.15.102:0/750384294 shutdown_connections 2024-04-17T22:31:21.590 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.578 7f564b9b1700 1 -- 172.21.15.102:0/750384294 wait complete. 2024-04-17T22:31:21.590 INFO:tasks.cephfs.cephfs_test_case:Removing 1 blacklist entries 2024-04-17T22:31:21.591 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blacklist rm 172.21.15.102:6817/1944542598 2024-04-17T22:31:21.720 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.733 7fcc9f071700 1 Processor -- start 2024-04-17T22:31:21.720 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.733 7fcc9f071700 1 -- start start 2024-04-17T22:31:21.720 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.733 7fcc9f071700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fcc9810e240 0x7fcc9810e690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:21.720 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.733 7fcc9f071700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fcc9810ecc0 0x7fcc980f1730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:21.720 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.733 7fcc9f071700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fcc980f1d60 0x7fcc981aa4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:21.720 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.733 7fcc9f071700 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc980f9f10 con 0x7fcc9810e240 2024-04-17T22:31:21.720 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.733 7fcc9f071700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc980f9030 con 0x7fcc980f1d60 2024-04-17T22:31:21.720 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.733 7fcc9f071700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcc980fa1b0 con 0x7fcc9810ecc0 2024-04-17T22:31:21.720 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.733 7fcc96ffd700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fcc9810ecc0 0x7fcc980f1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:21.720 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.733 7fcc97fff700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fcc980f1d60 0x7fcc981aa4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:21.721 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.733 7fcc977fe700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fcc9810e240 0x7fcc9810e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:21.721 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.733 7fcc96ffd700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fcc9810ecc0 0x7fcc980f1730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:49064/0 (socket says 172.21.15.102:49064) 2024-04-17T22:31:21.721 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.733 7fcc97fff700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fcc980f1d60 0x7fcc981aa4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:58986/0 (socket says 172.21.15.102:58986) 2024-04-17T22:31:21.721 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.733 7fcc977fe700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fcc9810e240 0x7fcc9810e690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:44236/0 (socket says 172.21.15.102:44236) 2024-04-17T22:31:21.721 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.733 7fcc96ffd700 1 -- 172.21.15.102:0/2890653558 learned_addr learned my addr 172.21.15.102:0/2890653558 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:31:21.721 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.733 7fcc96ffd700 1 -- 172.21.15.102:0/2890653558 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fcc980f1d60 msgr2=0x7fcc981aa4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:31:21.721 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.733 7fcc96ffd700 1 --2- 172.21.15.102:0/2890653558 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fcc980f1d60 0x7fcc981aa4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:21.721 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.733 7fcc96ffd700 1 -- 172.21.15.102:0/2890653558 >> v2:172.21.15.102:3300/0 conn(0x7fcc9810e240 msgr2=0x7fcc9810e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:31:21.721 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.734 7fcc96ffd700 1 --2- 172.21.15.102:0/2890653558 >> v2:172.21.15.102:3300/0 conn(0x7fcc9810e240 0x7fcc9810e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:21.721 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.734 7fcc96ffd700 1 -- 172.21.15.102:0/2890653558 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc980f21b0 con 0x7fcc9810ecc0 2024-04-17T22:31:21.721 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.734 7fcc97fff700 1 --2- 172.21.15.102:0/2890653558 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fcc980f1d60 0x7fcc981aa4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:21.721 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.734 7fcc96ffd700 1 --2- 172.21.15.102:0/2890653558 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fcc9810ecc0 0x7fcc980f1730 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 rx=0x7fcc980f46a0 tx=0x7fcc80009dc0).ready entity=mon.2 client_cookie=f584cc346ce5b2a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:21.721 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.734 7fcc967fc700 1 -- 172.21.15.102:0/2890653558 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fcc980f21b0 con 0x7fcc9810ecc0 2024-04-17T22:31:21.721 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.734 7fcc967fc700 1 -- 172.21.15.102:0/2890653558 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc8000b8d0 con 0x7fcc9810ecc0 2024-04-17T22:31:21.721 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.734 7fcc967fc700 1 -- 172.21.15.102:0/2890653558 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fcc8000bab0 con 0x7fcc9810ecc0 2024-04-17T22:31:21.722 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.734 7fcc9f071700 1 -- 172.21.15.102:0/2890653558 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fcc9810ecc0 msgr2=0x7fcc980f1730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:21.722 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.734 7fcc9f071700 1 --2- 172.21.15.102:0/2890653558 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fcc9810ecc0 0x7fcc980f1730 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 rx=0x7fcc980f46a0 tx=0x7fcc80009dc0).stop 2024-04-17T22:31:21.722 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.734 7fcc9f071700 1 -- 172.21.15.102:0/2890653558 shutdown_connections 2024-04-17T22:31:21.722 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.734 7fcc9f071700 1 --2- 172.21.15.102:0/2890653558 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fcc980f1d60 0x7fcc981aa4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:21.722 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.734 7fcc9f071700 1 --2- 172.21.15.102:0/2890653558 >> v2:172.21.15.102:3300/0 conn(0x7fcc9810e240 0x7fcc9810e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:21.722 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.734 7fcc9f071700 1 --2- 172.21.15.102:0/2890653558 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fcc9810ecc0 0x7fcc980f1730 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:21.722 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.734 7fcc9f071700 1 -- 172.21.15.102:0/2890653558 >> 172.21.15.102:0/2890653558 conn(0x7fcc981138a0 msgr2=0x7fcc980f4f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:21.722 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.734 7fcc9f071700 1 -- 172.21.15.102:0/2890653558 shutdown_connections 2024-04-17T22:31:21.722 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.734 7fcc9f071700 1 -- 172.21.15.102:0/2890653558 wait complete. 2024-04-17T22:31:21.742 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.757 7fcc9f071700 1 Processor -- start 2024-04-17T22:31:21.742 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.757 7fcc9f071700 1 -- start start 2024-04-17T22:31:21.744 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.757 7fcc9f071700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcc980cd970 0x7fcc980d5e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:21.744 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.758 7fcc9f071700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fcc980d6430 0x7fcc981b3d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:21.744 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.758 7fcc9f071700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fcc980d6880 0x7fcc981b4370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:21.745 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.758 7fcc9f071700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc980a8530 con 0x7fcc980cd970 2024-04-17T22:31:21.745 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.758 7fcc9f071700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc980c4300 con 0x7fcc980d6880 2024-04-17T22:31:21.745 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.758 7fcc9f071700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcc981b49a0 con 0x7fcc980d6430 2024-04-17T22:31:21.745 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.758 7fcc977fe700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcc980cd970 0x7fcc980d5e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:21.745 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.758 7fcc96ffd700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fcc980d6430 0x7fcc981b3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:21.745 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.758 7fcc97fff700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fcc980d6880 0x7fcc981b4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:21.745 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.758 7fcc977fe700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcc980cd970 0x7fcc980d5e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:44242/0 (socket says 172.21.15.102:44242) 2024-04-17T22:31:21.745 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.758 7fcc96ffd700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fcc980d6430 0x7fcc981b3d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:49078/0 (socket says 172.21.15.102:49078) 2024-04-17T22:31:21.745 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.758 7fcc977fe700 1 -- 172.21.15.102:0/1675022654 learned_addr learned my addr 172.21.15.102:0/1675022654 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:31:21.745 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.758 7fcc97fff700 1 -- 172.21.15.102:0/1675022654 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fcc980d6430 msgr2=0x7fcc981b3d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:21.745 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.758 7fcc97fff700 1 --2- 172.21.15.102:0/1675022654 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fcc980d6430 0x7fcc981b3d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:21.745 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.758 7fcc97fff700 1 -- 172.21.15.102:0/1675022654 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcc980cd970 msgr2=0x7fcc980d5e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:21.745 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.758 7fcc97fff700 1 --2- 172.21.15.102:0/1675022654 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcc980cd970 0x7fcc980d5e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:21.745 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.758 7fcc97fff700 1 -- 172.21.15.102:0/1675022654 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc981b4bd0 con 0x7fcc980d6880 2024-04-17T22:31:21.746 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.758 7fcc96ffd700 1 --2- 172.21.15.102:0/1675022654 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fcc980d6430 0x7fcc981b3d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:21.746 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.758 7fcc977fe700 1 --2- 172.21.15.102:0/1675022654 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcc980cd970 0x7fcc980d5e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:21.746 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.758 7fcc97fff700 1 --2- 172.21.15.102:0/1675022654 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fcc980d6880 0x7fcc981b4370 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 rx=0x7fcc980c2100 tx=0x7fcc8800ec70).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:21.746 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.758 7fcc94ff9700 1 -- 172.21.15.102:0/1675022654 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fcc981b4bd0 con 0x7fcc980d6880 2024-04-17T22:31:21.746 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.758 7fcc94ff9700 1 -- 172.21.15.102:0/1675022654 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc8800c790 con 0x7fcc980d6880 2024-04-17T22:31:21.746 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.758 7fcc9f071700 1 -- 172.21.15.102:0/1675022654 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc981bb430 con 0x7fcc980d6880 2024-04-17T22:31:21.746 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.758 7fcc94ff9700 1 -- 172.21.15.102:0/1675022654 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fcc8800f040 con 0x7fcc980d6880 2024-04-17T22:31:21.746 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.758 7fcc9f071700 1 -- 172.21.15.102:0/1675022654 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc981bb810 con 0x7fcc980d6880 2024-04-17T22:31:21.746 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.759 7fcc94ff9700 1 -- 172.21.15.102:0/1675022654 <== mon.1 v2:172.21.15.102:3301/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7fcc88011070 con 0x7fcc980d6880 2024-04-17T22:31:21.746 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.759 7fcc94ff9700 1 -- 172.21.15.102:0/1675022654 <== mon.1 v2:172.21.15.102:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 2417+0+0 (secure 0 0 0) 0x7fcc88010040 con 0x7fcc980d6880 2024-04-17T22:31:21.746 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.759 7fcc9f071700 1 -- 172.21.15.102:0/1675022654 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc980f0800 con 0x7fcc980d6880 2024-04-17T22:31:21.747 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.762 7fcc94ff9700 1 -- 172.21.15.102:0/1675022654 <== mon.1 v2:172.21.15.102:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+110455 (secure 0 0 0) 0x7fcc8800edb0 con 0x7fcc980d6880 2024-04-17T22:31:21.864 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:21.879 7fcc9f071700 1 -- 172.21.15.102:0/1675022654 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "osd blacklist", "blacklistop": "rm", "addr": "172.21.15.102:6817/1944542598"} v 0) v1 -- 0x7fcc981ae4c0 con 0x7fcc980d6880 2024-04-17T22:31:22.618 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:22.633 7fcc94ff9700 1 -- 172.21.15.102:0/1675022654 <== mon.1 v2:172.21.15.102:3301/0 7 ==== mon_command_ack([{"prefix": "osd blacklist", "blacklistop": "rm", "addr": "172.21.15.102:6817/1944542598"}]=0 un-blacklisting 172.21.15.102:6817/1944542598 v21) v1 ==== 168+0+0 (secure 0 0 0) 0x7fcc98114420 con 0x7fcc980d6880 2024-04-17T22:31:22.620 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:22.635 7fcc9f071700 1 -- 172.21.15.102:0/1675022654 >> v1:172.21.15.102:6816/42360 conn(0x7fcc7c0145a0 legacy=0x7fcc7c016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:22.620 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:22.635 7fcc9f071700 1 -- 172.21.15.102:0/1675022654 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fcc980d6880 msgr2=0x7fcc981b4370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:22.620 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:22.635 7fcc9f071700 1 --2- 172.21.15.102:0/1675022654 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fcc980d6880 0x7fcc981b4370 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 rx=0x7fcc980c2100 tx=0x7fcc8800ec70).stop 2024-04-17T22:31:22.620 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:22.635 7fcc9f071700 1 -- 172.21.15.102:0/1675022654 shutdown_connections 2024-04-17T22:31:22.620 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:22.635 7fcc9f071700 1 --2- 172.21.15.102:0/1675022654 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fcc980d6880 0x7fcc981b4370 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:22.620 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:22.635 7fcc9f071700 1 --2- 172.21.15.102:0/1675022654 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcc980cd970 0x7fcc980d5e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:22.621 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:22.635 7fcc9f071700 1 --2- 172.21.15.102:0/1675022654 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fcc980d6430 0x7fcc981b3d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:22.621 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:22.635 7fcc9f071700 1 -- 172.21.15.102:0/1675022654 >> 172.21.15.102:0/1675022654 conn(0x7fcc980e0f60 msgr2=0x7fcc980dac80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:22.621 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:22.635 7fcc9f071700 1 -- 172.21.15.102:0/1675022654 shutdown_connections 2024-04-17T22:31:22.621 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:22.635 7fcc9f071700 1 -- 172.21.15.102:0/1675022654 wait complete. 2024-04-17T22:31:22.621 INFO:teuthology.orchestra.run.smithi102.stderr:un-blacklisting 172.21.15.102:6817/1944542598 2024-04-17T22:31:22.633 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-17T22:31:22.765 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:22.778 7fb5befc5700 1 Processor -- start 2024-04-17T22:31:22.765 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:22.778 7fb5befc5700 1 -- start start 2024-04-17T22:31:22.765 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:22.779 7fb5befc5700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fb5b810e260 0x7fb5b810e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:22.766 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:22.779 7fb5befc5700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb5b810ece0 0x7fb5b80f1750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:22.766 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:22.779 7fb5befc5700 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5b80f21d0 con 0x7fb5b80f1d80 2024-04-17T22:31:22.766 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:22.779 7fb5befc5700 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5b80f9f30 con 0x7fb5b810e260 2024-04-17T22:31:22.766 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:22.779 7fb5befc5700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb5b80f9050 con 0x7fb5b810ece0 2024-04-17T22:31:22.766 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:22.779 7fb5b77fe700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fb5b810e260 0x7fb5b810e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:22.766 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:22.779 7fb5b6ffd700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb5b810ece0 0x7fb5b80f1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:22.766 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:22.779 7fb5b7fff700 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7fb5b80f1d80 0x7fb5b81aa4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:36472/0 (socket says 172.21.15.102:36472) 2024-04-17T22:31:22.766 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:22.779 7fb5b7fff700 1 -- 172.21.15.102:0/3493877076 learned_addr learned my addr 172.21.15.102:0/3493877076 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-17T22:31:22.766 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:22.779 7fb5b77fe700 1 -- 172.21.15.102:0/3493877076 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb5b810ece0 msgr2=0x7fb5b80f1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:31:22.766 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:22.779 7fb5b77fe700 1 --2- 172.21.15.102:0/3493877076 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb5b810ece0 0x7fb5b80f1750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:22.766 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:22.779 7fb5b77fe700 1 -- 172.21.15.102:0/3493877076 >> v1:172.21.15.102:6789/0 conn(0x7fb5b80f1d80 legacy=0x7fb5b81aa4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:22.766 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:22.779 7fb5b77fe700 1 -- 172.21.15.102:0/3493877076 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5b80f23f0 con 0x7fb5b810e260 2024-04-17T22:31:22.766 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:22.779 7fb5b6ffd700 1 --2- 172.21.15.102:0/3493877076 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb5b810ece0 0x7fb5b80f1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:22.767 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:22.779 7fb5b77fe700 1 --2- 172.21.15.102:0/3493877076 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb5b810e260 0x7fb5b810e6b0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 rx=0x7fb5ac002f70 tx=0x7fb5ac00b9f0).ready entity=mon.0 client_cookie=ea610ab368af20 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:22.767 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:22.779 7fb5b67fc700 1 -- 172.21.15.102:0/3493877076 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb5b80f23f0 con 0x7fb5b810e260 2024-04-17T22:31:22.767 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:22.779 7fb5b67fc700 1 -- 172.21.15.102:0/3493877076 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5ac00d610 con 0x7fb5b810e260 2024-04-17T22:31:22.767 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:22.779 7fb5b67fc700 1 -- 172.21.15.102:0/3493877076 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb5ac00d9e0 con 0x7fb5b810e260 2024-04-17T22:31:22.767 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:22.779 7fb5befc5700 1 -- 172.21.15.102:0/3493877076 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb5b810e260 msgr2=0x7fb5b810e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:22.767 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:22.779 7fb5befc5700 1 --2- 172.21.15.102:0/3493877076 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb5b810e260 0x7fb5b810e6b0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 rx=0x7fb5ac002f70 tx=0x7fb5ac00b9f0).stop 2024-04-17T22:31:22.767 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:22.780 7fb5befc5700 1 -- 172.21.15.102:0/3493877076 shutdown_connections 2024-04-17T22:31:22.767 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:22.780 7fb5befc5700 1 --2- 172.21.15.102:0/3493877076 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb5b810e260 0x7fb5b810e6b0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:22.767 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:22.780 7fb5befc5700 1 --2- 172.21.15.102:0/3493877076 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb5b810ece0 0x7fb5b80f1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:22.767 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:22.780 7fb5befc5700 1 -- 172.21.15.102:0/3493877076 >> 172.21.15.102:0/3493877076 conn(0x7fb5b81138c0 msgr2=0x7fb5b80f4f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:22.767 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:22.780 7fb5befc5700 1 -- 172.21.15.102:0/3493877076 shutdown_connections 2024-04-17T22:31:22.767 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:22.780 7fb5befc5700 1 -- 172.21.15.102:0/3493877076 wait complete. 2024-04-17T22:31:22.787 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:22.802 7fb5befc5700 1 Processor -- start 2024-04-17T22:31:22.787 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:22.803 7fb5befc5700 1 -- start start 2024-04-17T22:31:22.790 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:22.803 7fb5befc5700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb5b80da510 0x7fb5b80d5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:22.790 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:22.803 7fb5befc5700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb5b80d5890 0x7fb5b81b4260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:22.790 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:22.803 7fb5befc5700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb5b80d5ce0 0x7fb5b81b4890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:22.790 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:22.803 7fb5befc5700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5b808f2e0 con 0x7fb5b80d5ce0 2024-04-17T22:31:22.790 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:22.803 7fb5befc5700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5b808ed60 con 0x7fb5b80d5890 2024-04-17T22:31:22.790 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:22.803 7fb5befc5700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb5b80fa1d0 con 0x7fb5b80da510 2024-04-17T22:31:22.790 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:22.803 7fb5b77fe700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb5b80da510 0x7fb5b80d5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:22.790 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:22.803 7fb5b7fff700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb5b80d5ce0 0x7fb5b81b4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:22.790 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:22.803 7fb5b6ffd700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb5b80d5890 0x7fb5b81b4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:22.790 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:22.803 7fb5b77fe700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb5b80da510 0x7fb5b80d5260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:49094/0 (socket says 172.21.15.102:49094) 2024-04-17T22:31:22.790 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:22.803 7fb5b77fe700 1 -- 172.21.15.102:0/2978003118 learned_addr learned my addr 172.21.15.102:0/2978003118 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:31:22.790 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:22.803 7fb5b6ffd700 1 -- 172.21.15.102:0/2978003118 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb5b80da510 msgr2=0x7fb5b80d5260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:22.791 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:22.803 7fb5b6ffd700 1 --2- 172.21.15.102:0/2978003118 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb5b80da510 0x7fb5b80d5260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:22.791 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:22.803 7fb5b6ffd700 1 -- 172.21.15.102:0/2978003118 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb5b80d5ce0 msgr2=0x7fb5b81b4890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:22.791 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:22.803 7fb5b6ffd700 1 --2- 172.21.15.102:0/2978003118 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb5b80d5ce0 0x7fb5b81b4890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:22.791 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:22.803 7fb5b6ffd700 1 -- 172.21.15.102:0/2978003118 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5b80da960 con 0x7fb5b80d5890 2024-04-17T22:31:22.791 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:22.803 7fb5b7fff700 1 --2- 172.21.15.102:0/2978003118 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb5b80d5ce0 0x7fb5b81b4890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:22.791 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:22.803 7fb5b77fe700 1 --2- 172.21.15.102:0/2978003118 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb5b80da510 0x7fb5b80d5260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:22.791 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:22.803 7fb5b6ffd700 1 --2- 172.21.15.102:0/2978003118 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb5b80d5890 0x7fb5b81b4260 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 rx=0x7fb5b80c7080 tx=0x7fb5a800ed00).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:22.791 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:22.804 7fb5b4ff9700 1 -- 172.21.15.102:0/2978003118 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb5b80da960 con 0x7fb5b80d5890 2024-04-17T22:31:22.791 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:22.804 7fb5b4ff9700 1 -- 172.21.15.102:0/2978003118 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5a800c7b0 con 0x7fb5b80d5890 2024-04-17T22:31:22.791 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:22.804 7fb5befc5700 1 -- 172.21.15.102:0/2978003118 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5b81b4ec0 con 0x7fb5b80d5890 2024-04-17T22:31:22.791 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:22.804 7fb5b4ff9700 1 -- 172.21.15.102:0/2978003118 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb5a800f040 con 0x7fb5b80d5890 2024-04-17T22:31:22.791 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:22.804 7fb5befc5700 1 -- 172.21.15.102:0/2978003118 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5b81bba60 con 0x7fb5b80d5890 2024-04-17T22:31:22.791 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:22.804 7fb5b4ff9700 1 -- 172.21.15.102:0/2978003118 <== mon.1 v2:172.21.15.102:3301/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7fb5a8011070 con 0x7fb5b80d5890 2024-04-17T22:31:22.792 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:22.804 7fb5b4ff9700 1 -- 172.21.15.102:0/2978003118 <== mon.1 v2:172.21.15.102:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 2374+0+0 (secure 0 0 0) 0x7fb5a8010040 con 0x7fb5b80d5890 2024-04-17T22:31:22.792 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:22.804 7fb5befc5700 1 -- 172.21.15.102:0/2978003118 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5b80f0820 con 0x7fb5b80d5890 2024-04-17T22:31:22.792 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:22.808 7fb5b4ff9700 1 -- 172.21.15.102:0/2978003118 <== mon.1 v2:172.21.15.102:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+110455 (secure 0 0 0) 0x7fb5a800c5a0 con 0x7fb5b80d5890 2024-04-17T22:31:22.927 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:22.942 7fb5befc5700 1 -- 172.21.15.102:0/2978003118 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fb5b80f5550 con 0x7fb5b80d5890 2024-04-17T22:31:22.927 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:22.942 7fb5b4ff9700 1 -- 172.21.15.102:0/2978003118 <== mon.1 v2:172.21.15.102:3301/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v2) v1 ==== 80+0+3573 (secure 0 0 0) 0x7fb5a8000c30 con 0x7fb5b80d5890 2024-04-17T22:31:22.928 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-17T22:31:22.928 INFO:teuthology.orchestra.run.smithi102.stdout:{ 2024-04-17T22:31:22.928 INFO:teuthology.orchestra.run.smithi102.stdout: "auth_dump": [ 2024-04-17T22:31:22.928 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-04-17T22:31:22.928 INFO:teuthology.orchestra.run.smithi102.stdout: "entity": "mds.a", 2024-04-17T22:31:22.928 INFO:teuthology.orchestra.run.smithi102.stdout: "key": "AQAJTSBmD30qLBAAIPhihrDWdFgNV27UuM4C5Q==", 2024-04-17T22:31:22.928 INFO:teuthology.orchestra.run.smithi102.stdout: "caps": { 2024-04-17T22:31:22.928 INFO:teuthology.orchestra.run.smithi102.stdout: "mds": "allow", 2024-04-17T22:31:22.928 INFO:teuthology.orchestra.run.smithi102.stdout: "mgr": "allow *", 2024-04-17T22:31:22.929 INFO:teuthology.orchestra.run.smithi102.stdout: "mon": "allow *", 2024-04-17T22:31:22.929 INFO:teuthology.orchestra.run.smithi102.stdout: "osd": "allow *" 2024-04-17T22:31:22.929 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-17T22:31:22.929 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-17T22:31:22.929 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-04-17T22:31:22.929 INFO:teuthology.orchestra.run.smithi102.stdout: "entity": "osd.0", 2024-04-17T22:31:22.929 INFO:teuthology.orchestra.run.smithi102.stdout: "key": "AQANTSBmc2wAChAARaR//sfymHYNQZngBEXzlg==", 2024-04-17T22:31:22.929 INFO:teuthology.orchestra.run.smithi102.stdout: "caps": { 2024-04-17T22:31:22.929 INFO:teuthology.orchestra.run.smithi102.stdout: "mgr": "allow profile osd", 2024-04-17T22:31:22.929 INFO:teuthology.orchestra.run.smithi102.stdout: "mon": "allow profile osd", 2024-04-17T22:31:22.929 INFO:teuthology.orchestra.run.smithi102.stdout: "osd": "allow *" 2024-04-17T22:31:22.929 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-17T22:31:22.929 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-17T22:31:22.929 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-04-17T22:31:22.929 INFO:teuthology.orchestra.run.smithi102.stdout: "entity": "osd.1", 2024-04-17T22:31:22.930 INFO:teuthology.orchestra.run.smithi102.stdout: "key": "AQAQTSBmnIpmCBAAMw5q2mXkJxKuMhjiEq26xQ==", 2024-04-17T22:31:22.930 INFO:teuthology.orchestra.run.smithi102.stdout: "caps": { 2024-04-17T22:31:22.930 INFO:teuthology.orchestra.run.smithi102.stdout: "mgr": "allow profile osd", 2024-04-17T22:31:22.930 INFO:teuthology.orchestra.run.smithi102.stdout: "mon": "allow profile osd", 2024-04-17T22:31:22.930 INFO:teuthology.orchestra.run.smithi102.stdout: "osd": "allow *" 2024-04-17T22:31:22.930 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-17T22:31:22.930 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-17T22:31:22.930 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-04-17T22:31:22.930 INFO:teuthology.orchestra.run.smithi102.stdout: "entity": "osd.2", 2024-04-17T22:31:22.930 INFO:teuthology.orchestra.run.smithi102.stdout: "key": "AQATTSBmnAq/BhAANhjLDl3tH9XeRhOlnJsgjw==", 2024-04-17T22:31:22.930 INFO:teuthology.orchestra.run.smithi102.stdout: "caps": { 2024-04-17T22:31:22.930 INFO:teuthology.orchestra.run.smithi102.stdout: "mgr": "allow profile osd", 2024-04-17T22:31:22.930 INFO:teuthology.orchestra.run.smithi102.stdout: "mon": "allow profile osd", 2024-04-17T22:31:22.931 INFO:teuthology.orchestra.run.smithi102.stdout: "osd": "allow *" 2024-04-17T22:31:22.931 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-17T22:31:22.931 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-17T22:31:22.931 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-04-17T22:31:22.931 INFO:teuthology.orchestra.run.smithi102.stdout: "entity": "osd.3", 2024-04-17T22:31:22.931 INFO:teuthology.orchestra.run.smithi102.stdout: "key": "AQAWTSBmWq4vBRAATEjnXjqBqfsyiX7MaBNkYw==", 2024-04-17T22:31:22.931 INFO:teuthology.orchestra.run.smithi102.stdout: "caps": { 2024-04-17T22:31:22.931 INFO:teuthology.orchestra.run.smithi102.stdout: "mgr": "allow profile osd", 2024-04-17T22:31:22.931 INFO:teuthology.orchestra.run.smithi102.stdout: "mon": "allow profile osd", 2024-04-17T22:31:22.931 INFO:teuthology.orchestra.run.smithi102.stdout: "osd": "allow *" 2024-04-17T22:31:22.931 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-17T22:31:22.931 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-17T22:31:22.931 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-04-17T22:31:22.931 INFO:teuthology.orchestra.run.smithi102.stdout: "entity": "client.0", 2024-04-17T22:31:22.931 INFO:teuthology.orchestra.run.smithi102.stdout: "key": "AQAKTSBmLWP4ARAAV7xuVZUCRfvs7Zs54H89Qw==", 2024-04-17T22:31:22.932 INFO:teuthology.orchestra.run.smithi102.stdout: "caps": { 2024-04-17T22:31:22.932 INFO:teuthology.orchestra.run.smithi102.stdout: "mds": "allow", 2024-04-17T22:31:22.932 INFO:teuthology.orchestra.run.smithi102.stdout: "mgr": "allow r", 2024-04-17T22:31:22.932 INFO:teuthology.orchestra.run.smithi102.stdout: "mon": "allow rw", 2024-04-17T22:31:22.932 INFO:teuthology.orchestra.run.smithi102.stdout: "osd": "allow rwx" 2024-04-17T22:31:22.932 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-17T22:31:22.932 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-17T22:31:22.932 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-04-17T22:31:22.932 INFO:teuthology.orchestra.run.smithi102.stdout: "entity": "client.admin", 2024-04-17T22:31:22.932 INFO:teuthology.orchestra.run.smithi102.stdout: "key": "AQAJTSBmktjgEBAAmzv77Chs4a+851+GueLDoA==", 2024-04-17T22:31:22.932 INFO:teuthology.orchestra.run.smithi102.stdout: "caps": { 2024-04-17T22:31:22.932 INFO:teuthology.orchestra.run.smithi102.stdout: "mds": "allow *", 2024-04-17T22:31:22.932 INFO:teuthology.orchestra.run.smithi102.stdout: "mgr": "allow *", 2024-04-17T22:31:22.932 INFO:teuthology.orchestra.run.smithi102.stdout: "mon": "allow *", 2024-04-17T22:31:22.933 INFO:teuthology.orchestra.run.smithi102.stdout: "osd": "allow *" 2024-04-17T22:31:22.933 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-17T22:31:22.933 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-17T22:31:22.933 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-04-17T22:31:22.933 INFO:teuthology.orchestra.run.smithi102.stdout: "entity": "client.bootstrap-mds", 2024-04-17T22:31:22.933 INFO:teuthology.orchestra.run.smithi102.stdout: "key": "AQAgTSBmKB6GDBAAVVlb37rbrUSwkf6ZEVEMpw==", 2024-04-17T22:31:22.933 INFO:teuthology.orchestra.run.smithi102.stdout: "caps": { 2024-04-17T22:31:22.933 INFO:teuthology.orchestra.run.smithi102.stdout: "mon": "allow profile bootstrap-mds" 2024-04-17T22:31:22.933 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-17T22:31:22.933 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-17T22:31:22.933 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-04-17T22:31:22.933 INFO:teuthology.orchestra.run.smithi102.stdout: "entity": "client.bootstrap-mgr", 2024-04-17T22:31:22.933 INFO:teuthology.orchestra.run.smithi102.stdout: "key": "AQAgTSBmQSeGDBAA5tel7ZssLTWihwG6LLsm0Q==", 2024-04-17T22:31:22.933 INFO:teuthology.orchestra.run.smithi102.stdout: "caps": { 2024-04-17T22:31:22.934 INFO:teuthology.orchestra.run.smithi102.stdout: "mon": "allow profile bootstrap-mgr" 2024-04-17T22:31:22.934 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-17T22:31:22.934 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-17T22:31:22.934 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-04-17T22:31:22.934 INFO:teuthology.orchestra.run.smithi102.stdout: "entity": "client.bootstrap-osd", 2024-04-17T22:31:22.934 INFO:teuthology.orchestra.run.smithi102.stdout: "key": "AQAgTSBm30eGDBAAXHo3LGSKUAEIB8FMDensOQ==", 2024-04-17T22:31:22.934 INFO:teuthology.orchestra.run.smithi102.stdout: "caps": { 2024-04-17T22:31:22.934 INFO:teuthology.orchestra.run.smithi102.stdout: "mon": "allow profile bootstrap-osd" 2024-04-17T22:31:22.934 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-17T22:31:22.934 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-17T22:31:22.934 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-04-17T22:31:22.934 INFO:teuthology.orchestra.run.smithi102.stdout: "entity": "client.bootstrap-rbd", 2024-04-17T22:31:22.934 INFO:teuthology.orchestra.run.smithi102.stdout: "key": "AQAgTSBmBFGGDBAAbITQ4s6kvui6Rib+ZGcggQ==", 2024-04-17T22:31:22.934 INFO:teuthology.orchestra.run.smithi102.stdout: "caps": { 2024-04-17T22:31:22.935 INFO:teuthology.orchestra.run.smithi102.stdout: "mon": "allow profile bootstrap-rbd" 2024-04-17T22:31:22.935 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-17T22:31:22.935 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-17T22:31:22.935 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-04-17T22:31:22.935 INFO:teuthology.orchestra.run.smithi102.stdout: "entity": "client.bootstrap-rbd-mirror", 2024-04-17T22:31:22.935 INFO:teuthology.orchestra.run.smithi102.stdout: "key": "AQAgTSBmm1mGDBAAXeMreWHotPSrt8vz1VhQgw==", 2024-04-17T22:31:22.935 INFO:teuthology.orchestra.run.smithi102.stdout: "caps": { 2024-04-17T22:31:22.935 INFO:teuthology.orchestra.run.smithi102.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2024-04-17T22:31:22.935 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-17T22:31:22.935 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-17T22:31:22.935 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-04-17T22:31:22.935 INFO:teuthology.orchestra.run.smithi102.stdout: "entity": "client.bootstrap-rgw", 2024-04-17T22:31:22.935 INFO:teuthology.orchestra.run.smithi102.stdout: "key": "AQAgTSBmAmKGDBAAoeKKO3hMYkR3nbIThNQkKg==", 2024-04-17T22:31:22.936 INFO:teuthology.orchestra.run.smithi102.stdout: "caps": { 2024-04-17T22:31:22.936 INFO:teuthology.orchestra.run.smithi102.stdout: "mon": "allow profile bootstrap-rgw" 2024-04-17T22:31:22.936 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-17T22:31:22.936 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-17T22:31:22.936 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-04-17T22:31:22.936 INFO:teuthology.orchestra.run.smithi102.stdout: "entity": "mgr.x", 2024-04-17T22:31:22.936 INFO:teuthology.orchestra.run.smithi102.stdout: "key": "AQAJTSBmVVS9JxAAVo4zURUUQ/1CNk/ZkIe0FQ==", 2024-04-17T22:31:22.936 INFO:teuthology.orchestra.run.smithi102.stdout: "caps": { 2024-04-17T22:31:22.936 INFO:teuthology.orchestra.run.smithi102.stdout: "mds": "allow *", 2024-04-17T22:31:22.936 INFO:teuthology.orchestra.run.smithi102.stdout: "mon": "allow profile mgr", 2024-04-17T22:31:22.936 INFO:teuthology.orchestra.run.smithi102.stdout: "osd": "allow *" 2024-04-17T22:31:22.936 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-17T22:31:22.936 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-17T22:31:22.936 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-04-17T22:31:22.937 INFO:teuthology.orchestra.run.smithi102.stdout:} 2024-04-17T22:31:22.938 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:22.943 7fb5befc5700 1 -- 172.21.15.102:0/2978003118 >> v1:172.21.15.102:6816/42360 conn(0x7fb5980145a0 legacy=0x7fb598016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:22.938 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:22.943 7fb5befc5700 1 -- 172.21.15.102:0/2978003118 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb5b80d5890 msgr2=0x7fb5b81b4260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:22.938 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:22.943 7fb5befc5700 1 --2- 172.21.15.102:0/2978003118 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb5b80d5890 0x7fb5b81b4260 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 rx=0x7fb5b80c7080 tx=0x7fb5a800ed00).stop 2024-04-17T22:31:22.938 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:22.944 7fb5befc5700 1 -- 172.21.15.102:0/2978003118 shutdown_connections 2024-04-17T22:31:22.938 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:22.944 7fb5befc5700 1 --2- 172.21.15.102:0/2978003118 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb5b80d5ce0 0x7fb5b81b4890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:22.938 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:22.944 7fb5befc5700 1 --2- 172.21.15.102:0/2978003118 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb5b80da510 0x7fb5b80d5260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:22.938 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:22.944 7fb5befc5700 1 --2- 172.21.15.102:0/2978003118 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb5b80d5890 0x7fb5b81b4260 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:22.938 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:22.944 7fb5befc5700 1 -- 172.21.15.102:0/2978003118 >> 172.21.15.102:0/2978003118 conn(0x7fb5b80c6a60 msgr2=0x7fb5b81a8300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:22.938 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:22.944 7fb5befc5700 1 -- 172.21.15.102:0/2978003118 shutdown_connections 2024-04-17T22:31:22.938 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:22.944 7fb5befc5700 1 -- 172.21.15.102:0/2978003118 wait complete. 2024-04-17T22:31:22.940 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2024-04-17T22:31:23.067 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.080 7f40f80cb700 1 Processor -- start 2024-04-17T22:31:23.068 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.080 7f40f80cb700 1 -- start start 2024-04-17T22:31:23.068 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.080 7f40f80cb700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f40f00f57d0 0x7f40f010afa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:23.068 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.080 7f40f80cb700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f40f010b5d0 0x7f40f019fa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:23.068 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.080 7f40f80cb700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f40f010ba20 0x7f40f01a00b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:23.068 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.080 7f40f80cb700 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40f010be70 con 0x7f40f010b5d0 2024-04-17T22:31:23.068 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.080 7f40f80cb700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40f00fa300 con 0x7f40f00f57d0 2024-04-17T22:31:23.068 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.081 7f40f80cb700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f40f0103ad0 con 0x7f40f010ba20 2024-04-17T22:31:23.068 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.081 7f40f56c7700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f40f010ba20 0x7f40f01a00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:23.068 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.081 7f40f4ec6700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f40f00f57d0 0x7f40f010afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:23.068 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.081 7f40effff700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f40f010b5d0 0x7f40f019fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:23.068 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.081 7f40f56c7700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f40f010ba20 0x7f40f01a00b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:49100/0 (socket says 172.21.15.102:49100) 2024-04-17T22:31:23.069 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.081 7f40f56c7700 1 -- 172.21.15.102:0/3292577337 learned_addr learned my addr 172.21.15.102:0/3292577337 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:31:23.069 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.081 7f40f56c7700 1 -- 172.21.15.102:0/3292577337 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f40f00f57d0 msgr2=0x7f40f010afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:31:23.069 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.081 7f40f56c7700 1 --2- 172.21.15.102:0/3292577337 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f40f00f57d0 0x7f40f010afa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:23.069 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.081 7f40f56c7700 1 -- 172.21.15.102:0/3292577337 >> v2:172.21.15.102:3300/0 conn(0x7f40f010b5d0 msgr2=0x7f40f019fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:31:23.069 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.081 7f40f56c7700 1 --2- 172.21.15.102:0/3292577337 >> v2:172.21.15.102:3300/0 conn(0x7f40f010b5d0 0x7f40f019fa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:23.069 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.081 7f40f56c7700 1 -- 172.21.15.102:0/3292577337 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40f01a06e0 con 0x7f40f010ba20 2024-04-17T22:31:23.069 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.081 7f40effff700 1 --2- 172.21.15.102:0/3292577337 >> v2:172.21.15.102:3300/0 conn(0x7f40f010b5d0 0x7f40f019fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:23.069 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.081 7f40f4ec6700 1 --2- 172.21.15.102:0/3292577337 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f40f00f57d0 0x7f40f010afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:23.069 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.081 7f40f56c7700 1 --2- 172.21.15.102:0/3292577337 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f40f010ba20 0x7f40f01a00b0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 rx=0x7f40f00f4630 tx=0x7f40e800d5d0).ready entity=mon.2 client_cookie=fa60c7a5e6277af2 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:23.069 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.081 7f40ef7fe700 1 -- 172.21.15.102:0/3292577337 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f40f01a06e0 con 0x7f40f010ba20 2024-04-17T22:31:23.069 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.081 7f40ef7fe700 1 -- 172.21.15.102:0/3292577337 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40e8010660 con 0x7f40f010ba20 2024-04-17T22:31:23.069 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.081 7f40ef7fe700 1 -- 172.21.15.102:0/3292577337 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f40e8010a30 con 0x7f40f010ba20 2024-04-17T22:31:23.069 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.081 7f40f80cb700 1 -- 172.21.15.102:0/3292577337 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f40f010ba20 msgr2=0x7f40f01a00b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:23.070 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.081 7f40f80cb700 1 --2- 172.21.15.102:0/3292577337 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f40f010ba20 0x7f40f01a00b0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 rx=0x7f40f00f4630 tx=0x7f40e800d5d0).stop 2024-04-17T22:31:23.070 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.082 7f40f80cb700 1 -- 172.21.15.102:0/3292577337 shutdown_connections 2024-04-17T22:31:23.070 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.082 7f40f80cb700 1 --2- 172.21.15.102:0/3292577337 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f40f010ba20 0x7f40f01a00b0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:23.070 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.082 7f40f80cb700 1 --2- 172.21.15.102:0/3292577337 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f40f00f57d0 0x7f40f010afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:23.070 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.082 7f40f80cb700 1 --2- 172.21.15.102:0/3292577337 >> v2:172.21.15.102:3300/0 conn(0x7f40f010b5d0 0x7f40f019fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:23.070 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.082 7f40f80cb700 1 -- 172.21.15.102:0/3292577337 >> 172.21.15.102:0/3292577337 conn(0x7f40f0113830 msgr2=0x7f40f00f4ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:23.070 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.082 7f40f80cb700 1 -- 172.21.15.102:0/3292577337 shutdown_connections 2024-04-17T22:31:23.070 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.082 7f40f80cb700 1 -- 172.21.15.102:0/3292577337 wait complete. 2024-04-17T22:31:23.089 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.104 7f40f80cb700 1 Processor -- start 2024-04-17T22:31:23.089 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.104 7f40f80cb700 1 -- start start 2024-04-17T22:31:23.092 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.105 7f40f80cb700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f40f00c1d30 0x7f40f01ad0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:23.092 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.105 7f40f80cb700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f40f01ad6f0 0x7f40f01b3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:23.092 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.105 7f40f80cb700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f40f01adb40 0x7f40f01b4160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:23.092 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.105 7f40f80cb700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40f00c25d0 con 0x7f40f01adb40 2024-04-17T22:31:23.092 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.105 7f40f80cb700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40f00c2810 con 0x7f40f00c1d30 2024-04-17T22:31:23.093 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.105 7f40f80cb700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f40f01b4790 con 0x7f40f01ad6f0 2024-04-17T22:31:23.093 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.105 7f40f4ec6700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f40f00c1d30 0x7f40f01ad0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:23.093 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.105 7f40f56c7700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f40f01adb40 0x7f40f01b4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:23.093 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.105 7f40f4ec6700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f40f00c1d30 0x7f40f01ad0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:59024/0 (socket says 172.21.15.102:59024) 2024-04-17T22:31:23.093 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.105 7f40f4ec6700 1 -- 172.21.15.102:0/1575979054 learned_addr learned my addr 172.21.15.102:0/1575979054 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:31:23.093 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.105 7f40f56c7700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f40f01adb40 0x7f40f01b4160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:44296/0 (socket says 172.21.15.102:44296) 2024-04-17T22:31:23.093 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.105 7f40effff700 1 --2- 172.21.15.102:0/1575979054 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f40f01ad6f0 0x7f40f01b3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:23.093 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.105 7f40f56c7700 1 -- 172.21.15.102:0/1575979054 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f40f01ad6f0 msgr2=0x7f40f01b3b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:23.093 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.105 7f40f56c7700 1 --2- 172.21.15.102:0/1575979054 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f40f01ad6f0 0x7f40f01b3b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:23.093 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.105 7f40f56c7700 1 -- 172.21.15.102:0/1575979054 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f40f00c1d30 msgr2=0x7f40f01ad0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:23.093 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.105 7f40f56c7700 1 --2- 172.21.15.102:0/1575979054 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f40f00c1d30 0x7f40f01ad0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:23.093 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.105 7f40f56c7700 1 -- 172.21.15.102:0/1575979054 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40f01b4970 con 0x7f40f01adb40 2024-04-17T22:31:23.093 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.105 7f40f4ec6700 1 --2- 172.21.15.102:0/1575979054 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f40f00c1d30 0x7f40f01ad0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:23.093 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.105 7f40effff700 1 --2- 172.21.15.102:0/1575979054 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f40f01ad6f0 0x7f40f01b3b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:23.094 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.105 7f40f56c7700 1 --2- 172.21.15.102:0/1575979054 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f40f01adb40 0x7f40f01b4160 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 rx=0x7f40f00f4630 tx=0x7f40e8006c40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:23.094 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.106 7f40edffb700 1 -- 172.21.15.102:0/1575979054 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f40f01b4970 con 0x7f40f01adb40 2024-04-17T22:31:23.094 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.106 7f40edffb700 1 -- 172.21.15.102:0/1575979054 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40e8010e20 con 0x7f40f01adb40 2024-04-17T22:31:23.094 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.106 7f40f80cb700 1 -- 172.21.15.102:0/1575979054 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40f01bb220 con 0x7f40f01adb40 2024-04-17T22:31:23.094 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.106 7f40edffb700 1 -- 172.21.15.102:0/1575979054 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f40e80189f0 con 0x7f40f01adb40 2024-04-17T22:31:23.094 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.106 7f40f80cb700 1 -- 172.21.15.102:0/1575979054 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40f01bb600 con 0x7f40f01adb40 2024-04-17T22:31:23.094 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.106 7f40edffb700 1 -- 172.21.15.102:0/1575979054 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7f40e8018bf0 con 0x7f40f01adb40 2024-04-17T22:31:23.094 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.106 7f40edffb700 1 -- 172.21.15.102:0/1575979054 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 2374+0+0 (secure 0 0 0) 0x7f40e801aa00 con 0x7f40f01adb40 2024-04-17T22:31:23.094 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.107 7f40f80cb700 1 -- 172.21.15.102:0/1575979054 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40f00f0790 con 0x7f40f01adb40 2024-04-17T22:31:23.095 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.110 7f40edffb700 1 -- 172.21.15.102:0/1575979054 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+110455 (secure 0 0 0) 0x7f40e8018400 con 0x7f40f01adb40 2024-04-17T22:31:23.233 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.248 7f40f80cb700 1 -- 172.21.15.102:0/1575979054 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) v1 -- 0x7f40f01ae270 con 0x7f40f01adb40 2024-04-17T22:31:23.236 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.251 7f40edffb700 1 -- 172.21.15.102:0/1575979054 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 updated v3) v1 ==== 97+0+0 (secure 0 0 0) 0x7f40e800dc20 con 0x7f40f01adb40 2024-04-17T22:31:23.238 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.252 7f40f80cb700 1 -- 172.21.15.102:0/1575979054 >> v1:172.21.15.102:6816/42360 conn(0x7f40d80145a0 legacy=0x7f40d8016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:23.238 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.252 7f40f80cb700 1 -- 172.21.15.102:0/1575979054 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f40f01adb40 msgr2=0x7f40f01b4160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:23.238 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.252 7f40f80cb700 1 --2- 172.21.15.102:0/1575979054 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f40f01adb40 0x7f40f01b4160 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 rx=0x7f40f00f4630 tx=0x7f40e8006c40).stop 2024-04-17T22:31:23.238 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.252 7f40f80cb700 1 -- 172.21.15.102:0/1575979054 shutdown_connections 2024-04-17T22:31:23.238 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.252 7f40f80cb700 1 --2- 172.21.15.102:0/1575979054 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f40f01adb40 0x7f40f01b4160 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:23.238 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.252 7f40f80cb700 1 --2- 172.21.15.102:0/1575979054 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f40f00c1d30 0x7f40f01ad0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:23.238 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.252 7f40f80cb700 1 --2- 172.21.15.102:0/1575979054 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f40f01ad6f0 0x7f40f01b3b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:23.238 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.252 7f40f80cb700 1 -- 172.21.15.102:0/1575979054 >> 172.21.15.102:0/1575979054 conn(0x7f40f00c41e0 msgr2=0x7f40f00d9d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:23.238 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.252 7f40f80cb700 1 -- 172.21.15.102:0/1575979054 shutdown_connections 2024-04-17T22:31:23.238 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.252 7f40f80cb700 1 -- 172.21.15.102:0/1575979054 wait complete. 2024-04-17T22:31:23.239 INFO:teuthology.orchestra.run.smithi102.stderr:updated 2024-04-17T22:31:23.249 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2024-04-17T22:31:23.376 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.389 7f363abea700 1 Processor -- start 2024-04-17T22:31:23.376 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.389 7f363abea700 1 -- start start 2024-04-17T22:31:23.376 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.389 7f363abea700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f36340f5750 0x7f363410c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:23.376 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.389 7f363abea700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f363410cb70 0x7f36341a13e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:23.376 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.389 7f363abea700 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36341a1a10 con 0x7f363410c720 2024-04-17T22:31:23.376 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.389 7f363abea700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36340f93c0 con 0x7f363410cb70 2024-04-17T22:31:23.377 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.389 7f363abea700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f36341a1c30 con 0x7f36340f5750 2024-04-17T22:31:23.377 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.389 7f3633fff700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f363410cb70 0x7f36341a13e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:23.377 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.389 7f3632ffd700 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f363410c720 0x7f36341a0be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:33636/0 (socket says 172.21.15.102:33636) 2024-04-17T22:31:23.377 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.389 7f3632ffd700 1 -- 172.21.15.102:0/3930873632 learned_addr learned my addr 172.21.15.102:0/3930873632 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-17T22:31:23.377 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.389 7f36337fe700 1 --2- 172.21.15.102:0/3930873632 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f36340f5750 0x7f363410c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:23.377 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.389 7f3633fff700 1 -- 172.21.15.102:0/3930873632 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f36340f5750 msgr2=0x7f363410c0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:31:23.377 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.389 7f3633fff700 1 --2- 172.21.15.102:0/3930873632 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f36340f5750 0x7f363410c0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:23.377 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.389 7f3633fff700 1 -- 172.21.15.102:0/3930873632 >> v1:172.21.15.102:6789/0 conn(0x7f363410c720 legacy=0x7f36341a0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:23.377 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.389 7f3633fff700 1 -- 172.21.15.102:0/3930873632 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36341aa310 con 0x7f363410cb70 2024-04-17T22:31:23.377 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.389 7f36327fc700 1 -- 172.21.15.102:0/3930873632 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2625999128 0 0) 0x7f3624001110 con 0x7f363410c720 2024-04-17T22:31:23.377 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.389 7f36337fe700 1 --2- 172.21.15.102:0/3930873632 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f36340f5750 0x7f363410c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:23.377 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.390 7f3633fff700 1 --2- 172.21.15.102:0/3930873632 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f363410cb70 0x7f36341a13e0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 rx=0x7f36340f4470 tx=0x7f362c009d70).ready entity=mon.1 client_cookie=4cf768a7da138e1c server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:23.377 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.390 7f36327fc700 1 -- 172.21.15.102:0/3930873632 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f36341aa310 con 0x7f363410cb70 2024-04-17T22:31:23.377 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.390 7f36327fc700 1 -- 172.21.15.102:0/3930873632 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f362c00b7c0 con 0x7f363410cb70 2024-04-17T22:31:23.378 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.390 7f36327fc700 1 -- 172.21.15.102:0/3930873632 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f362c00b9a0 con 0x7f363410cb70 2024-04-17T22:31:23.378 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.390 7f363abea700 1 -- 172.21.15.102:0/3930873632 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f363410cb70 msgr2=0x7f36341a13e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:23.378 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.390 7f363abea700 1 --2- 172.21.15.102:0/3930873632 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f363410cb70 0x7f36341a13e0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 rx=0x7f36340f4470 tx=0x7f362c009d70).stop 2024-04-17T22:31:23.378 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.390 7f363abea700 1 -- 172.21.15.102:0/3930873632 shutdown_connections 2024-04-17T22:31:23.378 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.390 7f363abea700 1 --2- 172.21.15.102:0/3930873632 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f363410cb70 0x7f36341a13e0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:23.378 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.390 7f363abea700 1 --2- 172.21.15.102:0/3930873632 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f36340f5750 0x7f363410c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:23.378 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.390 7f363abea700 1 -- 172.21.15.102:0/3930873632 >> 172.21.15.102:0/3930873632 conn(0x7f36341138a0 msgr2=0x7f36340f4f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:23.378 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.390 7f363abea700 1 -- 172.21.15.102:0/3930873632 shutdown_connections 2024-04-17T22:31:23.378 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.390 7f363abea700 1 -- 172.21.15.102:0/3930873632 wait complete. 2024-04-17T22:31:23.398 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.413 7f363abea700 1 Processor -- start 2024-04-17T22:31:23.398 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.413 7f363abea700 1 -- start start 2024-04-17T22:31:23.400 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.414 7f363abea700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f36340cf300 0x7f36341ab0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:23.400 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.414 7f363abea700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f36341ab6f0 0x7f36341b1900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:23.400 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.414 7f363abea700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f36341abb40 0x7f36341b1f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:23.400 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.414 7f363abea700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f363408a9c0 con 0x7f36340cf300 2024-04-17T22:31:23.401 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.414 7f363abea700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36340d6790 con 0x7f36341abb40 2024-04-17T22:31:23.401 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.414 7f363abea700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f36340cd490 con 0x7f36341ab6f0 2024-04-17T22:31:23.401 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.414 7f36337fe700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f36340cf300 0x7f36341ab0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:23.401 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.414 7f3632ffd700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f36341ab6f0 0x7f36341b1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:23.401 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.414 7f3633fff700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f36341abb40 0x7f36341b1f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:23.401 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.414 7f36337fe700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f36340cf300 0x7f36341ab0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:51820/0 (socket says 172.21.15.102:51820) 2024-04-17T22:31:23.401 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.414 7f36337fe700 1 -- 172.21.15.102:0/4088082868 learned_addr learned my addr 172.21.15.102:0/4088082868 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:31:23.401 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.414 7f3632ffd700 1 -- 172.21.15.102:0/4088082868 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f36341abb40 msgr2=0x7f36341b1f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:23.401 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.414 7f3632ffd700 1 --2- 172.21.15.102:0/4088082868 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f36341abb40 0x7f36341b1f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:23.401 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.414 7f3632ffd700 1 -- 172.21.15.102:0/4088082868 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f36340cf300 msgr2=0x7f36341ab0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:23.401 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.414 7f3632ffd700 1 --2- 172.21.15.102:0/4088082868 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f36340cf300 0x7f36341ab0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:23.401 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.414 7f3632ffd700 1 -- 172.21.15.102:0/4088082868 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36341b2560 con 0x7f36341ab6f0 2024-04-17T22:31:23.401 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.414 7f3633fff700 1 --2- 172.21.15.102:0/4088082868 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f36341abb40 0x7f36341b1f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:23.401 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.414 7f36337fe700 1 --2- 172.21.15.102:0/4088082868 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f36340cf300 0x7f36341ab0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:23.402 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.414 7f3632ffd700 1 --2- 172.21.15.102:0/4088082868 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f36341ab6f0 0x7f36341b1900 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 rx=0x7f36340e10d0 tx=0x7f362400c780).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:23.402 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.414 7f3630ff9700 1 -- 172.21.15.102:0/4088082868 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f36341b2560 con 0x7f36341ab6f0 2024-04-17T22:31:23.402 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.414 7f3630ff9700 1 -- 172.21.15.102:0/4088082868 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36240070d0 con 0x7f36341ab6f0 2024-04-17T22:31:23.402 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.414 7f363abea700 1 -- 172.21.15.102:0/4088082868 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36341b9070 con 0x7f36341ab6f0 2024-04-17T22:31:23.402 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.414 7f3630ff9700 1 -- 172.21.15.102:0/4088082868 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f3624005620 con 0x7f36341ab6f0 2024-04-17T22:31:23.402 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.414 7f363abea700 1 -- 172.21.15.102:0/4088082868 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36341b9270 con 0x7f36341ab6f0 2024-04-17T22:31:23.402 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.415 7f3630ff9700 1 -- 172.21.15.102:0/4088082868 <== mon.2 v2:172.21.15.102:3302/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7f3624005820 con 0x7f36341ab6f0 2024-04-17T22:31:23.402 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.415 7f3630ff9700 1 -- 172.21.15.102:0/4088082868 <== mon.2 v2:172.21.15.102:3302/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 2374+0+0 (secure 0 0 0) 0x7f362401ee30 con 0x7f36341ab6f0 2024-04-17T22:31:23.402 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.415 7f363abea700 1 -- 172.21.15.102:0/4088082868 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36340f0800 con 0x7f36341ab6f0 2024-04-17T22:31:23.403 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.419 7f3630ff9700 1 -- 172.21.15.102:0/4088082868 <== mon.2 v2:172.21.15.102:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+110455 (secure 0 0 0) 0x7f362400cd70 con 0x7f36341ab6f0 2024-04-17T22:31:23.544 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.559 7f363abea700 1 -- 172.21.15.102:0/4088082868 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) v1 -- 0x7f36341b2820 con 0x7f36341ab6f0 2024-04-17T22:31:23.549 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.564 7f3630ff9700 1 -- 172.21.15.102:0/4088082868 <== mon.2 v2:172.21.15.102:3302/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 updated v4) v1 ==== 97+0+0 (secure 0 0 0) 0x7f3624000c30 con 0x7f36341ab6f0 2024-04-17T22:31:23.551 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.566 7f363abea700 1 -- 172.21.15.102:0/4088082868 >> v1:172.21.15.102:6816/42360 conn(0x7f36140145a0 legacy=0x7f3614016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:23.551 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.566 7f363abea700 1 -- 172.21.15.102:0/4088082868 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f36341ab6f0 msgr2=0x7f36341b1900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:23.551 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.566 7f363abea700 1 --2- 172.21.15.102:0/4088082868 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f36341ab6f0 0x7f36341b1900 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 rx=0x7f36340e10d0 tx=0x7f362400c780).stop 2024-04-17T22:31:23.552 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.566 7f363abea700 1 -- 172.21.15.102:0/4088082868 shutdown_connections 2024-04-17T22:31:23.552 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.566 7f363abea700 1 --2- 172.21.15.102:0/4088082868 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f36341abb40 0x7f36341b1f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:23.552 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.566 7f363abea700 1 --2- 172.21.15.102:0/4088082868 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f36340cf300 0x7f36341ab0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:23.552 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.566 7f363abea700 1 --2- 172.21.15.102:0/4088082868 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f36341ab6f0 0x7f36341b1900 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:23.552 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.566 7f363abea700 1 -- 172.21.15.102:0/4088082868 >> 172.21.15.102:0/4088082868 conn(0x7f36340c36b0 msgr2=0x7f36340dac80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:23.552 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.566 7f363abea700 1 -- 172.21.15.102:0/4088082868 shutdown_connections 2024-04-17T22:31:23.552 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.566 7f363abea700 1 -- 172.21.15.102:0/4088082868 wait complete. 2024-04-17T22:31:23.552 INFO:teuthology.orchestra.run.smithi102.stderr:updated 2024-04-17T22:31:23.562 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2024-04-17T22:31:23.691 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.704 7fef5cf04700 1 Processor -- start 2024-04-17T22:31:23.691 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.704 7fef5cf04700 1 -- start start 2024-04-17T22:31:23.691 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.704 7fef5cf04700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fef581a64b0 0x7fef581a8910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:23.691 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.704 7fef5cf04700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fef581a8f40 0x7fef581ab3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:23.692 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.704 7fef5cf04700 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef5810ee90 con 0x7fef5810e240 2024-04-17T22:31:23.692 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.704 7fef5cf04700 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef580f9f10 con 0x7fef581a8f40 2024-04-17T22:31:23.692 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.704 7fef5cf04700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fef580f9030 con 0x7fef581a64b0 2024-04-17T22:31:23.692 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.704 7fef56365700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fef581a8f40 0x7fef581ab3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:23.692 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.704 7fef55363700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fef581a64b0 0x7fef581a8910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:23.692 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.704 7fef55b64700 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7fef5810e240 0x7fef5810e690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:33650/0 (socket says 172.21.15.102:33650) 2024-04-17T22:31:23.692 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.704 7fef55b64700 1 -- 172.21.15.102:0/1101333343 learned_addr learned my addr 172.21.15.102:0/1101333343 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-17T22:31:23.692 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.705 7fef54b62700 1 -- 172.21.15.102:0/1101333343 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1875044077 0 0) 0x7fef4c001110 con 0x7fef5810e240 2024-04-17T22:31:23.692 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.705 7fef54b62700 1 -- 172.21.15.102:0/1101333343 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fef3c003270 con 0x7fef5810e240 2024-04-17T22:31:23.692 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.705 7fef55363700 1 -- 172.21.15.102:0/1101333343 >> v2:172.21.15.102:3300/0 conn(0x7fef581a8f40 msgr2=0x7fef581ab3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:31:23.692 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.705 7fef55363700 1 --2- 172.21.15.102:0/1101333343 >> v2:172.21.15.102:3300/0 conn(0x7fef581a8f40 0x7fef581ab3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:23.692 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.705 7fef55363700 1 -- 172.21.15.102:0/1101333343 >> v1:172.21.15.102:6789/0 conn(0x7fef5810e240 legacy=0x7fef5810e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:23.692 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.705 7fef55363700 1 -- 172.21.15.102:0/1101333343 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef581ac9e0 con 0x7fef581a64b0 2024-04-17T22:31:23.692 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.705 7fef56365700 1 --2- 172.21.15.102:0/1101333343 >> v2:172.21.15.102:3300/0 conn(0x7fef581a8f40 0x7fef581ab3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:23.693 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.705 7fef55363700 1 --2- 172.21.15.102:0/1101333343 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fef581a64b0 0x7fef581a8910 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 rx=0x7fef48002f70 tx=0x7fef4800b970).ready entity=mon.2 client_cookie=4abd786b0f64e331 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:23.693 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.705 7fef54b62700 1 -- 172.21.15.102:0/1101333343 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fef581ac9e0 con 0x7fef581a64b0 2024-04-17T22:31:23.693 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.705 7fef54b62700 1 -- 172.21.15.102:0/1101333343 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef4800d610 con 0x7fef581a64b0 2024-04-17T22:31:23.693 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.705 7fef54b62700 1 -- 172.21.15.102:0/1101333343 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fef4800d9e0 con 0x7fef581a64b0 2024-04-17T22:31:23.693 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.705 7fef5cf04700 1 -- 172.21.15.102:0/1101333343 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fef581a64b0 msgr2=0x7fef581a8910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:23.693 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.705 7fef5cf04700 1 --2- 172.21.15.102:0/1101333343 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fef581a64b0 0x7fef581a8910 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 rx=0x7fef48002f70 tx=0x7fef4800b970).stop 2024-04-17T22:31:23.693 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.706 7fef5cf04700 1 -- 172.21.15.102:0/1101333343 shutdown_connections 2024-04-17T22:31:23.693 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.706 7fef5cf04700 1 --2- 172.21.15.102:0/1101333343 >> v2:172.21.15.102:3300/0 conn(0x7fef581a8f40 0x7fef581ab3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:23.693 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.706 7fef5cf04700 1 --2- 172.21.15.102:0/1101333343 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fef581a64b0 0x7fef581a8910 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:23.693 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.706 7fef5cf04700 1 -- 172.21.15.102:0/1101333343 >> 172.21.15.102:0/1101333343 conn(0x7fef581138a0 msgr2=0x7fef580f4f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:23.693 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.706 7fef5cf04700 1 -- 172.21.15.102:0/1101333343 shutdown_connections 2024-04-17T22:31:23.693 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.706 7fef5cf04700 1 -- 172.21.15.102:0/1101333343 wait complete. 2024-04-17T22:31:23.715 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.728 7fef5cf04700 1 Processor -- start 2024-04-17T22:31:23.716 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.729 7fef5cf04700 1 -- start start 2024-04-17T22:31:23.716 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.729 7fef5cf04700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fef580cddf0 0x7fef580d5ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:23.716 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.729 7fef5cf04700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fef580d64f0 0x7fef581b3ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:23.716 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.729 7fef5cf04700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fef580d6940 0x7fef581b4500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:23.716 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.729 7fef5cf04700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef58113a40 con 0x7fef580d6940 2024-04-17T22:31:23.716 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.729 7fef5cf04700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef580fa1b0 con 0x7fef580cddf0 2024-04-17T22:31:23.716 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.729 7fef5cf04700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fef580ce240 con 0x7fef580d64f0 2024-04-17T22:31:23.716 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.729 7fef56365700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fef580d6940 0x7fef581b4500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:23.716 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.729 7fef55b64700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fef580cddf0 0x7fef580d5ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:23.716 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.729 7fef55363700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fef580d64f0 0x7fef581b3ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:23.716 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.729 7fef56365700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fef580d6940 0x7fef581b4500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:51836/0 (socket says 172.21.15.102:51836) 2024-04-17T22:31:23.716 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.729 7fef56365700 1 -- 172.21.15.102:0/1939550399 learned_addr learned my addr 172.21.15.102:0/1939550399 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:31:23.716 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.729 7fef55b64700 1 --2- 172.21.15.102:0/1939550399 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fef580cddf0 0x7fef580d5ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:60800/0 (socket says 172.21.15.102:60800) 2024-04-17T22:31:23.717 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.729 7fef56365700 1 -- 172.21.15.102:0/1939550399 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fef580d64f0 msgr2=0x7fef581b3ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:23.717 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.729 7fef56365700 1 --2- 172.21.15.102:0/1939550399 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fef580d64f0 0x7fef581b3ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:23.717 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.729 7fef56365700 1 -- 172.21.15.102:0/1939550399 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fef580cddf0 msgr2=0x7fef580d5ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:23.717 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.729 7fef56365700 1 --2- 172.21.15.102:0/1939550399 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fef580cddf0 0x7fef580d5ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:23.717 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.729 7fef56365700 1 -- 172.21.15.102:0/1939550399 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef580cd2f0 con 0x7fef580d6940 2024-04-17T22:31:23.717 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.729 7fef55363700 1 --2- 172.21.15.102:0/1939550399 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fef580d64f0 0x7fef581b3ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:23.717 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.729 7fef56365700 1 --2- 172.21.15.102:0/1939550399 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fef580d6940 0x7fef581b4500 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 rx=0x7fef580d7370 tx=0x7fef5000ee80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:23.717 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.729 7fef46ffd700 1 -- 172.21.15.102:0/1939550399 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fef580cd2f0 con 0x7fef580d6940 2024-04-17T22:31:23.717 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.730 7fef46ffd700 1 -- 172.21.15.102:0/1939550399 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef5000c880 con 0x7fef580d6940 2024-04-17T22:31:23.717 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.730 7fef46ffd700 1 -- 172.21.15.102:0/1939550399 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fef5000f040 con 0x7fef580d6940 2024-04-17T22:31:23.717 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.730 7fef5cf04700 1 -- 172.21.15.102:0/1939550399 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef581b4b30 con 0x7fef580d6940 2024-04-17T22:31:23.717 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.730 7fef5cf04700 1 -- 172.21.15.102:0/1939550399 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef581bb5c0 con 0x7fef580d6940 2024-04-17T22:31:23.717 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.730 7fef46ffd700 1 -- 172.21.15.102:0/1939550399 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7fef50022060 con 0x7fef580d6940 2024-04-17T22:31:23.718 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.730 7fef46ffd700 1 -- 172.21.15.102:0/1939550399 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 2374+0+0 (secure 0 0 0) 0x7fef50010040 con 0x7fef580d6940 2024-04-17T22:31:23.718 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.731 7fef5cf04700 1 -- 172.21.15.102:0/1939550399 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef580f0800 con 0x7fef580d6940 2024-04-17T22:31:23.719 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.734 7fef46ffd700 1 -- 172.21.15.102:0/1939550399 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+110455 (secure 0 0 0) 0x7fef5000c670 con 0x7fef580d6940 2024-04-17T22:31:23.858 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.873 7fef5cf04700 1 -- 172.21.15.102:0/1939550399 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) v1 -- 0x7fef581ae5d0 con 0x7fef580d6940 2024-04-17T22:31:23.862 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.877 7fef46ffd700 1 -- 172.21.15.102:0/1939550399 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 updated v5) v1 ==== 97+0+0 (secure 0 0 0) 0x7fef58114420 con 0x7fef580d6940 2024-04-17T22:31:23.864 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.879 7fef5cf04700 1 -- 172.21.15.102:0/1939550399 >> v1:172.21.15.102:6816/42360 conn(0x7fef340145a0 legacy=0x7fef34016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:23.864 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.879 7fef5cf04700 1 -- 172.21.15.102:0/1939550399 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fef580d6940 msgr2=0x7fef581b4500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:23.864 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.879 7fef5cf04700 1 --2- 172.21.15.102:0/1939550399 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fef580d6940 0x7fef581b4500 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 rx=0x7fef580d7370 tx=0x7fef5000ee80).stop 2024-04-17T22:31:23.864 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.879 7fef5cf04700 1 -- 172.21.15.102:0/1939550399 shutdown_connections 2024-04-17T22:31:23.864 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.879 7fef5cf04700 1 --2- 172.21.15.102:0/1939550399 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fef580d6940 0x7fef581b4500 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:23.864 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.879 7fef5cf04700 1 --2- 172.21.15.102:0/1939550399 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fef580cddf0 0x7fef580d5ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:23.865 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.879 7fef5cf04700 1 --2- 172.21.15.102:0/1939550399 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fef580d64f0 0x7fef581b3ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:23.865 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.879 7fef5cf04700 1 -- 172.21.15.102:0/1939550399 >> 172.21.15.102:0/1939550399 conn(0x7fef580c37a0 msgr2=0x7fef580d7640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:23.865 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.879 7fef5cf04700 1 -- 172.21.15.102:0/1939550399 shutdown_connections 2024-04-17T22:31:23.865 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:23.879 7fef5cf04700 1 -- 172.21.15.102:0/1939550399 wait complete. 2024-04-17T22:31:23.865 INFO:teuthology.orchestra.run.smithi102.stderr:updated 2024-04-17T22:31:23.877 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2024-04-17T22:31:24.004 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.017 7f21a16f9700 1 Processor -- start 2024-04-17T22:31:24.004 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.017 7f21a16f9700 1 -- start start 2024-04-17T22:31:24.004 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.017 7f21a16f9700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f219c10e240 0x7f219c10e690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:24.005 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.017 7f21a16f9700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f219c10ecc0 0x7f219c0f1730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:24.005 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.017 7f21a16f9700 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f219c0f21b0 con 0x7f219c0f1d60 2024-04-17T22:31:24.005 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.017 7f21a16f9700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f219c0f9f10 con 0x7f219c10e240 2024-04-17T22:31:24.005 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.017 7f21a16f9700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f219c0f9030 con 0x7f219c10ecc0 2024-04-17T22:31:24.005 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.017 7f219a4ec700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f219c10e240 0x7f219c10e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:24.005 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.017 7f219aced700 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f219c0f1d60 0x7f219c1aa4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:33666/0 (socket says 172.21.15.102:33666) 2024-04-17T22:31:24.005 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.017 7f219aced700 1 -- 172.21.15.102:0/2978129462 learned_addr learned my addr 172.21.15.102:0/2978129462 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-17T22:31:24.005 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.017 7f2193fff700 1 --2- 172.21.15.102:0/2978129462 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f219c10ecc0 0x7f219c0f1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:24.005 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.018 7f2199ceb700 1 -- 172.21.15.102:0/2978129462 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2730021225 0 0) 0x7f2194001490 con 0x7f219c0f1d60 2024-04-17T22:31:24.005 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.018 7f2193fff700 1 -- 172.21.15.102:0/2978129462 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f219c10e240 msgr2=0x7f219c10e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:31:24.005 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.018 7f2193fff700 1 --2- 172.21.15.102:0/2978129462 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f219c10e240 0x7f219c10e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:24.005 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.018 7f2193fff700 1 -- 172.21.15.102:0/2978129462 >> v1:172.21.15.102:6789/0 conn(0x7f219c0f1d60 legacy=0x7f219c1aa4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:24.005 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.018 7f2193fff700 1 -- 172.21.15.102:0/2978129462 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f219c0f23d0 con 0x7f219c10ecc0 2024-04-17T22:31:24.005 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.018 7f219a4ec700 1 --2- 172.21.15.102:0/2978129462 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f219c10e240 0x7f219c10e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:24.006 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.018 7f2193fff700 1 --2- 172.21.15.102:0/2978129462 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f219c10ecc0 0x7f219c0f1730 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 rx=0x7f2184002f70 tx=0x7f218400b9a0).ready entity=mon.2 client_cookie=a30c4e566da75b4b server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:24.006 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.018 7f2199ceb700 1 -- 172.21.15.102:0/2978129462 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f219c0f23d0 con 0x7f219c10ecc0 2024-04-17T22:31:24.006 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.018 7f2199ceb700 1 -- 172.21.15.102:0/2978129462 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f218400d570 con 0x7f219c10ecc0 2024-04-17T22:31:24.006 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.018 7f2199ceb700 1 -- 172.21.15.102:0/2978129462 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f218400d940 con 0x7f219c10ecc0 2024-04-17T22:31:24.006 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.018 7f21a16f9700 1 -- 172.21.15.102:0/2978129462 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f219c10ecc0 msgr2=0x7f219c0f1730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:24.006 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.018 7f21a16f9700 1 --2- 172.21.15.102:0/2978129462 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f219c10ecc0 0x7f219c0f1730 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 rx=0x7f2184002f70 tx=0x7f218400b9a0).stop 2024-04-17T22:31:24.006 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.018 7f21a16f9700 1 -- 172.21.15.102:0/2978129462 shutdown_connections 2024-04-17T22:31:24.006 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.018 7f21a16f9700 1 --2- 172.21.15.102:0/2978129462 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f219c10e240 0x7f219c10e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:24.006 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.018 7f21a16f9700 1 --2- 172.21.15.102:0/2978129462 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f219c10ecc0 0x7f219c0f1730 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:24.006 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.018 7f21a16f9700 1 -- 172.21.15.102:0/2978129462 >> 172.21.15.102:0/2978129462 conn(0x7f219c1138a0 msgr2=0x7f219c0f4f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:24.006 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.019 7f21a16f9700 1 -- 172.21.15.102:0/2978129462 shutdown_connections 2024-04-17T22:31:24.006 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.019 7f21a16f9700 1 -- 172.21.15.102:0/2978129462 wait complete. 2024-04-17T22:31:24.028 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.041 7f21a16f9700 1 Processor -- start 2024-04-17T22:31:24.028 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.041 7f21a16f9700 1 -- start start 2024-04-17T22:31:24.028 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.041 7f21a16f9700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f219c0d5e00 0x7f219c0d6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:24.028 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.041 7f21a16f9700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f219c0d6880 0x7f219c1b3e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:24.028 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.041 7f21a16f9700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f219c1b4480 0x7f219c1b48d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:24.028 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.041 7f21a16f9700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f219c08abc0 con 0x7f219c0d5e00 2024-04-17T22:31:24.028 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.041 7f21a16f9700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f219c0fa1b0 con 0x7f219c1b4480 2024-04-17T22:31:24.028 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.041 7f21a16f9700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f219c0c4300 con 0x7f219c0d6880 2024-04-17T22:31:24.029 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.041 7f219a4ec700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f219c0d5e00 0x7f219c0d6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:24.029 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.041 7f2193fff700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f219c0d6880 0x7f219c1b3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:24.029 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.042 7f219a4ec700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f219c0d5e00 0x7f219c0d6250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:51852/0 (socket says 172.21.15.102:51852) 2024-04-17T22:31:24.029 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.042 7f219a4ec700 1 -- 172.21.15.102:0/287808417 learned_addr learned my addr 172.21.15.102:0/287808417 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:31:24.029 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.042 7f219aced700 1 --2- 172.21.15.102:0/287808417 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f219c1b4480 0x7f219c1b48d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:24.029 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.042 7f2193fff700 1 -- 172.21.15.102:0/287808417 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f219c1b4480 msgr2=0x7f219c1b48d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:24.029 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.042 7f2193fff700 1 --2- 172.21.15.102:0/287808417 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f219c1b4480 0x7f219c1b48d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:24.029 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.042 7f2193fff700 1 -- 172.21.15.102:0/287808417 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f219c0d5e00 msgr2=0x7f219c0d6250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:24.029 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.042 7f2193fff700 1 --2- 172.21.15.102:0/287808417 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f219c0d5e00 0x7f219c0d6250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:24.029 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.042 7f2193fff700 1 -- 172.21.15.102:0/287808417 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f219c0cd170 con 0x7f219c0d6880 2024-04-17T22:31:24.029 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.042 7f219a4ec700 1 --2- 172.21.15.102:0/287808417 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f219c0d5e00 0x7f219c0d6250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:24.029 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.042 7f2193fff700 1 --2- 172.21.15.102:0/287808417 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f219c0d6880 0x7f219c1b3e50 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 rx=0x7f219c0d72a0 tx=0x7f21840068c0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:24.029 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.042 7f21937fe700 1 -- 172.21.15.102:0/287808417 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f219c0cd170 con 0x7f219c0d6880 2024-04-17T22:31:24.030 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.042 7f21937fe700 1 -- 172.21.15.102:0/287808417 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2184011980 con 0x7f219c0d6880 2024-04-17T22:31:24.030 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.042 7f21a16f9700 1 -- 172.21.15.102:0/287808417 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f219c1bb540 con 0x7f219c0d6880 2024-04-17T22:31:24.030 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.042 7f21937fe700 1 -- 172.21.15.102:0/287808417 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f2184017040 con 0x7f219c0d6880 2024-04-17T22:31:24.030 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.042 7f21a16f9700 1 -- 172.21.15.102:0/287808417 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f219c1bb740 con 0x7f219c0d6880 2024-04-17T22:31:24.030 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.043 7f21937fe700 1 -- 172.21.15.102:0/287808417 <== mon.2 v2:172.21.15.102:3302/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7f21840273b0 con 0x7f219c0d6880 2024-04-17T22:31:24.030 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.043 7f21937fe700 1 -- 172.21.15.102:0/287808417 <== mon.2 v2:172.21.15.102:3302/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 2374+0+0 (secure 0 0 0) 0x7f2184018070 con 0x7f219c0d6880 2024-04-17T22:31:24.030 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.043 7f21a16f9700 1 -- 172.21.15.102:0/287808417 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f219c0f0800 con 0x7f219c0d6880 2024-04-17T22:31:24.031 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.046 7f21937fe700 1 -- 172.21.15.102:0/287808417 <== mon.2 v2:172.21.15.102:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+110455 (secure 0 0 0) 0x7f2184011770 con 0x7f219c0d6880 2024-04-17T22:31:24.170 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.185 7f21a16f9700 1 -- 172.21.15.102:0/287808417 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) v1 -- 0x7f219c1ae5d0 con 0x7f219c0d6880 2024-04-17T22:31:24.175 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.190 7f21937fe700 1 -- 172.21.15.102:0/287808417 <== mon.2 v2:172.21.15.102:3302/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 updated v6) v1 ==== 97+0+0 (secure 0 0 0) 0x7f218400d360 con 0x7f219c0d6880 2024-04-17T22:31:24.177 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.191 7f21a16f9700 1 -- 172.21.15.102:0/287808417 >> v1:172.21.15.102:6816/42360 conn(0x7f21800145a0 legacy=0x7f2180016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:24.177 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.191 7f21a16f9700 1 -- 172.21.15.102:0/287808417 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f219c0d6880 msgr2=0x7f219c1b3e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:24.177 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.191 7f21a16f9700 1 --2- 172.21.15.102:0/287808417 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f219c0d6880 0x7f219c1b3e50 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 rx=0x7f219c0d72a0 tx=0x7f21840068c0).stop 2024-04-17T22:31:24.177 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.191 7f21a16f9700 1 -- 172.21.15.102:0/287808417 shutdown_connections 2024-04-17T22:31:24.177 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.191 7f21a16f9700 1 --2- 172.21.15.102:0/287808417 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f219c1b4480 0x7f219c1b48d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:24.177 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.191 7f21a16f9700 1 --2- 172.21.15.102:0/287808417 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f219c0d5e00 0x7f219c0d6250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:24.177 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.191 7f21a16f9700 1 --2- 172.21.15.102:0/287808417 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f219c0d6880 0x7f219c1b3e50 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:24.177 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.191 7f21a16f9700 1 -- 172.21.15.102:0/287808417 >> 172.21.15.102:0/287808417 conn(0x7f219c0e0f60 msgr2=0x7f219c0dac80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:24.178 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.192 7f21a16f9700 1 -- 172.21.15.102:0/287808417 shutdown_connections 2024-04-17T22:31:24.178 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.192 7f21a16f9700 1 -- 172.21.15.102:0/287808417 wait complete. 2024-04-17T22:31:24.178 INFO:teuthology.orchestra.run.smithi102.stderr:updated 2024-04-17T22:31:24.188 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2024-04-17T22:31:24.316 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.328 7f9cb29c1700 1 Processor -- start 2024-04-17T22:31:24.316 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.328 7f9cb29c1700 1 -- start start 2024-04-17T22:31:24.316 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.329 7f9cb29c1700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f9cac10e240 0x7f9cac10e690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:24.317 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.329 7f9cb29c1700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f9cac10ecc0 0x7f9cac0f1730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:24.317 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.329 7f9cb29c1700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f9cac0f1d60 0x7f9cac1aa4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:24.317 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.329 7f9cb29c1700 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9cac0f9f10 con 0x7f9cac0f1d60 2024-04-17T22:31:24.317 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.329 7f9cb29c1700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cac0f9030 con 0x7f9cac10ecc0 2024-04-17T22:31:24.317 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.329 7f9cb29c1700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9cac0fa1b0 con 0x7f9cac10e240 2024-04-17T22:31:24.317 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.329 7f9cab5eb700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f9cac10e240 0x7f9cac10e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:24.317 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.329 7f9caadea700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f9cac10ecc0 0x7f9cac0f1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:24.317 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.329 7f9cabdec700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f9cac0f1d60 0x7f9cac1aa4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:24.317 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.329 7f9cab5eb700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f9cac10e240 0x7f9cac10e690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:33556/0 (socket says 172.21.15.102:33556) 2024-04-17T22:31:24.317 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.329 7f9cab5eb700 1 -- 172.21.15.102:0/2566935734 learned_addr learned my addr 172.21.15.102:0/2566935734 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:31:24.317 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.329 7f9caadea700 1 -- 172.21.15.102:0/2566935734 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f9cac10e240 msgr2=0x7f9cac10e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:31:24.317 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.329 7f9caadea700 1 --2- 172.21.15.102:0/2566935734 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f9cac10e240 0x7f9cac10e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:24.318 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.329 7f9caadea700 1 -- 172.21.15.102:0/2566935734 >> v2:172.21.15.102:3300/0 conn(0x7f9cac0f1d60 msgr2=0x7f9cac1aa4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:31:24.318 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.329 7f9caadea700 1 --2- 172.21.15.102:0/2566935734 >> v2:172.21.15.102:3300/0 conn(0x7f9cac0f1d60 0x7f9cac1aa4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:24.318 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.329 7f9caadea700 1 -- 172.21.15.102:0/2566935734 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cac0f21b0 con 0x7f9cac10ecc0 2024-04-17T22:31:24.318 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.329 7f9cab5eb700 1 --2- 172.21.15.102:0/2566935734 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f9cac10e240 0x7f9cac10e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:24.318 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.329 7f9cabdec700 1 --2- 172.21.15.102:0/2566935734 >> v2:172.21.15.102:3300/0 conn(0x7f9cac0f1d60 0x7f9cac1aa4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:24.318 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.329 7f9caadea700 1 --2- 172.21.15.102:0/2566935734 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f9cac10ecc0 0x7f9cac0f1730 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 rx=0x7f9cac114360 tx=0x7f9c9c00ba20).ready entity=mon.1 client_cookie=74dd192f6bce5659 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:24.318 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.329 7f9caa5e9700 1 -- 172.21.15.102:0/2566935734 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9cac0f21b0 con 0x7f9cac10ecc0 2024-04-17T22:31:24.318 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.329 7f9caa5e9700 1 -- 172.21.15.102:0/2566935734 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c9c00d570 con 0x7f9cac10ecc0 2024-04-17T22:31:24.318 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.329 7f9caa5e9700 1 -- 172.21.15.102:0/2566935734 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9c9c00d940 con 0x7f9cac10ecc0 2024-04-17T22:31:24.319 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.329 7f9cb29c1700 1 -- 172.21.15.102:0/2566935734 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f9cac10ecc0 msgr2=0x7f9cac0f1730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:24.319 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.329 7f9cb29c1700 1 --2- 172.21.15.102:0/2566935734 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f9cac10ecc0 0x7f9cac0f1730 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 rx=0x7f9cac114360 tx=0x7f9c9c00ba20).stop 2024-04-17T22:31:24.319 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.330 7f9cb29c1700 1 -- 172.21.15.102:0/2566935734 shutdown_connections 2024-04-17T22:31:24.319 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.330 7f9cb29c1700 1 --2- 172.21.15.102:0/2566935734 >> v2:172.21.15.102:3300/0 conn(0x7f9cac0f1d60 0x7f9cac1aa4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:24.319 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.330 7f9cb29c1700 1 --2- 172.21.15.102:0/2566935734 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f9cac10e240 0x7f9cac10e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:24.319 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.330 7f9cb29c1700 1 --2- 172.21.15.102:0/2566935734 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f9cac10ecc0 0x7f9cac0f1730 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:24.319 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.330 7f9cb29c1700 1 -- 172.21.15.102:0/2566935734 >> 172.21.15.102:0/2566935734 conn(0x7f9cac1138a0 msgr2=0x7f9cac0f4f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:24.319 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.330 7f9cb29c1700 1 -- 172.21.15.102:0/2566935734 shutdown_connections 2024-04-17T22:31:24.319 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.330 7f9cb29c1700 1 -- 172.21.15.102:0/2566935734 wait complete. 2024-04-17T22:31:24.337 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.352 7f9cb29c1700 1 Processor -- start 2024-04-17T22:31:24.338 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.353 7f9cb29c1700 1 -- start start 2024-04-17T22:31:24.341 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.353 7f9cb29c1700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9cac0cf350 0x7f9cac1ad2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:24.341 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.353 7f9cb29c1700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f9cac1ad8d0 0x7f9cac1b3c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:24.341 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.353 7f9cb29c1700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f9cac1add20 0x7f9cac1b4260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:24.341 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.353 7f9cb29c1700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cac0c41d0 con 0x7f9cac0cf350 2024-04-17T22:31:24.341 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.353 7f9cb29c1700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cac0ccf80 con 0x7f9cac1add20 2024-04-17T22:31:24.341 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.353 7f9cb29c1700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9cac1b4890 con 0x7f9cac1ad8d0 2024-04-17T22:31:24.341 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.353 7f9cab5eb700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9cac0cf350 0x7f9cac1ad2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:24.341 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.353 7f9cab5eb700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9cac0cf350 0x7f9cac1ad2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:51876/0 (socket says 172.21.15.102:51876) 2024-04-17T22:31:24.341 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.353 7f9cab5eb700 1 -- 172.21.15.102:0/2488971207 learned_addr learned my addr 172.21.15.102:0/2488971207 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:31:24.342 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.353 7f9cabdec700 1 --2- 172.21.15.102:0/2488971207 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f9cac1add20 0x7f9cac1b4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:24.342 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.353 7f9caadea700 1 --2- 172.21.15.102:0/2488971207 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f9cac1ad8d0 0x7f9cac1b3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:24.342 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.354 7f9cab5eb700 1 -- 172.21.15.102:0/2488971207 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f9cac1ad8d0 msgr2=0x7f9cac1b3c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:24.342 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.354 7f9cab5eb700 1 --2- 172.21.15.102:0/2488971207 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f9cac1ad8d0 0x7f9cac1b3c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:24.342 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.354 7f9cab5eb700 1 -- 172.21.15.102:0/2488971207 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f9cac1add20 msgr2=0x7f9cac1b4260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:24.342 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.354 7f9cab5eb700 1 --2- 172.21.15.102:0/2488971207 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f9cac1add20 0x7f9cac1b4260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:24.342 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.354 7f9cab5eb700 1 -- 172.21.15.102:0/2488971207 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cac1b4ac0 con 0x7f9cac0cf350 2024-04-17T22:31:24.342 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.354 7f9cabdec700 1 --2- 172.21.15.102:0/2488971207 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f9cac1add20 0x7f9cac1b4260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:24.342 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.354 7f9cab5eb700 1 --2- 172.21.15.102:0/2488971207 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9cac0cf350 0x7f9cac1ad2a0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 rx=0x7f9cac0c5d20 tx=0x7f9ca000eac0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:24.342 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.354 7f9ca8de6700 1 -- 172.21.15.102:0/2488971207 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9cac1b4ac0 con 0x7f9cac0cf350 2024-04-17T22:31:24.343 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.354 7f9ca8de6700 1 -- 172.21.15.102:0/2488971207 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ca0012600 con 0x7f9cac0cf350 2024-04-17T22:31:24.343 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.354 7f9ca8de6700 1 -- 172.21.15.102:0/2488971207 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9ca000f040 con 0x7f9cac0cf350 2024-04-17T22:31:24.343 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.354 7f9cb29c1700 1 -- 172.21.15.102:0/2488971207 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9cac1bb3b0 con 0x7f9cac0cf350 2024-04-17T22:31:24.343 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.354 7f9cb29c1700 1 -- 172.21.15.102:0/2488971207 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cac1bb790 con 0x7f9cac0cf350 2024-04-17T22:31:24.343 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.354 7f9ca8de6700 1 -- 172.21.15.102:0/2488971207 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7f9ca0022060 con 0x7f9cac0cf350 2024-04-17T22:31:24.343 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.354 7f9ca8de6700 1 -- 172.21.15.102:0/2488971207 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 2374+0+0 (secure 0 0 0) 0x7f9ca0010040 con 0x7f9cac0cf350 2024-04-17T22:31:24.343 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.355 7f9cb29c1700 1 -- 172.21.15.102:0/2488971207 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9cac0f0800 con 0x7f9cac0cf350 2024-04-17T22:31:24.343 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.358 7f9ca8de6700 1 -- 172.21.15.102:0/2488971207 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+110455 (secure 0 0 0) 0x7f9ca00123f0 con 0x7f9cac0cf350 2024-04-17T22:31:24.480 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.495 7f9cb29c1700 1 -- 172.21.15.102:0/2488971207 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) v1 -- 0x7f9cac1ae330 con 0x7f9cac0cf350 2024-04-17T22:31:24.485 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.500 7f9ca8de6700 1 -- 172.21.15.102:0/2488971207 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 updated v7) v1 ==== 104+0+0 (secure 0 0 0) 0x7f9ca0000c30 con 0x7f9cac0cf350 2024-04-17T22:31:24.487 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.501 7f9cb29c1700 1 -- 172.21.15.102:0/2488971207 >> v1:172.21.15.102:6816/42360 conn(0x7f9c8c0145a0 legacy=0x7f9c8c016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:24.487 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.501 7f9cb29c1700 1 -- 172.21.15.102:0/2488971207 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9cac0cf350 msgr2=0x7f9cac1ad2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:24.487 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.501 7f9cb29c1700 1 --2- 172.21.15.102:0/2488971207 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9cac0cf350 0x7f9cac1ad2a0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 rx=0x7f9cac0c5d20 tx=0x7f9ca000eac0).stop 2024-04-17T22:31:24.487 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.501 7f9cb29c1700 1 -- 172.21.15.102:0/2488971207 shutdown_connections 2024-04-17T22:31:24.487 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.501 7f9cb29c1700 1 --2- 172.21.15.102:0/2488971207 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f9cac1add20 0x7f9cac1b4260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:24.487 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.501 7f9cb29c1700 1 --2- 172.21.15.102:0/2488971207 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9cac0cf350 0x7f9cac1ad2a0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:24.487 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.501 7f9cb29c1700 1 --2- 172.21.15.102:0/2488971207 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f9cac1ad8d0 0x7f9cac1b3c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:24.487 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.501 7f9cb29c1700 1 -- 172.21.15.102:0/2488971207 >> 172.21.15.102:0/2488971207 conn(0x7f9cac0e0eb0 msgr2=0x7f9cac0dabd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:24.487 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.501 7f9cb29c1700 1 -- 172.21.15.102:0/2488971207 shutdown_connections 2024-04-17T22:31:24.487 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.501 7f9cb29c1700 1 -- 172.21.15.102:0/2488971207 wait complete. 2024-04-17T22:31:24.488 INFO:teuthology.orchestra.run.smithi102.stderr:updated 2024-04-17T22:31:24.497 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2024-04-17T22:31:24.625 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.638 7f4e59f07700 1 Processor -- start 2024-04-17T22:31:24.625 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.638 7f4e59f07700 1 -- start start 2024-04-17T22:31:24.626 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.639 7f4e59f07700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f4e5410e710 0x7f4e5410eb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:24.626 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.639 7f4e59f07700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f4e540f0eb0 0x7f4e540f1300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:24.626 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.639 7f4e59f07700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4e540f1930 0x7f4e541aa240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:24.626 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.639 7f4e59f07700 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e541062a0 con 0x7f4e540f0eb0 2024-04-17T22:31:24.626 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.639 7f4e59f07700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e540fb3f0 con 0x7f4e540f1930 2024-04-17T22:31:24.626 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.639 7f4e59f07700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4e54111c70 con 0x7f4e5410e710 2024-04-17T22:31:24.626 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.639 7f4e524ec700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f4e540f0eb0 0x7f4e540f1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:24.626 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.639 7f4e534ee700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4e540f1930 0x7f4e541aa240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:24.626 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.639 7f4e52ced700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f4e5410e710 0x7f4e5410eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:24.626 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.639 7f4e52ced700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f4e5410e710 0x7f4e5410eb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:33580/0 (socket says 172.21.15.102:33580) 2024-04-17T22:31:24.626 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.639 7f4e534ee700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4e540f1930 0x7f4e541aa240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:60856/0 (socket says 172.21.15.102:60856) 2024-04-17T22:31:24.626 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.639 7f4e52ced700 1 -- 172.21.15.102:0/3539795398 learned_addr learned my addr 172.21.15.102:0/3539795398 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:31:24.626 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.639 7f4e524ec700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f4e540f0eb0 0x7f4e540f1300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:51878/0 (socket says 172.21.15.102:51878) 2024-04-17T22:31:24.626 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.639 7f4e524ec700 1 -- 172.21.15.102:0/3539795398 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f4e5410e710 msgr2=0x7f4e5410eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:31:24.627 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.639 7f4e524ec700 1 --2- 172.21.15.102:0/3539795398 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f4e5410e710 0x7f4e5410eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:24.627 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.639 7f4e524ec700 1 -- 172.21.15.102:0/3539795398 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4e540f1930 msgr2=0x7f4e541aa240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:31:24.627 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.639 7f4e524ec700 1 --2- 172.21.15.102:0/3539795398 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4e540f1930 0x7f4e541aa240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:24.627 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.639 7f4e524ec700 1 -- 172.21.15.102:0/3539795398 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e541aa870 con 0x7f4e540f0eb0 2024-04-17T22:31:24.627 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.639 7f4e534ee700 1 --2- 172.21.15.102:0/3539795398 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4e540f1930 0x7f4e541aa240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:24.627 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.639 7f4e524ec700 1 --2- 172.21.15.102:0/3539795398 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4e540f0eb0 0x7f4e540f1300 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 rx=0x7f4e54104850 tx=0x7f4e4800ba20).ready entity=mon.0 client_cookie=f6fc77b596404a72 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:24.627 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.639 7f4e51ceb700 1 -- 172.21.15.102:0/3539795398 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f4e541aa870 con 0x7f4e540f0eb0 2024-04-17T22:31:24.627 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.639 7f4e51ceb700 1 -- 172.21.15.102:0/3539795398 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e4800d570 con 0x7f4e540f0eb0 2024-04-17T22:31:24.627 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.639 7f4e51ceb700 1 -- 172.21.15.102:0/3539795398 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f4e4800d940 con 0x7f4e540f0eb0 2024-04-17T22:31:24.627 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.639 7f4e59f07700 1 -- 172.21.15.102:0/3539795398 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4e540f0eb0 msgr2=0x7f4e540f1300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:24.627 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.639 7f4e59f07700 1 --2- 172.21.15.102:0/3539795398 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4e540f0eb0 0x7f4e540f1300 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 rx=0x7f4e54104850 tx=0x7f4e4800ba20).stop 2024-04-17T22:31:24.627 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.640 7f4e59f07700 1 -- 172.21.15.102:0/3539795398 shutdown_connections 2024-04-17T22:31:24.627 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.640 7f4e59f07700 1 --2- 172.21.15.102:0/3539795398 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4e540f1930 0x7f4e541aa240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:24.627 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.640 7f4e59f07700 1 --2- 172.21.15.102:0/3539795398 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f4e5410e710 0x7f4e5410eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:24.628 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.640 7f4e59f07700 1 --2- 172.21.15.102:0/3539795398 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4e540f0eb0 0x7f4e540f1300 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:24.628 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.640 7f4e59f07700 1 -- 172.21.15.102:0/3539795398 >> 172.21.15.102:0/3539795398 conn(0x7f4e541146f0 msgr2=0x7f4e54114b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:24.628 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.640 7f4e59f07700 1 -- 172.21.15.102:0/3539795398 shutdown_connections 2024-04-17T22:31:24.628 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.640 7f4e59f07700 1 -- 172.21.15.102:0/3539795398 wait complete. 2024-04-17T22:31:24.650 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.663 7f4e59f07700 1 Processor -- start 2024-04-17T22:31:24.650 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.663 7f4e59f07700 1 -- start start 2024-04-17T22:31:24.650 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.663 7f4e59f07700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4e541b09c0 0x7f4e541b0e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:24.650 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.663 7f4e59f07700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f4e541b1440 0x7f4e541b3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:24.650 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.663 7f4e59f07700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4e541b45b0 0x7f4e541b4a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:24.651 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.663 7f4e59f07700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e541b1890 con 0x7f4e541b45b0 2024-04-17T22:31:24.651 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.663 7f4e59f07700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e540e0380 con 0x7f4e541b09c0 2024-04-17T22:31:24.651 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.663 7f4e59f07700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4e541bb460 con 0x7f4e541b1440 2024-04-17T22:31:24.651 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.663 7f4e52ced700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4e541b09c0 0x7f4e541b0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:24.651 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.663 7f4e524ec700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f4e541b1440 0x7f4e541b3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:24.651 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.663 7f4e52ced700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4e541b09c0 0x7f4e541b0e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:60862/0 (socket says 172.21.15.102:60862) 2024-04-17T22:31:24.651 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.663 7f4e52ced700 1 -- 172.21.15.102:0/4077271151 learned_addr learned my addr 172.21.15.102:0/4077271151 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:31:24.651 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.663 7f4e524ec700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f4e541b1440 0x7f4e541b3f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:33588/0 (socket says 172.21.15.102:33588) 2024-04-17T22:31:24.651 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.663 7f4e534ee700 1 --2- 172.21.15.102:0/4077271151 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4e541b45b0 0x7f4e541b4a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:24.651 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.664 7f4e52ced700 1 -- 172.21.15.102:0/4077271151 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f4e541b1440 msgr2=0x7f4e541b3f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:24.651 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.664 7f4e52ced700 1 --2- 172.21.15.102:0/4077271151 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f4e541b1440 0x7f4e541b3f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:24.651 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.664 7f4e52ced700 1 -- 172.21.15.102:0/4077271151 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4e541b45b0 msgr2=0x7f4e541b4a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:24.651 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.664 7f4e52ced700 1 --2- 172.21.15.102:0/4077271151 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4e541b45b0 0x7f4e541b4a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:24.651 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.664 7f4e52ced700 1 -- 172.21.15.102:0/4077271151 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e541bb610 con 0x7f4e541b09c0 2024-04-17T22:31:24.651 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.664 7f4e524ec700 1 --2- 172.21.15.102:0/4077271151 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f4e541b1440 0x7f4e541b3f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:24.652 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.664 7f4e52ced700 1 --2- 172.21.15.102:0/4077271151 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4e541b09c0 0x7f4e541b0e10 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 rx=0x7f4e540db420 tx=0x7f4e4400ecd0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:24.652 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.664 7f4e43fff700 1 -- 172.21.15.102:0/4077271151 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f4e541bb610 con 0x7f4e541b09c0 2024-04-17T22:31:24.652 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.664 7f4e43fff700 1 -- 172.21.15.102:0/4077271151 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e4400c5a0 con 0x7f4e541b09c0 2024-04-17T22:31:24.652 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.664 7f4e59f07700 1 -- 172.21.15.102:0/4077271151 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e541bb810 con 0x7f4e541b09c0 2024-04-17T22:31:24.652 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.664 7f4e43fff700 1 -- 172.21.15.102:0/4077271151 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f4e4400f040 con 0x7f4e541b09c0 2024-04-17T22:31:24.652 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.664 7f4e59f07700 1 -- 172.21.15.102:0/4077271151 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e541bbbf0 con 0x7f4e541b09c0 2024-04-17T22:31:24.652 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.664 7f4e43fff700 1 -- 172.21.15.102:0/4077271151 <== mon.1 v2:172.21.15.102:3301/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7f4e44022060 con 0x7f4e541b09c0 2024-04-17T22:31:24.652 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.664 7f4e43fff700 1 -- 172.21.15.102:0/4077271151 <== mon.1 v2:172.21.15.102:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 2374+0+0 (secure 0 0 0) 0x7f4e44010040 con 0x7f4e541b09c0 2024-04-17T22:31:24.652 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.665 7f4e59f07700 1 -- 172.21.15.102:0/4077271151 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e541ae680 con 0x7f4e541b09c0 2024-04-17T22:31:24.652 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.668 7f4e43fff700 1 -- 172.21.15.102:0/4077271151 <== mon.1 v2:172.21.15.102:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+110455 (secure 0 0 0) 0x7f4e4400ee10 con 0x7f4e541b09c0 2024-04-17T22:31:24.790 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.805 7f4e59f07700 1 -- 172.21.15.102:0/4077271151 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) v1 -- 0x7f4e540a6510 con 0x7f4e541b09c0 2024-04-17T22:31:24.794 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.809 7f4e43fff700 1 -- 172.21.15.102:0/4077271151 <== mon.1 v2:172.21.15.102:3301/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 updated v8) v1 ==== 97+0+0 (secure 0 0 0) 0x7f4e44000c30 con 0x7f4e541b09c0 2024-04-17T22:31:24.796 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.811 7f4e59f07700 1 -- 172.21.15.102:0/4077271151 >> v1:172.21.15.102:6816/42360 conn(0x7f4e300145a0 legacy=0x7f4e30016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:24.796 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.811 7f4e59f07700 1 -- 172.21.15.102:0/4077271151 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4e541b09c0 msgr2=0x7f4e541b0e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:24.796 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.811 7f4e59f07700 1 --2- 172.21.15.102:0/4077271151 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4e541b09c0 0x7f4e541b0e10 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 rx=0x7f4e540db420 tx=0x7f4e4400ecd0).stop 2024-04-17T22:31:24.796 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.811 7f4e59f07700 1 -- 172.21.15.102:0/4077271151 shutdown_connections 2024-04-17T22:31:24.796 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.811 7f4e59f07700 1 --2- 172.21.15.102:0/4077271151 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4e541b45b0 0x7f4e541b4a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:24.796 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.811 7f4e59f07700 1 --2- 172.21.15.102:0/4077271151 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4e541b09c0 0x7f4e541b0e10 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:24.796 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.811 7f4e59f07700 1 --2- 172.21.15.102:0/4077271151 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f4e541b1440 0x7f4e541b3f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:24.797 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.811 7f4e59f07700 1 -- 172.21.15.102:0/4077271151 >> 172.21.15.102:0/4077271151 conn(0x7f4e540c6d00 msgr2=0x7f4e541a8160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:24.797 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.811 7f4e59f07700 1 -- 172.21.15.102:0/4077271151 shutdown_connections 2024-04-17T22:31:24.797 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.811 7f4e59f07700 1 -- 172.21.15.102:0/4077271151 wait complete. 2024-04-17T22:31:24.797 INFO:teuthology.orchestra.run.smithi102.stderr:updated 2024-04-17T22:31:24.807 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T22:31:24.933 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.947 7f88befbb700 1 Processor -- start 2024-04-17T22:31:24.934 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.947 7f88befbb700 1 -- start start 2024-04-17T22:31:24.934 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.947 7f88befbb700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f88b810b830 0x7f88b81a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:24.934 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.947 7f88befbb700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f88b810bc80 0x7f88b81a7920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:24.934 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.947 7f88befbb700 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88b8103b60 con 0x7f88b80f5860 2024-04-17T22:31:24.934 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.947 7f88befbb700 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88b80fa390 con 0x7f88b810b830 2024-04-17T22:31:24.934 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.947 7f88befbb700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88b81a0b20 con 0x7f88b810bc80 2024-04-17T22:31:24.934 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.947 7f88b6ffd700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f88b810b830 0x7f88b81a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:24.934 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.947 7f88b7fff700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f88b810bc80 0x7f88b81a7920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:24.934 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.947 7f88b6ffd700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f88b810b830 0x7f88b81a52e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:51890/0 (socket says 172.21.15.102:51890) 2024-04-17T22:31:24.934 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.947 7f88b7fff700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f88b810bc80 0x7f88b81a7920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:60874/0 (socket says 172.21.15.102:60874) 2024-04-17T22:31:24.934 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.947 7f88b6ffd700 1 -- 172.21.15.102:0/4094603005 learned_addr learned my addr 172.21.15.102:0/4094603005 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:31:24.934 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.947 7f88b77fe700 1 --1- 172.21.15.102:0/4094603005 >> v1:172.21.15.102:6789/0 conn(0x7f88b80f5860 0x7f88b810b030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:33678/0 (socket says 172.21.15.102:33678) 2024-04-17T22:31:24.934 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.947 7f88b6ffd700 1 -- 172.21.15.102:0/4094603005 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f88b810bc80 msgr2=0x7f88b81a7920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:31:24.935 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.947 7f88b6ffd700 1 --2- 172.21.15.102:0/4094603005 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f88b810bc80 0x7f88b81a7920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:24.935 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.947 7f88b6ffd700 1 -- 172.21.15.102:0/4094603005 >> v1:172.21.15.102:6789/0 conn(0x7f88b80f5860 legacy=0x7f88b810b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:24.935 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.947 7f88b67fc700 1 -- 172.21.15.102:0/4094603005 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3896224856 0 0) 0x7f88b0001110 con 0x7f88b80f5860 2024-04-17T22:31:24.935 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.947 7f88b6ffd700 1 -- 172.21.15.102:0/4094603005 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88b81a8f60 con 0x7f88b810b830 2024-04-17T22:31:24.935 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.947 7f88b7fff700 1 --2- 172.21.15.102:0/4094603005 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f88b810bc80 0x7f88b81a7920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:24.935 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.948 7f88b6ffd700 1 --2- 172.21.15.102:0/4094603005 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f88b810b830 0x7f88b81a52e0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 rx=0x7f88a0002f70 tx=0x7f88a000de80).ready entity=mon.0 client_cookie=7d92f5a5003039bf server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:24.935 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.948 7f88b67fc700 1 -- 172.21.15.102:0/4094603005 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f88b81a8f60 con 0x7f88b810b830 2024-04-17T22:31:24.935 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.948 7f88b67fc700 1 -- 172.21.15.102:0/4094603005 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88a000fb70 con 0x7f88b810b830 2024-04-17T22:31:24.935 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.948 7f88b67fc700 1 -- 172.21.15.102:0/4094603005 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f88a0013070 con 0x7f88b810b830 2024-04-17T22:31:24.935 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.948 7f88befbb700 1 -- 172.21.15.102:0/4094603005 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f88b810b830 msgr2=0x7f88b81a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:24.935 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.948 7f88befbb700 1 --2- 172.21.15.102:0/4094603005 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f88b810b830 0x7f88b81a52e0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 rx=0x7f88a0002f70 tx=0x7f88a000de80).stop 2024-04-17T22:31:24.935 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.948 7f88befbb700 1 -- 172.21.15.102:0/4094603005 shutdown_connections 2024-04-17T22:31:24.935 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.948 7f88befbb700 1 --2- 172.21.15.102:0/4094603005 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f88b810bc80 0x7f88b81a7920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:24.935 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.948 7f88befbb700 1 --2- 172.21.15.102:0/4094603005 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f88b810b830 0x7f88b81a52e0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:24.935 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.948 7f88befbb700 1 -- 172.21.15.102:0/4094603005 >> 172.21.15.102:0/4094603005 conn(0x7f88b81138c0 msgr2=0x7f88b80f4f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:24.936 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.948 7f88befbb700 1 -- 172.21.15.102:0/4094603005 shutdown_connections 2024-04-17T22:31:24.936 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.948 7f88befbb700 1 -- 172.21.15.102:0/4094603005 wait complete. 2024-04-17T22:31:24.961 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.971 7f88befbb700 1 Processor -- start 2024-04-17T22:31:24.961 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.971 7f88befbb700 1 -- start start 2024-04-17T22:31:24.962 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.971 7f88befbb700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f88b80ca0e0 0x7f88b80d5410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:24.962 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.971 7f88befbb700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f88b80d5a40 0x7f88b81afd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:24.962 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.971 7f88befbb700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f88b81b03b0 0x7f88b81b0800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:24.962 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.971 7f88befbb700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88b80cd7d0 con 0x7f88b81b03b0 2024-04-17T22:31:24.962 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.971 7f88befbb700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88b80d5e90 con 0x7f88b80d5a40 2024-04-17T22:31:24.963 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.971 7f88befbb700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f88b81b0e30 con 0x7f88b80ca0e0 2024-04-17T22:31:24.963 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.971 7f88b6ffd700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f88b80d5a40 0x7f88b81afd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:24.963 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.971 7f88b77fe700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f88b80ca0e0 0x7f88b80d5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:24.963 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.971 7f88b6ffd700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f88b80d5a40 0x7f88b81afd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:60886/0 (socket says 172.21.15.102:60886) 2024-04-17T22:31:24.963 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.971 7f88b6ffd700 1 -- 172.21.15.102:0/1737171844 learned_addr learned my addr 172.21.15.102:0/1737171844 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:31:24.963 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.971 7f88b7fff700 1 --2- 172.21.15.102:0/1737171844 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f88b81b03b0 0x7f88b81b0800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:24.963 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.972 7f88b6ffd700 1 -- 172.21.15.102:0/1737171844 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f88b80ca0e0 msgr2=0x7f88b80d5410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:24.964 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.972 7f88b6ffd700 1 --2- 172.21.15.102:0/1737171844 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f88b80ca0e0 0x7f88b80d5410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:24.964 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.972 7f88b6ffd700 1 -- 172.21.15.102:0/1737171844 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f88b81b03b0 msgr2=0x7f88b81b0800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:24.964 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.972 7f88b6ffd700 1 --2- 172.21.15.102:0/1737171844 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f88b81b03b0 0x7f88b81b0800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:24.964 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.972 7f88b6ffd700 1 -- 172.21.15.102:0/1737171844 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88b80cbb00 con 0x7f88b80d5a40 2024-04-17T22:31:24.964 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.972 7f88b77fe700 1 --2- 172.21.15.102:0/1737171844 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f88b80ca0e0 0x7f88b80d5410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:24.964 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.972 7f88b6ffd700 1 --2- 172.21.15.102:0/1737171844 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f88b80d5a40 0x7f88b81afd80 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 rx=0x7f88b80d6040 tx=0x7f88a00068c0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:24.965 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.972 7f88b4ff9700 1 -- 172.21.15.102:0/1737171844 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f88b80cbb00 con 0x7f88b80d5a40 2024-04-17T22:31:24.965 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.972 7f88b4ff9700 1 -- 172.21.15.102:0/1737171844 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88a0016dc0 con 0x7f88b80d5a40 2024-04-17T22:31:24.965 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.972 7f88befbb700 1 -- 172.21.15.102:0/1737171844 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88b81b7260 con 0x7f88b80d5a40 2024-04-17T22:31:24.965 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.972 7f88b4ff9700 1 -- 172.21.15.102:0/1737171844 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f88a001c040 con 0x7f88b80d5a40 2024-04-17T22:31:24.965 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.972 7f88befbb700 1 -- 172.21.15.102:0/1737171844 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88b81b7640 con 0x7f88b80d5a40 2024-04-17T22:31:24.965 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.972 7f88b4ff9700 1 -- 172.21.15.102:0/1737171844 <== mon.1 v2:172.21.15.102:3301/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7f88a0021660 con 0x7f88b80d5a40 2024-04-17T22:31:24.966 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.972 7f88b4ff9700 1 -- 172.21.15.102:0/1737171844 <== mon.1 v2:172.21.15.102:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 2374+0+0 (secure 0 0 0) 0x7f88a001d030 con 0x7f88b80d5a40 2024-04-17T22:31:24.966 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.973 7f88befbb700 1 -- 172.21.15.102:0/1737171844 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88b80f0820 con 0x7f88b80d5a40 2024-04-17T22:31:24.966 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:24.976 7f88b4ff9700 1 -- 172.21.15.102:0/1737171844 <== mon.1 v2:172.21.15.102:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+110455 (secure 0 0 0) 0x7f88a0016bb0 con 0x7f88b80d5a40 2024-04-17T22:31:25.066 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.081 7f88befbb700 1 -- 172.21.15.102:0/1737171844 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f88b81aa500 con 0x7f88b80d5a40 2024-04-17T22:31:25.066 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.081 7f88b4ff9700 1 -- 172.21.15.102:0/1737171844 <== mon.1 v2:172.21.15.102:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+5907 (secure 0 0 0) 0x7f88a000f770 con 0x7f88b80d5a40 2024-04-17T22:31:25.067 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-17T22:31:25.067 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":21,"fsid":"cb63e789-091d-49b1-b120-55f9db526871","created":"2024-04-17 22:28:48.209635","modified":"2024-04-17 22:31:22.620264","last_up_change":"2024-04-17 22:28:54.236215","last_in_change":"2024-04-17 22:28:54.236215","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":4,"require_min_compat_client":"nautilus","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-04-17 22:28:57.478545","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"3351ef85-ade4-4b90-a8f4-213f4e00ba01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6812","nonce":42955}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6813","nonce":42955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6815","nonce":42955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6814","nonce":42955}]},"public_addr":"172.21.15.102:6812/42955","cluster_addr":"172.21.15.102:6813/42955","heartbeat_back_addr":"172.21.15.102:6815/42955","heartbeat_front_addr":"172.21.15.102:6814/42955","state":["exists","up"]},{"osd":1,"uuid":"04f67eac-f36b-48f5-982d-4cfc776596d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6808","nonce":42958}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6809","nonce":42958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6811","nonce":42958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6810","nonce":42958}]},"public_addr":"172.21.15.102:6808/42958","cluster_addr":"172.21.15.102:6809/42958","heartbeat_back_addr":"172.21.15.102:6811/42958","heartbeat_front_addr":"172.21.15.102:6810/42958","state":["exists","up"]},{"osd":2,"uuid":"d524204b-70b2-40a8-8f73-f9e7d44318af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6804","nonce":42954}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6805","nonce":42954}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6807","nonce":42954}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6806","nonce":42954}]},"public_addr":"172.21.15.102:6804/42954","cluster_addr":"172.21.15.102:6805/42954","heartbeat_back_addr":"172.21.15.102:6807/42954","heartbeat_front_addr":"172.21.15.102:6806/42954","state":["exists","up"]},{"osd":3,"uuid":"4fc9d20c-1ad0-484f-aafa-2c9a7574245b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6800","nonce":42953}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6801","nonce":42953}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6803","nonce":42953}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6802","nonce":42953}]},"public_addr":"172.21.15.102:6800/42953","cluster_addr":"172.21.15.102:6801/42953","heartbeat_back_addr":"172.21.15.102:6803/42953","heartbeat_front_addr":"172.21.15.102:6802/42953","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2024-04-17T22:31:25.068 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.083 7f88befbb700 1 -- 172.21.15.102:0/1737171844 >> v1:172.21.15.102:6816/42360 conn(0x7f889c0145a0 legacy=0x7f889c016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:25.068 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.083 7f88befbb700 1 -- 172.21.15.102:0/1737171844 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f88b80d5a40 msgr2=0x7f88b81afd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:25.068 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.083 7f88befbb700 1 --2- 172.21.15.102:0/1737171844 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f88b80d5a40 0x7f88b81afd80 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 rx=0x7f88b80d6040 tx=0x7f88a00068c0).stop 2024-04-17T22:31:25.068 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.083 7f88befbb700 1 -- 172.21.15.102:0/1737171844 shutdown_connections 2024-04-17T22:31:25.068 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.083 7f88befbb700 1 --2- 172.21.15.102:0/1737171844 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f88b81b03b0 0x7f88b81b0800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:25.068 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.083 7f88befbb700 1 --2- 172.21.15.102:0/1737171844 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f88b80ca0e0 0x7f88b80d5410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:25.068 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.083 7f88befbb700 1 --2- 172.21.15.102:0/1737171844 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f88b80d5a40 0x7f88b81afd80 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:25.069 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.083 7f88befbb700 1 -- 172.21.15.102:0/1737171844 >> 172.21.15.102:0/1737171844 conn(0x7f88b80c7c40 msgr2=0x7f88b80c1fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:25.069 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.083 7f88befbb700 1 -- 172.21.15.102:0/1737171844 shutdown_connections 2024-04-17T22:31:25.069 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.083 7f88befbb700 1 -- 172.21.15.102:0/1737171844 wait complete. 2024-04-17T22:31:25.079 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2024-04-17 22:28:57.478545', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '12', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2024-04-17T22:31:25.079 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2024-04-17T22:31:25.208 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.221 7fa634158700 1 Processor -- start 2024-04-17T22:31:25.209 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.221 7fa634158700 1 -- start start 2024-04-17T22:31:25.209 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.221 7fa634158700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa62c0f5840 0x7fa62c10b010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:25.209 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.221 7fa634158700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fa62c10b640 0x7fa62c19faf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:25.209 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.221 7fa634158700 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa62c103b40 con 0x7fa62c10ba90 2024-04-17T22:31:25.209 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.221 7fa634158700 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa62c10bee0 con 0x7fa62c10b640 2024-04-17T22:31:25.209 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.221 7fa634158700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa62c0fa370 con 0x7fa62c0f5840 2024-04-17T22:31:25.209 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.221 7fa630f53700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa62c0f5840 0x7fa62c10b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:25.209 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.221 7fa631754700 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7fa62c10ba90 0x7fa62c1a0120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:33686/0 (socket says 172.21.15.102:33686) 2024-04-17T22:31:25.209 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.221 7fa631754700 1 -- 172.21.15.102:0/1034865708 learned_addr learned my addr 172.21.15.102:0/1034865708 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-17T22:31:25.209 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.221 7fa62bfff700 1 --2- 172.21.15.102:0/1034865708 >> v2:172.21.15.102:3300/0 conn(0x7fa62c10b640 0x7fa62c19faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:25.209 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.222 7fa62b7fe700 1 -- 172.21.15.102:0/1034865708 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3742757703 0 0) 0x7fa624001490 con 0x7fa62c10ba90 2024-04-17T22:31:25.209 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.222 7fa62b7fe700 1 -- 172.21.15.102:0/1034865708 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa610003270 con 0x7fa62c10ba90 2024-04-17T22:31:25.209 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.222 7fa62bfff700 1 -- 172.21.15.102:0/1034865708 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa62c0f5840 msgr2=0x7fa62c10b010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:31:25.210 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.222 7fa62bfff700 1 --2- 172.21.15.102:0/1034865708 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa62c0f5840 0x7fa62c10b010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:25.210 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.222 7fa62bfff700 1 -- 172.21.15.102:0/1034865708 >> v1:172.21.15.102:6789/0 conn(0x7fa62c10ba90 legacy=0x7fa62c1a0120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:25.210 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.222 7fa62bfff700 1 -- 172.21.15.102:0/1034865708 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa62c1a0920 con 0x7fa62c10b640 2024-04-17T22:31:25.210 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.222 7fa630f53700 1 --2- 172.21.15.102:0/1034865708 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa62c0f5840 0x7fa62c10b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:25.210 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.222 7fa62bfff700 1 --2- 172.21.15.102:0/1034865708 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa62c10b640 0x7fa62c19faf0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 rx=0x7fa618002f70 tx=0x7fa61800b9f0).ready entity=mon.0 client_cookie=6a49658fc8317f3f server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:25.210 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.222 7fa62b7fe700 1 -- 172.21.15.102:0/1034865708 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa62c1a0920 con 0x7fa62c10b640 2024-04-17T22:31:25.210 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.222 7fa62b7fe700 1 -- 172.21.15.102:0/1034865708 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa61800d5c0 con 0x7fa62c10b640 2024-04-17T22:31:25.210 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.222 7fa62b7fe700 1 -- 172.21.15.102:0/1034865708 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa61800d990 con 0x7fa62c10b640 2024-04-17T22:31:25.210 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.222 7fa634158700 1 -- 172.21.15.102:0/1034865708 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa62c10b640 msgr2=0x7fa62c19faf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:25.210 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.222 7fa634158700 1 --2- 172.21.15.102:0/1034865708 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa62c10b640 0x7fa62c19faf0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 rx=0x7fa618002f70 tx=0x7fa61800b9f0).stop 2024-04-17T22:31:25.210 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.222 7fa634158700 1 -- 172.21.15.102:0/1034865708 shutdown_connections 2024-04-17T22:31:25.210 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.223 7fa634158700 1 --2- 172.21.15.102:0/1034865708 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa62c0f5840 0x7fa62c10b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:25.210 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.223 7fa634158700 1 --2- 172.21.15.102:0/1034865708 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa62c10b640 0x7fa62c19faf0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:25.211 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.223 7fa634158700 1 -- 172.21.15.102:0/1034865708 >> 172.21.15.102:0/1034865708 conn(0x7fa62c1138a0 msgr2=0x7fa62c0f4f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:25.211 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.223 7fa634158700 1 -- 172.21.15.102:0/1034865708 shutdown_connections 2024-04-17T22:31:25.211 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.223 7fa634158700 1 -- 172.21.15.102:0/1034865708 wait complete. 2024-04-17T22:31:25.233 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.245 7fa634158700 1 Processor -- start 2024-04-17T22:31:25.233 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.245 7fa634158700 1 -- start start 2024-04-17T22:31:25.233 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.246 7fa634158700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fa62c0ce1e0 0x7fa62c0ce630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:25.233 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.246 7fa634158700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa62c0cec60 0x7fa62c0d9660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:25.233 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.246 7fa634158700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa62c0d9c90 0x7fa62c1bb7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:25.233 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.246 7fa634158700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa62c0bf520 con 0x7fa62c0d9c90 2024-04-17T22:31:25.233 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.246 7fa634158700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa62c0db390 con 0x7fa62c0cec60 2024-04-17T22:31:25.233 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.246 7fa634158700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa62c0de2b0 con 0x7fa62c0ce1e0 2024-04-17T22:31:25.233 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.246 7fa62bfff700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa62c0cec60 0x7fa62c0d9660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:25.233 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.246 7fa631754700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa62c0d9c90 0x7fa62c1bb7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:25.233 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.246 7fa62bfff700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa62c0cec60 0x7fa62c0d9660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:60902/0 (socket says 172.21.15.102:60902) 2024-04-17T22:31:25.234 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.246 7fa631754700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa62c0d9c90 0x7fa62c1bb7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:51914/0 (socket says 172.21.15.102:51914) 2024-04-17T22:31:25.234 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.246 7fa62bfff700 1 -- 172.21.15.102:0/3076686778 learned_addr learned my addr 172.21.15.102:0/3076686778 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:31:25.234 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.246 7fa630f53700 1 --2- 172.21.15.102:0/3076686778 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fa62c0ce1e0 0x7fa62c0ce630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:25.234 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.246 7fa62bfff700 1 -- 172.21.15.102:0/3076686778 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fa62c0ce1e0 msgr2=0x7fa62c0ce630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:25.234 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.246 7fa62bfff700 1 --2- 172.21.15.102:0/3076686778 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fa62c0ce1e0 0x7fa62c0ce630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:25.234 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.246 7fa62bfff700 1 -- 172.21.15.102:0/3076686778 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa62c0d9c90 msgr2=0x7fa62c1bb7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:25.234 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.246 7fa62bfff700 1 --2- 172.21.15.102:0/3076686778 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa62c0d9c90 0x7fa62c1bb7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:25.234 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.246 7fa62bfff700 1 -- 172.21.15.102:0/3076686778 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa62c0de6a0 con 0x7fa62c0cec60 2024-04-17T22:31:25.234 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.246 7fa630f53700 1 --2- 172.21.15.102:0/3076686778 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fa62c0ce1e0 0x7fa62c0ce630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:25.234 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.246 7fa631754700 1 --2- 172.21.15.102:0/3076686778 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa62c0d9c90 0x7fa62c1bb7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:25.235 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.246 7fa62bfff700 1 --2- 172.21.15.102:0/3076686778 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa62c0cec60 0x7fa62c0d9660 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 rx=0x7fa62c0d1600 tx=0x7fa6180068c0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:25.235 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.246 7fa629ffb700 1 -- 172.21.15.102:0/3076686778 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa62c0de6a0 con 0x7fa62c0cec60 2024-04-17T22:31:25.235 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.246 7fa629ffb700 1 -- 172.21.15.102:0/3076686778 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa618006a90 con 0x7fa62c0cec60 2024-04-17T22:31:25.235 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.247 7fa629ffb700 1 -- 172.21.15.102:0/3076686778 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa618017040 con 0x7fa62c0cec60 2024-04-17T22:31:25.235 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.247 7fa634158700 1 -- 172.21.15.102:0/3076686778 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa62c0da0e0 con 0x7fa62c0cec60 2024-04-17T22:31:25.235 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.247 7fa634158700 1 -- 172.21.15.102:0/3076686778 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa62c1bbe20 con 0x7fa62c0cec60 2024-04-17T22:31:25.235 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.247 7fa629ffb700 1 -- 172.21.15.102:0/3076686778 <== mon.1 v2:172.21.15.102:3301/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7fa618011be0 con 0x7fa62c0cec60 2024-04-17T22:31:25.235 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.247 7fa629ffb700 1 -- 172.21.15.102:0/3076686778 <== mon.1 v2:172.21.15.102:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 2374+0+0 (secure 0 0 0) 0x7fa618018070 con 0x7fa62c0cec60 2024-04-17T22:31:25.235 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.248 7fa634158700 1 -- 172.21.15.102:0/3076686778 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa62c0f0800 con 0x7fa62c0cec60 2024-04-17T22:31:25.236 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.251 7fa629ffb700 1 -- 172.21.15.102:0/3076686778 <== mon.1 v2:172.21.15.102:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+110455 (secure 0 0 0) 0x7fa6180117e0 con 0x7fa62c0cec60 2024-04-17T22:31:25.340 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.355 7fa634158700 1 -- 172.21.15.102:0/3076686778 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fa62c0efed0 con 0x7fa62c0cec60 2024-04-17T22:31:25.340 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.355 7fa629ffb700 1 -- 172.21.15.102:0/3076686778 <== mon.1 v2:172.21.15.102:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v21) v1 ==== 92+0+10 (secure 0 0 0) 0x7fa61800d3b0 con 0x7fa62c0cec60 2024-04-17T22:31:25.341 INFO:teuthology.orchestra.run.smithi102.stdout:pg_num: 8 2024-04-17T22:31:25.342 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.356 7fa634158700 1 -- 172.21.15.102:0/3076686778 >> v1:172.21.15.102:6816/42360 conn(0x7fa6140145a0 legacy=0x7fa614016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:25.342 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.356 7fa634158700 1 -- 172.21.15.102:0/3076686778 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa62c0cec60 msgr2=0x7fa62c0d9660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:25.342 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.356 7fa634158700 1 --2- 172.21.15.102:0/3076686778 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa62c0cec60 0x7fa62c0d9660 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 rx=0x7fa62c0d1600 tx=0x7fa6180068c0).stop 2024-04-17T22:31:25.343 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.356 7fa634158700 1 -- 172.21.15.102:0/3076686778 shutdown_connections 2024-04-17T22:31:25.343 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.356 7fa634158700 1 --2- 172.21.15.102:0/3076686778 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa62c0d9c90 0x7fa62c1bb7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:25.343 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.356 7fa634158700 1 --2- 172.21.15.102:0/3076686778 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fa62c0ce1e0 0x7fa62c0ce630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:25.343 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.356 7fa634158700 1 --2- 172.21.15.102:0/3076686778 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa62c0cec60 0x7fa62c0d9660 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:25.343 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.356 7fa634158700 1 -- 172.21.15.102:0/3076686778 >> 172.21.15.102:0/3076686778 conn(0x7fa62c0bf0b0 msgr2=0x7fa62c0c0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:25.343 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.356 7fa634158700 1 -- 172.21.15.102:0/3076686778 shutdown_connections 2024-04-17T22:31:25.343 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.356 7fa634158700 1 -- 172.21.15.102:0/3076686778 wait complete. 2024-04-17T22:31:25.352 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-17T22:31:25.480 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.492 7fc98ab87700 1 Processor -- start 2024-04-17T22:31:25.480 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.492 7fc98ab87700 1 -- start start 2024-04-17T22:31:25.480 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.492 7fc98ab87700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fc9840f57d0 0x7fc98410afa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:25.480 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.492 7fc98ab87700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc98410ba20 0x7fc9841a0280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:25.480 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.492 7fc98ab87700 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc984103ad0 con 0x7fc98410b5d0 2024-04-17T22:31:25.480 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.492 7fc98ab87700 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc98410be70 con 0x7fc9840f57d0 2024-04-17T22:31:25.480 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.492 7fc98ab87700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9840fa300 con 0x7fc98410ba20 2024-04-17T22:31:25.480 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.492 7fc983fff700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc98410ba20 0x7fc9841a0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:25.480 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.493 7fc9837fe700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fc9840f57d0 0x7fc98410afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:25.480 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.493 7fc9837fe700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fc9840f57d0 0x7fc98410afa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:51924/0 (socket says 172.21.15.102:51924) 2024-04-17T22:31:25.481 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.493 7fc983fff700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc98410ba20 0x7fc9841a0280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:60904/0 (socket says 172.21.15.102:60904) 2024-04-17T22:31:25.481 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.493 7fc9837fe700 1 -- 172.21.15.102:0/1804301742 learned_addr learned my addr 172.21.15.102:0/1804301742 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:31:25.481 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.493 7fc982ffd700 1 --1- 172.21.15.102:0/1804301742 >> v1:172.21.15.102:6789/0 conn(0x7fc98410b5d0 0x7fc98419fa80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:33688/0 (socket says 172.21.15.102:33688) 2024-04-17T22:31:25.481 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.493 7fc983fff700 1 -- 172.21.15.102:0/1804301742 >> v2:172.21.15.102:3300/0 conn(0x7fc9840f57d0 msgr2=0x7fc98410afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:31:25.481 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.493 7fc983fff700 1 --2- 172.21.15.102:0/1804301742 >> v2:172.21.15.102:3300/0 conn(0x7fc9840f57d0 0x7fc98410afa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:25.481 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.493 7fc983fff700 1 -- 172.21.15.102:0/1804301742 >> v1:172.21.15.102:6789/0 conn(0x7fc98410b5d0 legacy=0x7fc98419fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:25.481 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.493 7fc983fff700 1 -- 172.21.15.102:0/1804301742 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9841a08b0 con 0x7fc98410ba20 2024-04-17T22:31:25.481 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.493 7fc9837fe700 1 --2- 172.21.15.102:0/1804301742 >> v2:172.21.15.102:3300/0 conn(0x7fc9840f57d0 0x7fc98410afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:25.481 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.493 7fc983fff700 1 --2- 172.21.15.102:0/1804301742 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc98410ba20 0x7fc9841a0280 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 rx=0x7fc9840f4400 tx=0x7fc96c009cc0).ready entity=mon.1 client_cookie=518ccfdb2112808d server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:25.481 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.493 7fc9827fc700 1 -- 172.21.15.102:0/1804301742 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc9841a08b0 con 0x7fc98410ba20 2024-04-17T22:31:25.481 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.493 7fc9827fc700 1 -- 172.21.15.102:0/1804301742 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc96c00b840 con 0x7fc98410ba20 2024-04-17T22:31:25.481 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.493 7fc9827fc700 1 -- 172.21.15.102:0/1804301742 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc96c00ba20 con 0x7fc98410ba20 2024-04-17T22:31:25.481 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.493 7fc98ab87700 1 -- 172.21.15.102:0/1804301742 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc98410ba20 msgr2=0x7fc9841a0280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:25.482 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.493 7fc98ab87700 1 --2- 172.21.15.102:0/1804301742 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc98410ba20 0x7fc9841a0280 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 rx=0x7fc9840f4400 tx=0x7fc96c009cc0).stop 2024-04-17T22:31:25.482 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.494 7fc98ab87700 1 -- 172.21.15.102:0/1804301742 shutdown_connections 2024-04-17T22:31:25.482 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.494 7fc98ab87700 1 --2- 172.21.15.102:0/1804301742 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc98410ba20 0x7fc9841a0280 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:25.482 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.494 7fc98ab87700 1 --2- 172.21.15.102:0/1804301742 >> v2:172.21.15.102:3300/0 conn(0x7fc9840f57d0 0x7fc98410afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:25.482 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.494 7fc98ab87700 1 -- 172.21.15.102:0/1804301742 >> 172.21.15.102:0/1804301742 conn(0x7fc984113830 msgr2=0x7fc9840f4ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:25.482 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.494 7fc98ab87700 1 -- 172.21.15.102:0/1804301742 shutdown_connections 2024-04-17T22:31:25.482 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.494 7fc98ab87700 1 -- 172.21.15.102:0/1804301742 wait complete. 2024-04-17T22:31:25.507 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.516 7fc98ab87700 1 Processor -- start 2024-04-17T22:31:25.507 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.517 7fc98ab87700 1 -- start start 2024-04-17T22:31:25.507 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.517 7fc98ab87700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc9840d7170 0x7fc9840ce1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:25.507 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.517 7fc98ab87700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc9840ce810 0x7fc9841aff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:25.507 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.517 7fc98ab87700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fc9840cec60 0x7fc9841b05b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:25.508 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.517 7fc98ab87700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9840dcfa0 con 0x7fc9840ce810 2024-04-17T22:31:25.508 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.517 7fc98ab87700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9840d6a20 con 0x7fc9840d7170 2024-04-17T22:31:25.508 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.517 7fc98ab87700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc9841b0be0 con 0x7fc9840cec60 2024-04-17T22:31:25.508 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.517 7fc9837fe700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc9840d7170 0x7fc9840ce1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:25.508 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.517 7fc9837fe700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc9840d7170 0x7fc9840ce1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:60910/0 (socket says 172.21.15.102:60910) 2024-04-17T22:31:25.509 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.517 7fc9837fe700 1 -- 172.21.15.102:0/1646883485 learned_addr learned my addr 172.21.15.102:0/1646883485 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:31:25.509 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.517 7fc982ffd700 1 --2- 172.21.15.102:0/1646883485 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc9840ce810 0x7fc9841aff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:25.509 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.517 7fc983fff700 1 --2- 172.21.15.102:0/1646883485 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fc9840cec60 0x7fc9841b05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:25.509 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.517 7fc9837fe700 1 -- 172.21.15.102:0/1646883485 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fc9840cec60 msgr2=0x7fc9841b05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:25.509 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.517 7fc9837fe700 1 --2- 172.21.15.102:0/1646883485 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fc9840cec60 0x7fc9841b05b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:25.510 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.517 7fc9837fe700 1 -- 172.21.15.102:0/1646883485 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc9840ce810 msgr2=0x7fc9841aff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:25.510 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.517 7fc9837fe700 1 --2- 172.21.15.102:0/1646883485 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc9840ce810 0x7fc9841aff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:25.510 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.517 7fc9837fe700 1 -- 172.21.15.102:0/1646883485 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9841b0e10 con 0x7fc9840d7170 2024-04-17T22:31:25.510 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.517 7fc982ffd700 1 --2- 172.21.15.102:0/1646883485 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc9840ce810 0x7fc9841aff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:25.510 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.517 7fc9837fe700 1 --2- 172.21.15.102:0/1646883485 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc9840d7170 0x7fc9840ce1e0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 rx=0x7fc97c0067f0 tx=0x7fc97c00e640).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:25.511 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.518 7fc980ff9700 1 -- 172.21.15.102:0/1646883485 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc9841b0e10 con 0x7fc9840d7170 2024-04-17T22:31:25.511 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.518 7fc980ff9700 1 -- 172.21.15.102:0/1646883485 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc97c01a470 con 0x7fc9840d7170 2024-04-17T22:31:25.511 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.518 7fc98ab87700 1 -- 172.21.15.102:0/1646883485 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9841b7780 con 0x7fc9840d7170 2024-04-17T22:31:25.511 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.518 7fc980ff9700 1 -- 172.21.15.102:0/1646883485 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fc97c01a840 con 0x7fc9840d7170 2024-04-17T22:31:25.511 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.518 7fc98ab87700 1 -- 172.21.15.102:0/1646883485 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9841b7b60 con 0x7fc9840d7170 2024-04-17T22:31:25.511 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.518 7fc980ff9700 1 -- 172.21.15.102:0/1646883485 <== mon.1 v2:172.21.15.102:3301/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7fc97c01aa40 con 0x7fc9840d7170 2024-04-17T22:31:25.512 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.518 7fc980ff9700 1 -- 172.21.15.102:0/1646883485 <== mon.1 v2:172.21.15.102:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 2374+0+0 (secure 0 0 0) 0x7fc97c012a00 con 0x7fc9840d7170 2024-04-17T22:31:25.512 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.519 7fc98ab87700 1 -- 172.21.15.102:0/1646883485 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9840f0790 con 0x7fc9840d7170 2024-04-17T22:31:25.512 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.522 7fc980ff9700 1 -- 172.21.15.102:0/1646883485 <== mon.1 v2:172.21.15.102:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+110455 (secure 0 0 0) 0x7fc97c00ebd0 con 0x7fc9840d7170 2024-04-17T22:31:25.612 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.627 7fc98ab87700 1 -- 172.21.15.102:0/1646883485 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fc9841aa6e0 con 0x7fc9840d7170 2024-04-17T22:31:25.612 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.627 7fc980ff9700 1 -- 172.21.15.102:0/1646883485 <== mon.1 v2:172.21.15.102:3301/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v21) v1 ==== 84+0+64 (secure 0 0 0) 0x7fc97c000c30 con 0x7fc9840d7170 2024-04-17T22:31:25.613 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-17T22:31:25.613 INFO:teuthology.orchestra.run.smithi102.stdout:[ 2024-04-17T22:31:25.613 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-04-17T22:31:25.613 INFO:teuthology.orchestra.run.smithi102.stdout: "poolnum": 1, 2024-04-17T22:31:25.613 INFO:teuthology.orchestra.run.smithi102.stdout: "poolname": "rbd" 2024-04-17T22:31:25.613 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-17T22:31:25.613 INFO:teuthology.orchestra.run.smithi102.stdout:] 2024-04-17T22:31:25.614 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.628 7fc98ab87700 1 -- 172.21.15.102:0/1646883485 >> v1:172.21.15.102:6816/42360 conn(0x7fc95c0145a0 legacy=0x7fc95c016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:25.614 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.628 7fc98ab87700 1 -- 172.21.15.102:0/1646883485 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc9840d7170 msgr2=0x7fc9840ce1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:25.614 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.628 7fc98ab87700 1 --2- 172.21.15.102:0/1646883485 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc9840d7170 0x7fc9840ce1e0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 rx=0x7fc97c0067f0 tx=0x7fc97c00e640).stop 2024-04-17T22:31:25.614 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.628 7fc98ab87700 1 -- 172.21.15.102:0/1646883485 shutdown_connections 2024-04-17T22:31:25.614 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.628 7fc98ab87700 1 --2- 172.21.15.102:0/1646883485 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fc9840cec60 0x7fc9841b05b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:25.614 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.628 7fc98ab87700 1 --2- 172.21.15.102:0/1646883485 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc9840d7170 0x7fc9840ce1e0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:25.615 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.628 7fc98ab87700 1 --2- 172.21.15.102:0/1646883485 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc9840ce810 0x7fc9841aff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:25.615 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.628 7fc98ab87700 1 -- 172.21.15.102:0/1646883485 >> 172.21.15.102:0/1646883485 conn(0x7fc9840c3360 msgr2=0x7fc9841a0fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:25.615 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.629 7fc98ab87700 1 -- 172.21.15.102:0/1646883485 shutdown_connections 2024-04-17T22:31:25.615 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.629 7fc98ab87700 1 -- 172.21.15.102:0/1646883485 wait complete. 2024-04-17T22:31:25.625 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-17T22:31:25.625 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-17T22:31:25.756 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.768 7f4f5f739700 1 Processor -- start 2024-04-17T22:31:25.756 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.768 7f4f5f739700 1 -- start start 2024-04-17T22:31:25.756 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.768 7f4f5f739700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f4f580fab20 0x7f4f5810de30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:25.756 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.768 7f4f5f739700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f4f5810e460 0x7f4f580f0c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:25.756 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.768 7f4f5f739700 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f580fb1a0 con 0x7f4f5810e8b0 2024-04-17T22:31:25.756 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.769 7f4f5f739700 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f580f1a50 con 0x7f4f580fab20 2024-04-17T22:31:25.756 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.769 7f4f5f739700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4f581ac240 con 0x7f4f5810e460 2024-04-17T22:31:25.756 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.769 7f4f577fe700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f4f5810e460 0x7f4f580f0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:25.756 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.769 7f4f57fff700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f4f580fab20 0x7f4f5810de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:25.756 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.769 7f4f577fe700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f4f5810e460 0x7f4f580f0c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:33626/0 (socket says 172.21.15.102:33626) 2024-04-17T22:31:25.757 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.769 7f4f57fff700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f4f580fab20 0x7f4f5810de30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:51952/0 (socket says 172.21.15.102:51952) 2024-04-17T22:31:25.757 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.769 7f4f577fe700 1 -- 172.21.15.102:0/2826099210 learned_addr learned my addr 172.21.15.102:0/2826099210 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:31:25.757 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.769 7f4f57fff700 1 -- 172.21.15.102:0/2826099210 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f4f5810e460 msgr2=0x7f4f580f0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:31:25.757 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.769 7f4f57fff700 1 --2- 172.21.15.102:0/2826099210 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f4f5810e460 0x7f4f580f0c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:25.757 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.769 7f4f57fff700 1 -- 172.21.15.102:0/2826099210 >> v1:172.21.15.102:6789/0 conn(0x7f4f5810e8b0 legacy=0x7f4f580f1250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:25.757 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.769 7f4f57fff700 1 -- 172.21.15.102:0/2826099210 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f581ac3f0 con 0x7f4f580fab20 2024-04-17T22:31:25.757 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.769 7f4f56ffd700 1 -- 172.21.15.102:0/2826099210 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2772126961 0 0) 0x7f4f50001490 con 0x7f4f5810e8b0 2024-04-17T22:31:25.757 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.769 7f4f577fe700 1 --2- 172.21.15.102:0/2826099210 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f4f5810e460 0x7f4f580f0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:25.757 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.769 7f4f57fff700 1 --2- 172.21.15.102:0/2826099210 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4f580fab20 0x7f4f5810de30 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 rx=0x7f4f40002f70 tx=0x7f4f4000b9f0).ready entity=mon.0 client_cookie=728664e2e07b1056 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:25.757 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.769 7f4f56ffd700 1 -- 172.21.15.102:0/2826099210 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f4f581ac3f0 con 0x7f4f580fab20 2024-04-17T22:31:25.757 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.769 7f4f56ffd700 1 -- 172.21.15.102:0/2826099210 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f4000d610 con 0x7f4f580fab20 2024-04-17T22:31:25.757 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.769 7f4f56ffd700 1 -- 172.21.15.102:0/2826099210 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f4f4000d9e0 con 0x7f4f580fab20 2024-04-17T22:31:25.757 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.769 7f4f5f739700 1 -- 172.21.15.102:0/2826099210 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4f580fab20 msgr2=0x7f4f5810de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:25.758 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.769 7f4f5f739700 1 --2- 172.21.15.102:0/2826099210 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4f580fab20 0x7f4f5810de30 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 rx=0x7f4f40002f70 tx=0x7f4f4000b9f0).stop 2024-04-17T22:31:25.758 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.770 7f4f5f739700 1 -- 172.21.15.102:0/2826099210 shutdown_connections 2024-04-17T22:31:25.758 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.770 7f4f5f739700 1 --2- 172.21.15.102:0/2826099210 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4f580fab20 0x7f4f5810de30 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:25.758 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.770 7f4f5f739700 1 --2- 172.21.15.102:0/2826099210 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f4f5810e460 0x7f4f580f0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:25.758 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.770 7f4f5f739700 1 -- 172.21.15.102:0/2826099210 >> 172.21.15.102:0/2826099210 conn(0x7f4f58114460 msgr2=0x7f4f581148b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:25.758 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.770 7f4f5f739700 1 -- 172.21.15.102:0/2826099210 shutdown_connections 2024-04-17T22:31:25.758 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.770 7f4f5f739700 1 -- 172.21.15.102:0/2826099210 wait complete. 2024-04-17T22:31:25.780 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.793 7f4f5f739700 1 Processor -- start 2024-04-17T22:31:25.780 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.793 7f4f5f739700 1 -- start start 2024-04-17T22:31:25.780 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.793 7f4f5f739700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f4f580e1480 0x7f4f580c2b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:25.780 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.793 7f4f5f739700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4f580c31a0 0x7f4f581b3af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:25.780 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.793 7f4f5f739700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4f581b4120 0x7f4f581b4570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:25.780 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.793 7f4f5f739700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f581b4ba0 con 0x7f4f580c31a0 2024-04-17T22:31:25.780 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.793 7f4f5f739700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f580cfac0 con 0x7f4f581b4120 2024-04-17T22:31:25.781 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.793 7f4f5f739700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4f581bafb0 con 0x7f4f580e1480 2024-04-17T22:31:25.781 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.793 7f4f57fff700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f4f580e1480 0x7f4f580c2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:25.781 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.793 7f4f577fe700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4f580c31a0 0x7f4f581b3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:25.781 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.793 7f4f57fff700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f4f580e1480 0x7f4f580c2b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:33632/0 (socket says 172.21.15.102:33632) 2024-04-17T22:31:25.781 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.793 7f4f57fff700 1 -- 172.21.15.102:0/656426824 learned_addr learned my addr 172.21.15.102:0/656426824 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:31:25.781 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.793 7f4f5cd35700 1 --2- 172.21.15.102:0/656426824 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4f581b4120 0x7f4f581b4570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:25.781 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.794 7f4f577fe700 1 -- 172.21.15.102:0/656426824 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f4f580e1480 msgr2=0x7f4f580c2b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:25.781 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.794 7f4f577fe700 1 --2- 172.21.15.102:0/656426824 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f4f580e1480 0x7f4f580c2b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:25.781 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.794 7f4f577fe700 1 -- 172.21.15.102:0/656426824 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4f581b4120 msgr2=0x7f4f581b4570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:25.781 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.794 7f4f577fe700 1 --2- 172.21.15.102:0/656426824 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4f581b4120 0x7f4f581b4570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:25.781 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.794 7f4f577fe700 1 -- 172.21.15.102:0/656426824 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f581bb160 con 0x7f4f580c31a0 2024-04-17T22:31:25.781 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.794 7f4f57fff700 1 --2- 172.21.15.102:0/656426824 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f4f580e1480 0x7f4f580c2b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:25.781 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.794 7f4f5cd35700 1 --2- 172.21.15.102:0/656426824 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4f581b4120 0x7f4f581b4570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:25.781 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.794 7f4f577fe700 1 --2- 172.21.15.102:0/656426824 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4f580c31a0 0x7f4f581b3af0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 rx=0x7f4f4800c000 tx=0x7f4f4800ea70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:25.781 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.794 7f4f557fa700 1 -- 172.21.15.102:0/656426824 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f4f581bb160 con 0x7f4f580c31a0 2024-04-17T22:31:25.782 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.794 7f4f557fa700 1 -- 172.21.15.102:0/656426824 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f48012600 con 0x7f4f580c31a0 2024-04-17T22:31:25.782 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.794 7f4f557fa700 1 -- 172.21.15.102:0/656426824 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f4f4800f040 con 0x7f4f580c31a0 2024-04-17T22:31:25.782 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.794 7f4f5f739700 1 -- 172.21.15.102:0/656426824 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f581bb360 con 0x7f4f580c31a0 2024-04-17T22:31:25.782 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.794 7f4f5f739700 1 -- 172.21.15.102:0/656426824 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f581bb740 con 0x7f4f580c31a0 2024-04-17T22:31:25.782 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.794 7f4f557fa700 1 -- 172.21.15.102:0/656426824 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7f4f48011070 con 0x7f4f580c31a0 2024-04-17T22:31:25.782 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.794 7f4f557fa700 1 -- 172.21.15.102:0/656426824 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 2374+0+0 (secure 0 0 0) 0x7f4f48010040 con 0x7f4f580c31a0 2024-04-17T22:31:25.782 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.795 7f4f5f739700 1 -- 172.21.15.102:0/656426824 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f580f0500 con 0x7f4f580c31a0 2024-04-17T22:31:25.783 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.798 7f4f557fa700 1 -- 172.21.15.102:0/656426824 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+110455 (secure 0 0 0) 0x7f4f480123f0 con 0x7f4f580c31a0 2024-04-17T22:31:25.892 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.907 7f4f5f739700 1 -- 172.21.15.102:0/656426824 >> v1:172.21.15.102:6816/42360 conn(0x7f4f3c0145a0 legacy=0x7f4f3c016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:25.892 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.907 7f4f5f739700 1 -- 172.21.15.102:0/656426824 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4f580c31a0 msgr2=0x7f4f581b3af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:25.892 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.907 7f4f5f739700 1 --2- 172.21.15.102:0/656426824 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4f580c31a0 0x7f4f581b3af0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 rx=0x7f4f4800c000 tx=0x7f4f4800ea70).stop 2024-04-17T22:31:25.892 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.907 7f4f5f739700 1 -- 172.21.15.102:0/656426824 shutdown_connections 2024-04-17T22:31:25.892 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.907 7f4f5f739700 1 --2- 172.21.15.102:0/656426824 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4f581b4120 0x7f4f581b4570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:25.893 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.907 7f4f5f739700 1 --2- 172.21.15.102:0/656426824 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f4f580e1480 0x7f4f580c2b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:25.893 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.907 7f4f5f739700 1 --2- 172.21.15.102:0/656426824 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4f580c31a0 0x7f4f581b3af0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:25.893 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.907 7f4f5f739700 1 -- 172.21.15.102:0/656426824 >> 172.21.15.102:0/656426824 conn(0x7f4f580dcc50 msgr2=0x7f4f580d8d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:25.893 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.907 7f4f5f739700 1 -- 172.21.15.102:0/656426824 shutdown_connections 2024-04-17T22:31:25.893 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:25.907 7f4f5f739700 1 -- 172.21.15.102:0/656426824 wait complete. 2024-04-17T22:31:25.894 INFO:teuthology.orchestra.run.smithi102.stderr:Invalid command: missing required parameter pg_num() 2024-04-17T22:31:25.894 INFO:teuthology.orchestra.run.smithi102.stderr:osd pool create {} {replicated|erasure} {} {} {} {} {} {} {} : create pool 2024-04-17T22:31:25.894 INFO:teuthology.orchestra.run.smithi102.stderr:Error EINVAL: invalid command 2024-04-17T22:31:25.905 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-17T22:31:25.905 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata 64 2024-04-17T22:31:26.035 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:26.048 7f11091a5700 1 Processor -- start 2024-04-17T22:31:26.035 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:26.048 7f11091a5700 1 -- start start 2024-04-17T22:31:26.035 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:26.048 7f11091a5700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f110410f060 0x7f11041a6470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:26.035 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:26.048 7f11091a5700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f11041a6aa0 0x7f11041a8f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:26.035 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:26.048 7f11091a5700 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11040ad030 con 0x7f11040f95b0 2024-04-17T22:31:26.035 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:26.048 7f11091a5700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11041a1e00 con 0x7f11041a6aa0 2024-04-17T22:31:26.036 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:26.048 7f11091a5700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f11040f9d30 con 0x7f110410f060 2024-04-17T22:31:26.036 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:26.048 7f10fbfff700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f110410f060 0x7f11041a6470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:26.036 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:26.048 7f11025fd700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f11041a6aa0 0x7f11041a8f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:26.036 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:26.048 7f1101dfc700 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f11040f95b0 0x7f110410e860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:33704/0 (socket says 172.21.15.102:33704) 2024-04-17T22:31:26.036 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:26.048 7f1101dfc700 1 -- 172.21.15.102:0/90457666 learned_addr learned my addr 172.21.15.102:0/90457666 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-17T22:31:26.036 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:26.048 7f11015fb700 1 -- 172.21.15.102:0/90457666 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 281102040 0 0) 0x7f10f4001110 con 0x7f11040f95b0 2024-04-17T22:31:26.036 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:26.048 7f10fbfff700 1 -- 172.21.15.102:0/90457666 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f11041a6aa0 msgr2=0x7f11041a8f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:31:26.036 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:26.049 7f10fbfff700 1 --2- 172.21.15.102:0/90457666 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f11041a6aa0 0x7f11041a8f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:26.036 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:26.049 7f10fbfff700 1 -- 172.21.15.102:0/90457666 >> v1:172.21.15.102:6789/0 conn(0x7f11040f95b0 legacy=0x7f110410e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:26.036 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:26.049 7f10fbfff700 1 -- 172.21.15.102:0/90457666 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f110410f4b0 con 0x7f110410f060 2024-04-17T22:31:26.037 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:26.049 7f11025fd700 1 --2- 172.21.15.102:0/90457666 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f11041a6aa0 0x7f11041a8f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:26.037 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:26.049 7f10fbfff700 1 --2- 172.21.15.102:0/90457666 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f110410f060 0x7f11041a6470 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 rx=0x7f10ec002f70 tx=0x7f10ec00b9a0).ready entity=mon.2 client_cookie=b10e329bf20337e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:26.037 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:26.049 7f11015fb700 1 -- 172.21.15.102:0/90457666 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f110410f4b0 con 0x7f110410f060 2024-04-17T22:31:26.037 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:26.049 7f11015fb700 1 -- 172.21.15.102:0/90457666 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10ec00d570 con 0x7f110410f060 2024-04-17T22:31:26.037 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:26.049 7f11015fb700 1 -- 172.21.15.102:0/90457666 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f10ec00d940 con 0x7f110410f060 2024-04-17T22:31:26.037 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:26.049 7f11091a5700 1 -- 172.21.15.102:0/90457666 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f110410f060 msgr2=0x7f11041a6470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:26.037 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:26.049 7f11091a5700 1 --2- 172.21.15.102:0/90457666 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f110410f060 0x7f11041a6470 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 rx=0x7f10ec002f70 tx=0x7f10ec00b9a0).stop 2024-04-17T22:31:26.037 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:26.049 7f11091a5700 1 -- 172.21.15.102:0/90457666 shutdown_connections 2024-04-17T22:31:26.037 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:26.049 7f11091a5700 1 --2- 172.21.15.102:0/90457666 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f11041a6aa0 0x7f11041a8f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:26.037 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:26.049 7f11091a5700 1 --2- 172.21.15.102:0/90457666 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f110410f060 0x7f11041a6470 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:26.037 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:26.049 7f11091a5700 1 -- 172.21.15.102:0/90457666 >> 172.21.15.102:0/90457666 conn(0x7f1104113000 msgr2=0x7f1104113450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:26.038 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:26.049 7f11091a5700 1 -- 172.21.15.102:0/90457666 shutdown_connections 2024-04-17T22:31:26.038 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:26.049 7f11091a5700 1 -- 172.21.15.102:0/90457666 wait complete. 2024-04-17T22:31:26.060 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:26.072 7f11091a5700 1 Processor -- start 2024-04-17T22:31:26.060 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:26.072 7f11091a5700 1 -- start start 2024-04-17T22:31:26.060 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:26.073 7f11091a5700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f11040d4260 0x7f11040d46b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:26.060 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:26.073 7f11091a5700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f11040d4ce0 0x7f11041b19d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:26.060 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:26.073 7f11091a5700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f11041b2000 0x7f11041b2450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:26.060 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:26.073 7f11091a5700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11040e4310 con 0x7f11041b2000 2024-04-17T22:31:26.060 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:26.073 7f11091a5700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11040cd450 con 0x7f11040d4260 2024-04-17T22:31:26.061 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:26.073 7f11091a5700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f11040c44b0 con 0x7f11040d4ce0 2024-04-17T22:31:26.061 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:26.073 7f1101dfc700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f11040d4260 0x7f11040d46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:26.061 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:26.073 7f10fbfff700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f11040d4ce0 0x7f11041b19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:26.061 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:26.073 7f1101dfc700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f11040d4260 0x7f11040d46b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:60940/0 (socket says 172.21.15.102:60940) 2024-04-17T22:31:26.061 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:26.073 7f1101dfc700 1 -- 172.21.15.102:0/330146670 learned_addr learned my addr 172.21.15.102:0/330146670 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:31:26.061 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:26.073 7f11025fd700 1 --2- 172.21.15.102:0/330146670 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f11041b2000 0x7f11041b2450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:26.061 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:26.073 7f10fbfff700 1 -- 172.21.15.102:0/330146670 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f11040d4260 msgr2=0x7f11040d46b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:26.061 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:26.073 7f10fbfff700 1 --2- 172.21.15.102:0/330146670 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f11040d4260 0x7f11040d46b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:26.061 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:26.073 7f10fbfff700 1 -- 172.21.15.102:0/330146670 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f11041b2000 msgr2=0x7f11041b2450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:26.061 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:26.073 7f10fbfff700 1 --2- 172.21.15.102:0/330146670 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f11041b2000 0x7f11041b2450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:26.061 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:26.073 7f10fbfff700 1 -- 172.21.15.102:0/330146670 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11040cd690 con 0x7f11040d4ce0 2024-04-17T22:31:26.061 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:26.073 7f1101dfc700 1 --2- 172.21.15.102:0/330146670 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f11040d4260 0x7f11040d46b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:26.061 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:26.073 7f11025fd700 1 --2- 172.21.15.102:0/330146670 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f11041b2000 0x7f11041b2450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:26.061 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:26.073 7f10fbfff700 1 --2- 172.21.15.102:0/330146670 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f11040d4ce0 0x7f11041b19d0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 rx=0x7f10ec002f70 tx=0x7f10ec0068c0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:26.062 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:26.073 7f10faffd700 1 -- 172.21.15.102:0/330146670 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f11040cd690 con 0x7f11040d4ce0 2024-04-17T22:31:26.062 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:26.074 7f10faffd700 1 -- 172.21.15.102:0/330146670 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10ec00d9a0 con 0x7f11040d4ce0 2024-04-17T22:31:26.062 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:26.074 7f11091a5700 1 -- 172.21.15.102:0/330146670 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11040cf7d0 con 0x7f11040d4ce0 2024-04-17T22:31:26.062 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:26.074 7f10faffd700 1 -- 172.21.15.102:0/330146670 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f10ec017040 con 0x7f11040d4ce0 2024-04-17T22:31:26.062 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:26.074 7f11091a5700 1 -- 172.21.15.102:0/330146670 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11041b92a0 con 0x7f11040d4ce0 2024-04-17T22:31:26.062 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:26.074 7f10faffd700 1 -- 172.21.15.102:0/330146670 <== mon.2 v2:172.21.15.102:3302/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7f10ec011bb0 con 0x7f11040d4ce0 2024-04-17T22:31:26.062 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:26.074 7f10faffd700 1 -- 172.21.15.102:0/330146670 <== mon.2 v2:172.21.15.102:3302/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 2374+0+0 (secure 0 0 0) 0x7f10ec018070 con 0x7f11040d4ce0 2024-04-17T22:31:26.062 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:26.075 7f11091a5700 1 -- 172.21.15.102:0/330146670 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11040f09f0 con 0x7f11040d4ce0 2024-04-17T22:31:26.063 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:26.078 7f10faffd700 1 -- 172.21.15.102:0/330146670 <== mon.2 v2:172.21.15.102:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+110455 (secure 0 0 0) 0x7f10ec0117b0 con 0x7f11040d4ce0 2024-04-17T22:31:26.167 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:26.182 7f11091a5700 1 -- 172.21.15.102:0/330146670 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 64} v 0) v1 -- 0x7f11041ac2b0 con 0x7f11040d4ce0 2024-04-17T22:31:26.851 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:26.866 7f10faffd700 1 -- 172.21.15.102:0/330146670 <== mon.2 v2:172.21.15.102:3302/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 64}]=0 pool 'cephfs_metadata' created v22) v1 ==== 134+0+0 (secure 0 0 0) 0x7f10ec00d360 con 0x7f11040d4ce0 2024-04-17T22:31:26.857 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:26.872 7f11091a5700 1 -- 172.21.15.102:0/330146670 >> v1:172.21.15.102:6816/42360 conn(0x7f10e80145a0 legacy=0x7f10e8016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:26.857 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:26.872 7f11091a5700 1 -- 172.21.15.102:0/330146670 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f11040d4ce0 msgr2=0x7f11041b19d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:26.857 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:26.872 7f11091a5700 1 --2- 172.21.15.102:0/330146670 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f11040d4ce0 0x7f11041b19d0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 rx=0x7f10ec002f70 tx=0x7f10ec0068c0).stop 2024-04-17T22:31:26.860 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:26.875 7f11091a5700 1 -- 172.21.15.102:0/330146670 shutdown_connections 2024-04-17T22:31:26.860 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:26.875 7f11091a5700 1 --2- 172.21.15.102:0/330146670 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f11041b2000 0x7f11041b2450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:26.860 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:26.875 7f11091a5700 1 --2- 172.21.15.102:0/330146670 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f11040d4260 0x7f11040d46b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:26.860 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:26.875 7f11091a5700 1 --2- 172.21.15.102:0/330146670 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f11040d4ce0 0x7f11041b19d0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:26.860 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:26.875 7f11091a5700 1 -- 172.21.15.102:0/330146670 >> 172.21.15.102:0/330146670 conn(0x7f11040e1100 msgr2=0x7f11040dae20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:26.860 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:26.875 7f11091a5700 1 -- 172.21.15.102:0/330146670 shutdown_connections 2024-04-17T22:31:26.860 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:26.875 7f11091a5700 1 -- 172.21.15.102:0/330146670 wait complete. 2024-04-17T22:31:26.861 INFO:teuthology.orchestra.run.smithi102.stderr:pool 'cephfs_metadata' created 2024-04-17T22:31:26.872 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 64 2024-04-17T22:31:27.006 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:27.018 7f4088cc1700 1 Processor -- start 2024-04-17T22:31:27.006 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:27.018 7f4088cc1700 1 -- start start 2024-04-17T22:31:27.006 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:27.018 7f4088cc1700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f40840f54c0 0x7f408410b440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:27.006 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:27.018 7f4088cc1700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f40841a4ef0 0x7f40841a7350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:27.006 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:27.018 7f4088cc1700 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40841a07a0 con 0x7f40841050d0 2024-04-17T22:31:27.006 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:27.018 7f4088cc1700 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f408410ba70 con 0x7f40841a4ef0 2024-04-17T22:31:27.006 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:27.018 7f4088cc1700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40841a8990 con 0x7f40840f54c0 2024-04-17T22:31:27.007 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:27.019 7f408128a700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f40840f54c0 0x7f408410b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:27.007 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:27.019 7f4081a8b700 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f40841050d0 0x7f408410ac40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:33720/0 (socket says 172.21.15.102:33720) 2024-04-17T22:31:27.007 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:27.019 7f4081a8b700 1 -- 172.21.15.102:0/1733334728 learned_addr learned my addr 172.21.15.102:0/1733334728 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-17T22:31:27.007 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:27.019 7f408228c700 1 --2- 172.21.15.102:0/1733334728 >> v2:172.21.15.102:3300/0 conn(0x7f40841a4ef0 0x7f40841a7350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:27.007 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:27.019 7f4080a89700 1 -- 172.21.15.102:0/1733334728 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4124717279 0 0) 0x7f4074001110 con 0x7f40841050d0 2024-04-17T22:31:27.007 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:27.019 7f408128a700 1 -- 172.21.15.102:0/1733334728 >> v2:172.21.15.102:3300/0 conn(0x7f40841a4ef0 msgr2=0x7f40841a7350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:31:27.007 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:27.019 7f408128a700 1 --2- 172.21.15.102:0/1733334728 >> v2:172.21.15.102:3300/0 conn(0x7f40841a4ef0 0x7f40841a7350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:27.007 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:27.019 7f408128a700 1 -- 172.21.15.102:0/1733334728 >> v1:172.21.15.102:6789/0 conn(0x7f40841050d0 legacy=0x7f408410ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:27.007 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:27.019 7f408128a700 1 -- 172.21.15.102:0/1733334728 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40841a8b40 con 0x7f40840f54c0 2024-04-17T22:31:27.007 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:27.019 7f408228c700 1 --2- 172.21.15.102:0/1733334728 >> v2:172.21.15.102:3300/0 conn(0x7f40841a4ef0 0x7f40841a7350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:27.007 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:27.019 7f408128a700 1 --2- 172.21.15.102:0/1733334728 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f40840f54c0 0x7f408410b440 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 rx=0x7f406c002f70 tx=0x7f406c0091c0).ready entity=mon.1 client_cookie=51f260c937982fc1 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:27.008 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:27.019 7f4080a89700 1 -- 172.21.15.102:0/1733334728 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f40841a8b40 con 0x7f40840f54c0 2024-04-17T22:31:27.008 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:27.019 7f4080a89700 1 -- 172.21.15.102:0/1733334728 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f406c00d820 con 0x7f40840f54c0 2024-04-17T22:31:27.008 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:27.019 7f4080a89700 1 -- 172.21.15.102:0/1733334728 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f406c00dbf0 con 0x7f40840f54c0 2024-04-17T22:31:27.008 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:27.019 7f4088cc1700 1 -- 172.21.15.102:0/1733334728 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f40840f54c0 msgr2=0x7f408410b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:27.008 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:27.019 7f4088cc1700 1 --2- 172.21.15.102:0/1733334728 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f40840f54c0 0x7f408410b440 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 rx=0x7f406c002f70 tx=0x7f406c0091c0).stop 2024-04-17T22:31:27.008 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:27.020 7f4088cc1700 1 -- 172.21.15.102:0/1733334728 shutdown_connections 2024-04-17T22:31:27.009 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:27.020 7f4088cc1700 1 --2- 172.21.15.102:0/1733334728 >> v2:172.21.15.102:3300/0 conn(0x7f40841a4ef0 0x7f40841a7350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:27.009 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:27.020 7f4088cc1700 1 --2- 172.21.15.102:0/1733334728 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f40840f54c0 0x7f408410b440 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:27.009 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:27.020 7f4088cc1700 1 -- 172.21.15.102:0/1733334728 >> 172.21.15.102:0/1733334728 conn(0x7f40841135a0 msgr2=0x7f40840f4c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:27.009 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:27.020 7f4088cc1700 1 -- 172.21.15.102:0/1733334728 shutdown_connections 2024-04-17T22:31:27.009 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:27.020 7f4088cc1700 1 -- 172.21.15.102:0/1733334728 wait complete. 2024-04-17T22:31:27.030 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:27.043 7f4088cc1700 1 Processor -- start 2024-04-17T22:31:27.030 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:27.043 7f4088cc1700 1 -- start start 2024-04-17T22:31:27.030 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:27.043 7f4088cc1700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f40840d2870 0x7f40840d2cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:27.030 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:27.043 7f4088cc1700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f40840d32f0 0x7f40840cf2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:27.030 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:27.043 7f4088cc1700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f40840cf8d0 0x7f40841b6db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:27.030 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:27.043 7f4088cc1700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40840d3d60 con 0x7f40840d32f0 2024-04-17T22:31:27.030 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:27.043 7f4088cc1700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40840d4c70 con 0x7f40840d2870 2024-04-17T22:31:27.030 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:27.043 7f4088cc1700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f40840c51a0 con 0x7f40840cf8d0 2024-04-17T22:31:27.030 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:27.043 7f4081a8b700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f40840d2870 0x7f40840d2cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:27.030 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:27.043 7f408128a700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f40840d32f0 0x7f40840cf2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:27.031 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:27.043 7f4081a8b700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f40840d2870 0x7f40840d2cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:60960/0 (socket says 172.21.15.102:60960) 2024-04-17T22:31:27.031 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:27.043 7f408228c700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f40840cf8d0 0x7f40841b6db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:27.031 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:27.043 7f4081a8b700 1 -- 172.21.15.102:0/94072179 learned_addr learned my addr 172.21.15.102:0/94072179 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:31:27.031 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:27.043 7f408128a700 1 -- 172.21.15.102:0/94072179 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f40840cf8d0 msgr2=0x7f40841b6db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:27.031 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:27.043 7f408128a700 1 --2- 172.21.15.102:0/94072179 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f40840cf8d0 0x7f40841b6db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:27.031 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:27.043 7f408128a700 1 -- 172.21.15.102:0/94072179 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f40840d2870 msgr2=0x7f40840d2cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:27.031 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:27.043 7f408128a700 1 --2- 172.21.15.102:0/94072179 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f40840d2870 0x7f40840d2cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:27.031 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:27.043 7f408128a700 1 -- 172.21.15.102:0/94072179 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40840d3940 con 0x7f40840d32f0 2024-04-17T22:31:27.031 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:27.043 7f4081a8b700 1 --2- 172.21.15.102:0/94072179 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f40840d2870 0x7f40840d2cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:27.031 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:27.044 7f408128a700 1 --2- 172.21.15.102:0/94072179 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f40840d32f0 0x7f40840cf2a0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 rx=0x7f406c002f70 tx=0x7f406c006c10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:27.031 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:27.044 7f407affd700 1 -- 172.21.15.102:0/94072179 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f40840d3940 con 0x7f40840d32f0 2024-04-17T22:31:27.031 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:27.044 7f407affd700 1 -- 172.21.15.102:0/94072179 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f406c0169e0 con 0x7f40840d32f0 2024-04-17T22:31:27.032 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:27.044 7f4088cc1700 1 -- 172.21.15.102:0/94072179 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40840cfd20 con 0x7f40840d32f0 2024-04-17T22:31:27.032 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:27.044 7f407affd700 1 -- 172.21.15.102:0/94072179 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f406c016db0 con 0x7f40840d32f0 2024-04-17T22:31:27.032 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:27.044 7f4088cc1700 1 -- 172.21.15.102:0/94072179 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40841b73e0 con 0x7f40840d32f0 2024-04-17T22:31:27.032 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:27.044 7f407affd700 1 -- 172.21.15.102:0/94072179 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7f406c00e3f0 con 0x7f40840d32f0 2024-04-17T22:31:27.032 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:27.044 7f407affd700 1 -- 172.21.15.102:0/94072179 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 2728+0+0 (secure 0 0 0) 0x7f406c008c00 con 0x7f40840d32f0 2024-04-17T22:31:27.032 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:27.045 7f4088cc1700 1 -- 172.21.15.102:0/94072179 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40840f0500 con 0x7f40840d32f0 2024-04-17T22:31:27.033 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:27.048 7f407affd700 1 -- 172.21.15.102:0/94072179 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+110455 (secure 0 0 0) 0x7f406c0167d0 con 0x7f40840d32f0 2024-04-17T22:31:27.136 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:27.152 7f4088cc1700 1 -- 172.21.15.102:0/94072179 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pgp_num": 64} v 0) v1 -- 0x7f40841a9d80 con 0x7f40840d32f0 2024-04-17T22:31:27.854 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:27.868 7f407affd700 1 -- 172.21.15.102:0/94072179 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pgp_num": 64}]=0 pool 'cephfs_data' created v23) v1 ==== 141+0+0 (secure 0 0 0) 0x7f406c00d420 con 0x7f40840d32f0 2024-04-17T22:31:27.859 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:27.873 7f4088cc1700 1 -- 172.21.15.102:0/94072179 >> v1:172.21.15.102:6816/42360 conn(0x7f40680145a0 legacy=0x7f4068016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:27.859 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:27.873 7f4088cc1700 1 -- 172.21.15.102:0/94072179 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f40840d32f0 msgr2=0x7f40840cf2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:27.859 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:27.873 7f4088cc1700 1 --2- 172.21.15.102:0/94072179 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f40840d32f0 0x7f40840cf2a0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 rx=0x7f406c002f70 tx=0x7f406c006c10).stop 2024-04-17T22:31:27.861 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:27.875 7f4088cc1700 1 -- 172.21.15.102:0/94072179 shutdown_connections 2024-04-17T22:31:27.861 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:27.875 7f4088cc1700 1 --2- 172.21.15.102:0/94072179 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f40840cf8d0 0x7f40841b6db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:27.861 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:27.875 7f4088cc1700 1 --2- 172.21.15.102:0/94072179 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f40840d2870 0x7f40840d2cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:27.861 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:27.875 7f4088cc1700 1 --2- 172.21.15.102:0/94072179 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f40840d32f0 0x7f40840cf2a0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:27.861 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:27.875 7f4088cc1700 1 -- 172.21.15.102:0/94072179 >> 172.21.15.102:0/94072179 conn(0x7f40840c57a0 msgr2=0x7f40840bd6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:27.863 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:27.877 7f4088cc1700 1 -- 172.21.15.102:0/94072179 shutdown_connections 2024-04-17T22:31:27.864 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:27.878 7f4088cc1700 1 -- 172.21.15.102:0/94072179 wait complete. 2024-04-17T22:31:27.870 INFO:teuthology.orchestra.run.smithi102.stderr:pool 'cephfs_data' created 2024-04-17T22:31:27.892 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-17T22:31:28.026 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.037 7fabd0bbf700 1 Processor -- start 2024-04-17T22:31:28.026 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.038 7fabd0bbf700 1 -- start start 2024-04-17T22:31:28.026 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.038 7fabd0bbf700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fabcc0f5750 0x7fabcc10c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:28.026 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.038 7fabd0bbf700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fabcc10c720 0x7fabcc1a0be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:28.026 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.038 7fabd0bbf700 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabcc1a1a10 con 0x7fabcc10cb70 2024-04-17T22:31:28.026 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.038 7fabd0bbf700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabcc0f93c0 con 0x7fabcc0f5750 2024-04-17T22:31:28.026 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.038 7fabd0bbf700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fabcc1a1c30 con 0x7fabcc10c720 2024-04-17T22:31:28.026 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.038 7fabc904f700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fabcc10c720 0x7fabcc1a0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:28.026 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.038 7fabc9850700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fabcc0f5750 0x7fabcc10c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:28.027 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.038 7fabca051700 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7fabcc10cb70 0x7fabcc1a1210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:33736/0 (socket says 172.21.15.102:33736) 2024-04-17T22:31:28.027 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.038 7fabc904f700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fabcc10c720 0x7fabcc1a0be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:33670/0 (socket says 172.21.15.102:33670) 2024-04-17T22:31:28.027 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.038 7fabca051700 1 -- 172.21.15.102:0/3147325407 learned_addr learned my addr 172.21.15.102:0/3147325407 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-17T22:31:28.027 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.038 7fabc904f700 1 -- 172.21.15.102:0/3147325407 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fabcc0f5750 msgr2=0x7fabcc10c0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:31:28.027 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.038 7fabc904f700 1 --2- 172.21.15.102:0/3147325407 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fabcc0f5750 0x7fabcc10c0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:28.027 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.039 7fabc904f700 1 -- 172.21.15.102:0/3147325407 >> v1:172.21.15.102:6789/0 conn(0x7fabcc10cb70 legacy=0x7fabcc1a1210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:28.027 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.038 7fabc884e700 1 -- 172.21.15.102:0/3147325407 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1598625713 0 0) 0x7fabbc001110 con 0x7fabcc10cb70 2024-04-17T22:31:28.027 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.039 7fabc904f700 1 -- 172.21.15.102:0/3147325407 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabcc1aa310 con 0x7fabcc10c720 2024-04-17T22:31:28.027 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.039 7fabc9850700 1 --2- 172.21.15.102:0/3147325407 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fabcc0f5750 0x7fabcc10c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:28.027 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.039 7fabc904f700 1 --2- 172.21.15.102:0/3147325407 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fabcc10c720 0x7fabcc1a0be0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 rx=0x7fabc0002f70 tx=0x7fabc000b9a0).ready entity=mon.2 client_cookie=7764e4c6a43db93d server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:28.027 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.039 7fabc884e700 1 -- 172.21.15.102:0/3147325407 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fabcc1aa310 con 0x7fabcc10c720 2024-04-17T22:31:28.027 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.039 7fabc884e700 1 -- 172.21.15.102:0/3147325407 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabc000d570 con 0x7fabcc10c720 2024-04-17T22:31:28.027 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.039 7fabc884e700 1 -- 172.21.15.102:0/3147325407 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fabc000d940 con 0x7fabcc10c720 2024-04-17T22:31:28.027 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.039 7fabd0bbf700 1 -- 172.21.15.102:0/3147325407 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fabcc10c720 msgr2=0x7fabcc1a0be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:28.028 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.039 7fabd0bbf700 1 --2- 172.21.15.102:0/3147325407 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fabcc10c720 0x7fabcc1a0be0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 rx=0x7fabc0002f70 tx=0x7fabc000b9a0).stop 2024-04-17T22:31:28.028 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.039 7fabd0bbf700 1 -- 172.21.15.102:0/3147325407 shutdown_connections 2024-04-17T22:31:28.028 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.039 7fabd0bbf700 1 --2- 172.21.15.102:0/3147325407 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fabcc0f5750 0x7fabcc10c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:28.028 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.039 7fabd0bbf700 1 --2- 172.21.15.102:0/3147325407 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fabcc10c720 0x7fabcc1a0be0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:28.028 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.039 7fabd0bbf700 1 -- 172.21.15.102:0/3147325407 >> 172.21.15.102:0/3147325407 conn(0x7fabcc1138a0 msgr2=0x7fabcc0f4f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:28.028 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.039 7fabd0bbf700 1 -- 172.21.15.102:0/3147325407 shutdown_connections 2024-04-17T22:31:28.028 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.039 7fabd0bbf700 1 -- 172.21.15.102:0/3147325407 wait complete. 2024-04-17T22:31:28.050 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.062 7fabd0bbf700 1 Processor -- start 2024-04-17T22:31:28.050 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.062 7fabd0bbf700 1 -- start start 2024-04-17T22:31:28.050 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.062 7fabd0bbf700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fabcc0d5e00 0x7fabcc0d6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:28.050 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.062 7fabd0bbf700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fabcc0d6880 0x7fabcc1b1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:28.050 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.062 7fabd0bbf700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fabcc1b21f0 0x7fabcc1b2640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:28.050 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.062 7fabd0bbf700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabcc0e2bb0 con 0x7fabcc0d5e00 2024-04-17T22:31:28.050 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.062 7fabd0bbf700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabcc1b2c70 con 0x7fabcc0d6880 2024-04-17T22:31:28.050 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.062 7fabd0bbf700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fabcc0cd4b0 con 0x7fabcc1b21f0 2024-04-17T22:31:28.051 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.062 7fabc904f700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fabcc0d6880 0x7fabcc1b1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:28.051 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.062 7fabca051700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fabcc1b21f0 0x7fabcc1b2640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:28.051 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.062 7fabc9850700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fabcc0d5e00 0x7fabcc0d6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:28.051 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.062 7fabc904f700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fabcc0d6880 0x7fabcc1b1bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:60980/0 (socket says 172.21.15.102:60980) 2024-04-17T22:31:28.051 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.062 7fabc904f700 1 -- 172.21.15.102:0/2862622684 learned_addr learned my addr 172.21.15.102:0/2862622684 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:31:28.051 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.063 7fabca051700 1 -- 172.21.15.102:0/2862622684 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fabcc0d6880 msgr2=0x7fabcc1b1bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:28.052 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.063 7fabca051700 1 --2- 172.21.15.102:0/2862622684 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fabcc0d6880 0x7fabcc1b1bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:28.052 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.063 7fabca051700 1 -- 172.21.15.102:0/2862622684 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fabcc0d5e00 msgr2=0x7fabcc0d6250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:28.052 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.063 7fabca051700 1 --2- 172.21.15.102:0/2862622684 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fabcc0d5e00 0x7fabcc0d6250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:28.052 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.063 7fabca051700 1 -- 172.21.15.102:0/2862622684 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabcc0e2d60 con 0x7fabcc1b21f0 2024-04-17T22:31:28.052 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.063 7fabca051700 1 --2- 172.21.15.102:0/2862622684 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fabcc1b21f0 0x7fabcc1b2640 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 rx=0x7fabcc0c5da0 tx=0x7fabbc00c7b0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:28.052 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.063 7fabbaffd700 1 -- 172.21.15.102:0/2862622684 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fabcc0e2d60 con 0x7fabcc1b21f0 2024-04-17T22:31:28.052 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.063 7fabbaffd700 1 -- 172.21.15.102:0/2862622684 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabbc0070d0 con 0x7fabcc1b21f0 2024-04-17T22:31:28.052 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.063 7fabbaffd700 1 -- 172.21.15.102:0/2862622684 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fabbc005620 con 0x7fabcc1b21f0 2024-04-17T22:31:28.052 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.063 7fabd0bbf700 1 -- 172.21.15.102:0/2862622684 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabcc0cde30 con 0x7fabcc1b21f0 2024-04-17T22:31:28.053 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.063 7fabd0bbf700 1 -- 172.21.15.102:0/2862622684 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabcc1b9340 con 0x7fabcc1b21f0 2024-04-17T22:31:28.053 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.063 7fabbaffd700 1 -- 172.21.15.102:0/2862622684 <== mon.2 v2:172.21.15.102:3302/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7fabbc01b070 con 0x7fabcc1b21f0 2024-04-17T22:31:28.053 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.063 7fabbaffd700 1 -- 172.21.15.102:0/2862622684 <== mon.2 v2:172.21.15.102:3302/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 3078+0+0 (secure 0 0 0) 0x7fabbc004d40 con 0x7fabcc1b21f0 2024-04-17T22:31:28.053 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.064 7fabd0bbf700 1 -- 172.21.15.102:0/2862622684 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabcc0f0800 con 0x7fabcc1b21f0 2024-04-17T22:31:28.053 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.067 7fabbaffd700 1 -- 172.21.15.102:0/2862622684 <== mon.2 v2:172.21.15.102:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+110455 (secure 0 0 0) 0x7fabbc005820 con 0x7fabcc1b21f0 2024-04-17T22:31:28.188 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.202 7fabd0bbf700 1 -- 172.21.15.102:0/2862622684 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fabcc1ac360 con 0x7fabcc1b21f0 2024-04-17T22:31:28.194 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.207 7fabbaffd700 1 -- 172.21.15.102:0/2862622684 <== mon.2 v2:172.21.15.102:3302/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 4 and data pool 5 v9) v1 ==== 172+0+0 (secure 0 0 0) 0x7fabcc114420 con 0x7fabcc1b21f0 2024-04-17T22:31:28.199 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.213 7fabd0bbf700 1 -- 172.21.15.102:0/2862622684 >> v1:172.21.15.102:6816/42360 conn(0x7faba80145a0 legacy=0x7faba8016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:28.199 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.213 7fabd0bbf700 1 -- 172.21.15.102:0/2862622684 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fabcc1b21f0 msgr2=0x7fabcc1b2640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:28.199 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.213 7fabd0bbf700 1 --2- 172.21.15.102:0/2862622684 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fabcc1b21f0 0x7fabcc1b2640 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 rx=0x7fabcc0c5da0 tx=0x7fabbc00c7b0).stop 2024-04-17T22:31:28.199 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.213 7fabd0bbf700 1 -- 172.21.15.102:0/2862622684 shutdown_connections 2024-04-17T22:31:28.200 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.213 7fabd0bbf700 1 --2- 172.21.15.102:0/2862622684 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fabcc1b21f0 0x7fabcc1b2640 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:28.200 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.213 7fabd0bbf700 1 --2- 172.21.15.102:0/2862622684 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fabcc0d5e00 0x7fabcc0d6250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:28.200 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.213 7fabd0bbf700 1 --2- 172.21.15.102:0/2862622684 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fabcc0d6880 0x7fabcc1b1bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:28.200 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.213 7fabd0bbf700 1 -- 172.21.15.102:0/2862622684 >> 172.21.15.102:0/2862622684 conn(0x7fabcc0e0f60 msgr2=0x7fabcc0dac80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:28.200 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.214 7fabd0bbf700 1 -- 172.21.15.102:0/2862622684 shutdown_connections 2024-04-17T22:31:28.200 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.214 7fabd0bbf700 1 -- 172.21.15.102:0/2862622684 wait complete. 2024-04-17T22:31:28.216 INFO:teuthology.orchestra.run.smithi102.stderr:new fs with metadata pool 4 and data pool 5 2024-04-17T22:31:28.221 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T22:31:28.356 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.368 7fac0baa3700 1 Processor -- start 2024-04-17T22:31:28.356 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.368 7fac0baa3700 1 -- start start 2024-04-17T22:31:28.356 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.368 7fac0baa3700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fac0410c910 0x7fac041a6300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:28.356 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.368 7fac0baa3700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fac041a6930 0x7fac041a8d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:28.356 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.368 7fac0baa3700 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac041a1c10 con 0x7fac040f5770 2024-04-17T22:31:28.356 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.368 7fac0baa3700 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac040f93e0 con 0x7fac041a6930 2024-04-17T22:31:28.356 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.368 7fac0baa3700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac0410cd60 con 0x7fac0410c910 2024-04-17T22:31:28.357 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.368 7fac0909f700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fac041a6930 0x7fac041a8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:28.357 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.368 7fac03fff700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fac0410c910 0x7fac041a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:28.357 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.368 7fac0889e700 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7fac040f5770 0x7fac0410c110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:33752/0 (socket says 172.21.15.102:33752) 2024-04-17T22:31:28.357 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.368 7fac0909f700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fac041a6930 0x7fac041a8d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:52014/0 (socket says 172.21.15.102:52014) 2024-04-17T22:31:28.357 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.368 7fac03fff700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fac0410c910 0x7fac041a6300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:60992/0 (socket says 172.21.15.102:60992) 2024-04-17T22:31:28.357 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.368 7fac0889e700 1 -- 172.21.15.102:0/46861596 learned_addr learned my addr 172.21.15.102:0/46861596 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-17T22:31:28.357 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.369 7fac03fff700 1 -- 172.21.15.102:0/46861596 >> v2:172.21.15.102:3300/0 conn(0x7fac041a6930 msgr2=0x7fac041a8d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:31:28.357 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.369 7fac03fff700 1 --2- 172.21.15.102:0/46861596 >> v2:172.21.15.102:3300/0 conn(0x7fac041a6930 0x7fac041a8d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:28.357 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.369 7fac03fff700 1 -- 172.21.15.102:0/46861596 >> v1:172.21.15.102:6789/0 conn(0x7fac040f5770 legacy=0x7fac0410c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:28.357 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.369 7fac03fff700 1 -- 172.21.15.102:0/46861596 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac041aa3d0 con 0x7fac0410c910 2024-04-17T22:31:28.357 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.369 7fac037fe700 1 -- 172.21.15.102:0/46861596 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2746026226 0 0) 0x7fac041a1c10 con 0x7fac040f5770 2024-04-17T22:31:28.357 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.369 7fac0909f700 1 --2- 172.21.15.102:0/46861596 >> v2:172.21.15.102:3300/0 conn(0x7fac041a6930 0x7fac041a8d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:28.357 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.369 7fac03fff700 1 --2- 172.21.15.102:0/46861596 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fac0410c910 0x7fac041a6300 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 rx=0x7fabf0002f70 tx=0x7fabf000b970).ready entity=mon.1 client_cookie=600600cdd4270ea4 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:28.357 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.369 7fac037fe700 1 -- 172.21.15.102:0/46861596 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fac041aa3d0 con 0x7fac0410c910 2024-04-17T22:31:28.357 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.369 7fac037fe700 1 -- 172.21.15.102:0/46861596 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabf000d610 con 0x7fac0410c910 2024-04-17T22:31:28.358 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.369 7fac037fe700 1 -- 172.21.15.102:0/46861596 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fabf000d9e0 con 0x7fac0410c910 2024-04-17T22:31:28.358 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.369 7fac0baa3700 1 -- 172.21.15.102:0/46861596 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fac0410c910 msgr2=0x7fac041a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:28.358 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.369 7fac0baa3700 1 --2- 172.21.15.102:0/46861596 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fac0410c910 0x7fac041a6300 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 rx=0x7fabf0002f70 tx=0x7fabf000b970).stop 2024-04-17T22:31:28.358 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.369 7fac0baa3700 1 -- 172.21.15.102:0/46861596 shutdown_connections 2024-04-17T22:31:28.358 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.369 7fac0baa3700 1 --2- 172.21.15.102:0/46861596 >> v2:172.21.15.102:3300/0 conn(0x7fac041a6930 0x7fac041a8d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:28.358 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.369 7fac0baa3700 1 --2- 172.21.15.102:0/46861596 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fac0410c910 0x7fac041a6300 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:28.358 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.369 7fac0baa3700 1 -- 172.21.15.102:0/46861596 >> 172.21.15.102:0/46861596 conn(0x7fac041138c0 msgr2=0x7fac040f4f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:28.358 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.369 7fac0baa3700 1 -- 172.21.15.102:0/46861596 shutdown_connections 2024-04-17T22:31:28.358 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.369 7fac0baa3700 1 -- 172.21.15.102:0/46861596 wait complete. 2024-04-17T22:31:28.378 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.392 7fac0baa3700 1 Processor -- start 2024-04-17T22:31:28.378 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.392 7fac0baa3700 1 -- start start 2024-04-17T22:31:28.381 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.392 7fac0baa3700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fac040d5de0 0x7fac040d6230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:28.381 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.392 7fac0baa3700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fac040d6860 0x7fac041b1ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:28.381 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.392 7fac0baa3700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fac041b21d0 0x7fac041b2620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:28.381 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.392 7fac0baa3700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac040c4280 con 0x7fac040d6860 2024-04-17T22:31:28.381 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.392 7fac0baa3700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac041a6650 con 0x7fac040d5de0 2024-04-17T22:31:28.381 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.392 7fac0baa3700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fac040ccea0 con 0x7fac041b21d0 2024-04-17T22:31:28.381 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.393 7fac0889e700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fac040d5de0 0x7fac040d6230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:28.381 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.393 7fac03fff700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fac040d6860 0x7fac041b1ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:28.381 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.393 7fac0909f700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fac041b21d0 0x7fac041b2620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:28.381 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.393 7fac03fff700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fac040d6860 0x7fac041b1ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:52022/0 (socket says 172.21.15.102:52022) 2024-04-17T22:31:28.381 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.393 7fac0889e700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fac040d5de0 0x7fac040d6230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:32770/0 (socket says 172.21.15.102:32770) 2024-04-17T22:31:28.381 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.393 7fac0909f700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fac041b21d0 0x7fac041b2620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:33694/0 (socket says 172.21.15.102:33694) 2024-04-17T22:31:28.382 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.393 7fac03fff700 1 -- 172.21.15.102:0/1167598858 learned_addr learned my addr 172.21.15.102:0/1167598858 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:31:28.382 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.393 7fac03fff700 1 -- 172.21.15.102:0/1167598858 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fac041b21d0 msgr2=0x7fac041b2620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:28.382 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.393 7fac03fff700 1 --2- 172.21.15.102:0/1167598858 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fac041b21d0 0x7fac041b2620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:28.382 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.393 7fac03fff700 1 -- 172.21.15.102:0/1167598858 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fac040d5de0 msgr2=0x7fac040d6230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:28.382 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.393 7fac03fff700 1 --2- 172.21.15.102:0/1167598858 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fac040d5de0 0x7fac040d6230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:28.382 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.393 7fac03fff700 1 -- 172.21.15.102:0/1167598858 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac040cf6c0 con 0x7fac040d6860 2024-04-17T22:31:28.382 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.393 7fac0889e700 1 --2- 172.21.15.102:0/1167598858 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fac040d5de0 0x7fac040d6230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:28.382 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.393 7fac03fff700 1 --2- 172.21.15.102:0/1167598858 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fac040d6860 0x7fac041b1ba0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 rx=0x7fac041a6800 tx=0x7fabf000a040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:28.382 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.393 7fac01ffb700 1 -- 172.21.15.102:0/1167598858 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fac040cf6c0 con 0x7fac040d6860 2024-04-17T22:31:28.382 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.393 7fac01ffb700 1 -- 172.21.15.102:0/1167598858 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabf0013a50 con 0x7fac040d6860 2024-04-17T22:31:28.382 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.393 7fac0baa3700 1 -- 172.21.15.102:0/1167598858 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac041b9290 con 0x7fac040d6860 2024-04-17T22:31:28.382 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.393 7fac01ffb700 1 -- 172.21.15.102:0/1167598858 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fabf001f450 con 0x7fac040d6860 2024-04-17T22:31:28.382 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.393 7fac0baa3700 1 -- 172.21.15.102:0/1167598858 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac041b9670 con 0x7fac040d6860 2024-04-17T22:31:28.383 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.394 7fac01ffb700 1 -- 172.21.15.102:0/1167598858 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7fabf0027070 con 0x7fac040d6860 2024-04-17T22:31:28.383 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.394 7fac01ffb700 1 -- 172.21.15.102:0/1167598858 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 3194+0+0 (secure 0 0 0) 0x7fabf00087e0 con 0x7fac040d6860 2024-04-17T22:31:28.383 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.394 7fac0baa3700 1 -- 172.21.15.102:0/1167598858 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac040f0820 con 0x7fac040d6860 2024-04-17T22:31:28.383 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.397 7fac01ffb700 1 -- 172.21.15.102:0/1167598858 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+110455 (secure 0 0 0) 0x7fabf0013840 con 0x7fac040d6860 2024-04-17T22:31:28.488 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.502 7fac0baa3700 1 -- 172.21.15.102:0/1167598858 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fac041ac300 con 0x7fac040d6860 2024-04-17T22:31:28.489 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.503 7fac01ffb700 1 -- 172.21.15.102:0/1167598858 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+8777 (secure 0 0 0) 0x7fabf000d400 con 0x7fac040d6860 2024-04-17T22:31:28.489 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-17T22:31:28.489 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":24,"fsid":"cb63e789-091d-49b1-b120-55f9db526871","created":"2024-04-17 22:28:48.209635","modified":"2024-04-17 22:31:28.204618","last_up_change":"2024-04-17 22:28:54.236215","last_in_change":"2024-04-17 22:28:54.236215","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":4,"require_min_compat_client":"nautilus","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-04-17 22:28:57.478545","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-17 22:31:26.214102","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-17 22:31:27.183113","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3351ef85-ade4-4b90-a8f4-213f4e00ba01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6812","nonce":42955}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6813","nonce":42955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6815","nonce":42955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6814","nonce":42955}]},"public_addr":"172.21.15.102:6812/42955","cluster_addr":"172.21.15.102:6813/42955","heartbeat_back_addr":"172.21.15.102:6815/42955","heartbeat_front_addr":"172.21.15.102:6814/42955","state":["exists","up"]},{"osd":1,"uuid":"04f67eac-f36b-48f5-982d-4cfc776596d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6808","nonce":42958}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6809","nonce":42958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6811","nonce":42958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6810","nonce":42958}]},"public_addr":"172.21.15.102:6808/42958","cluster_addr":"172.21.15.102:6809/42958","heartbeat_back_addr":"172.21.15.102:6811/42958","heartbeat_front_addr":"172.21.15.102:6810/42958","state":["exists","up"]},{"osd":2,"uuid":"d524204b-70b2-40a8-8f73-f9e7d44318af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6804","nonce":42954}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6805","nonce":42954}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6807","nonce":42954}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6806","nonce":42954}]},"public_addr":"172.21.15.102:6804/42954","cluster_addr":"172.21.15.102:6805/42954","heartbeat_back_addr":"172.21.15.102:6807/42954","heartbeat_front_addr":"172.21.15.102:6806/42954","state":["exists","up"]},{"osd":3,"uuid":"4fc9d20c-1ad0-484f-aafa-2c9a7574245b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6800","nonce":42953}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6801","nonce":42953}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6803","nonce":42953}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6802","nonce":42953}]},"public_addr":"172.21.15.102:6800/42953","cluster_addr":"172.21.15.102:6801/42953","heartbeat_back_addr":"172.21.15.102:6803/42953","heartbeat_front_addr":"172.21.15.102:6802/42953","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2024-04-17T22:31:28.490 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.504 7fac0baa3700 1 -- 172.21.15.102:0/1167598858 >> v1:172.21.15.102:6816/42360 conn(0x7fabec0145a0 legacy=0x7fabec016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:28.490 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.504 7fac0baa3700 1 -- 172.21.15.102:0/1167598858 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fac040d6860 msgr2=0x7fac041b1ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:28.490 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.504 7fac0baa3700 1 --2- 172.21.15.102:0/1167598858 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fac040d6860 0x7fac041b1ba0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 rx=0x7fac041a6800 tx=0x7fabf000a040).stop 2024-04-17T22:31:28.491 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.504 7fac0baa3700 1 -- 172.21.15.102:0/1167598858 shutdown_connections 2024-04-17T22:31:28.491 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.504 7fac0baa3700 1 --2- 172.21.15.102:0/1167598858 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fac041b21d0 0x7fac041b2620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:28.491 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.504 7fac0baa3700 1 --2- 172.21.15.102:0/1167598858 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fac040d5de0 0x7fac040d6230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:28.491 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.504 7fac0baa3700 1 --2- 172.21.15.102:0/1167598858 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fac040d6860 0x7fac041b1ba0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:28.491 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.504 7fac0baa3700 1 -- 172.21.15.102:0/1167598858 >> 172.21.15.102:0/1167598858 conn(0x7fac040e0f20 msgr2=0x7fac040dac40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:28.491 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.504 7fac0baa3700 1 -- 172.21.15.102:0/1167598858 shutdown_connections 2024-04-17T22:31:28.491 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.504 7fac0baa3700 1 -- 172.21.15.102:0/1167598858 wait complete. 2024-04-17T22:31:28.501 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T22:31:28.627 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.639 7fd36ace7700 1 Processor -- start 2024-04-17T22:31:28.628 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.639 7fd36ace7700 1 -- start start 2024-04-17T22:31:28.628 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.640 7fd36ace7700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fd36410c880 0x7fd3641a6270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:28.628 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.640 7fd36ace7700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd3641a68a0 0x7fd3641a8d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:28.628 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.640 7fd36ace7700 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3641a1b80 con 0x7fd3640f56e0 2024-04-17T22:31:28.628 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.640 7fd36ace7700 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3640f9350 con 0x7fd36410c880 2024-04-17T22:31:28.628 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.640 7fd36ace7700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd36410ccd0 con 0x7fd3641a68a0 2024-04-17T22:31:28.628 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.640 7fd362ffd700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fd36410c880 0x7fd3641a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:28.628 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.640 7fd3637fe700 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7fd3640f56e0 0x7fd36410c080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:33758/0 (socket says 172.21.15.102:33758) 2024-04-17T22:31:28.628 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.640 7fd363fff700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd3641a68a0 0x7fd3641a8d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:28.628 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.640 7fd3637fe700 1 -- 172.21.15.102:0/2303480656 learned_addr learned my addr 172.21.15.102:0/2303480656 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-17T22:31:28.628 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.640 7fd363fff700 1 -- 172.21.15.102:0/2303480656 >> v2:172.21.15.102:3300/0 conn(0x7fd36410c880 msgr2=0x7fd3641a6270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:31:28.628 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.640 7fd363fff700 1 --2- 172.21.15.102:0/2303480656 >> v2:172.21.15.102:3300/0 conn(0x7fd36410c880 0x7fd3641a6270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:28.629 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.640 7fd363fff700 1 -- 172.21.15.102:0/2303480656 >> v1:172.21.15.102:6789/0 conn(0x7fd3640f56e0 legacy=0x7fd36410c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:28.629 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.640 7fd3627fc700 1 -- 172.21.15.102:0/2303480656 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1080306676 0 0) 0x7fd3641a1b80 con 0x7fd3640f56e0 2024-04-17T22:31:28.629 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.640 7fd363fff700 1 -- 172.21.15.102:0/2303480656 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3641aa340 con 0x7fd3641a68a0 2024-04-17T22:31:28.629 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.640 7fd363fff700 1 --2- 172.21.15.102:0/2303480656 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd3641a68a0 0x7fd3641a8d00 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 rx=0x7fd3640f4580 tx=0x7fd35c009cc0).ready entity=mon.1 client_cookie=9a3e59b5fdf07a5e server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:28.629 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.640 7fd3627fc700 1 -- 172.21.15.102:0/2303480656 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd3641aa340 con 0x7fd3641a68a0 2024-04-17T22:31:28.629 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.640 7fd3627fc700 1 -- 172.21.15.102:0/2303480656 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd35c00b840 con 0x7fd3641a68a0 2024-04-17T22:31:28.629 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.640 7fd3627fc700 1 -- 172.21.15.102:0/2303480656 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd35c00ba20 con 0x7fd3641a68a0 2024-04-17T22:31:28.629 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.640 7fd36ace7700 1 -- 172.21.15.102:0/2303480656 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd3641a68a0 msgr2=0x7fd3641a8d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:28.629 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.640 7fd36ace7700 1 --2- 172.21.15.102:0/2303480656 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd3641a68a0 0x7fd3641a8d00 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 rx=0x7fd3640f4580 tx=0x7fd35c009cc0).stop 2024-04-17T22:31:28.629 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.641 7fd36ace7700 1 -- 172.21.15.102:0/2303480656 shutdown_connections 2024-04-17T22:31:28.629 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.641 7fd36ace7700 1 --2- 172.21.15.102:0/2303480656 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd3641a68a0 0x7fd3641a8d00 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:28.629 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.641 7fd36ace7700 1 --2- 172.21.15.102:0/2303480656 >> v2:172.21.15.102:3300/0 conn(0x7fd36410c880 0x7fd3641a6270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:28.629 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.641 7fd36ace7700 1 -- 172.21.15.102:0/2303480656 >> 172.21.15.102:0/2303480656 conn(0x7fd364113830 msgr2=0x7fd3640f4ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:28.630 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.641 7fd36ace7700 1 -- 172.21.15.102:0/2303480656 shutdown_connections 2024-04-17T22:31:28.630 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.641 7fd36ace7700 1 -- 172.21.15.102:0/2303480656 wait complete. 2024-04-17T22:31:28.652 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.664 7fd36ace7700 1 Processor -- start 2024-04-17T22:31:28.652 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.664 7fd36ace7700 1 -- start start 2024-04-17T22:31:28.652 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.664 7fd36ace7700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd3641ab050 0x7fd3641ab4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:28.652 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.664 7fd36ace7700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd3641abad0 0x7fd3641b1ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:28.652 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.664 7fd36ace7700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd3641b2110 0x7fd3641b2560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:28.652 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.664 7fd36ace7700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3640cd0b0 con 0x7fd3641b2110 2024-04-17T22:31:28.652 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.664 7fd36ace7700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3640c4130 con 0x7fd3641ab050 2024-04-17T22:31:28.652 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.664 7fd36ace7700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd3641b2b90 con 0x7fd3641abad0 2024-04-17T22:31:28.652 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.664 7fd3637fe700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd3641ab050 0x7fd3641ab4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:28.652 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.664 7fd3637fe700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd3641ab050 0x7fd3641ab4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:32786/0 (socket says 172.21.15.102:32786) 2024-04-17T22:31:28.653 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.664 7fd363fff700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd3641b2110 0x7fd3641b2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:28.653 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.664 7fd3637fe700 1 -- 172.21.15.102:0/4155913342 learned_addr learned my addr 172.21.15.102:0/4155913342 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:31:28.653 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.664 7fd362ffd700 1 --2- 172.21.15.102:0/4155913342 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd3641abad0 0x7fd3641b1ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:28.653 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.664 7fd3637fe700 1 -- 172.21.15.102:0/4155913342 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd3641abad0 msgr2=0x7fd3641b1ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:28.653 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.664 7fd3637fe700 1 --2- 172.21.15.102:0/4155913342 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd3641abad0 0x7fd3641b1ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:28.653 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.664 7fd3637fe700 1 -- 172.21.15.102:0/4155913342 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd3641b2110 msgr2=0x7fd3641b2560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:28.653 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.665 7fd3637fe700 1 --2- 172.21.15.102:0/4155913342 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd3641b2110 0x7fd3641b2560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:28.653 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.665 7fd3637fe700 1 -- 172.21.15.102:0/4155913342 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3640cd2f0 con 0x7fd3641ab050 2024-04-17T22:31:28.653 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.665 7fd363fff700 1 --2- 172.21.15.102:0/4155913342 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd3641b2110 0x7fd3641b2560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:28.653 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.665 7fd362ffd700 1 --2- 172.21.15.102:0/4155913342 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd3641abad0 0x7fd3641b1ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:28.653 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.665 7fd3637fe700 1 --2- 172.21.15.102:0/4155913342 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd3641ab050 0x7fd3641ab4a0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 rx=0x7fd3640d6e00 tx=0x7fd34c007f70).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:28.653 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.665 7fd360ff9700 1 -- 172.21.15.102:0/4155913342 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd3640cd2f0 con 0x7fd3641ab050 2024-04-17T22:31:28.653 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.665 7fd360ff9700 1 -- 172.21.15.102:0/4155913342 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd34c0072e0 con 0x7fd3641ab050 2024-04-17T22:31:28.653 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.665 7fd36ace7700 1 -- 172.21.15.102:0/4155913342 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3640cf430 con 0x7fd3641ab050 2024-04-17T22:31:28.654 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.665 7fd360ff9700 1 -- 172.21.15.102:0/4155913342 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd34c005810 con 0x7fd3641ab050 2024-04-17T22:31:28.654 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.665 7fd36ace7700 1 -- 172.21.15.102:0/4155913342 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3641b93b0 con 0x7fd3641ab050 2024-04-17T22:31:28.654 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.665 7fd360ff9700 1 -- 172.21.15.102:0/4155913342 <== mon.1 v2:172.21.15.102:3301/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7fd34c005a10 con 0x7fd3641ab050 2024-04-17T22:31:28.654 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.665 7fd360ff9700 1 -- 172.21.15.102:0/4155913342 <== mon.1 v2:172.21.15.102:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 3194+0+0 (secure 0 0 0) 0x7fd34c014730 con 0x7fd3641ab050 2024-04-17T22:31:28.654 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.666 7fd36ace7700 1 -- 172.21.15.102:0/4155913342 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3640f0790 con 0x7fd3641ab050 2024-04-17T22:31:28.655 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.669 7fd360ff9700 1 -- 172.21.15.102:0/4155913342 <== mon.1 v2:172.21.15.102:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+110455 (secure 0 0 0) 0x7fd34c0070d0 con 0x7fd3641ab050 2024-04-17T22:31:28.763 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.776 7fd36ace7700 1 -- 172.21.15.102:0/4155913342 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd3640a6540 con 0x7fd3641ab050 2024-04-17T22:31:28.763 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.776 7fd360ff9700 1 -- 172.21.15.102:0/4155913342 <== mon.1 v2:172.21.15.102:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+8777 (secure 0 0 0) 0x7fd34c000c30 con 0x7fd3641ab050 2024-04-17T22:31:28.763 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-17T22:31:28.763 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":24,"fsid":"cb63e789-091d-49b1-b120-55f9db526871","created":"2024-04-17 22:28:48.209635","modified":"2024-04-17 22:31:28.204618","last_up_change":"2024-04-17 22:28:54.236215","last_in_change":"2024-04-17 22:28:54.236215","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":4,"require_min_compat_client":"nautilus","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-04-17 22:28:57.478545","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-17 22:31:26.214102","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-17 22:31:27.183113","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3351ef85-ade4-4b90-a8f4-213f4e00ba01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6812","nonce":42955}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6813","nonce":42955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6815","nonce":42955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6814","nonce":42955}]},"public_addr":"172.21.15.102:6812/42955","cluster_addr":"172.21.15.102:6813/42955","heartbeat_back_addr":"172.21.15.102:6815/42955","heartbeat_front_addr":"172.21.15.102:6814/42955","state":["exists","up"]},{"osd":1,"uuid":"04f67eac-f36b-48f5-982d-4cfc776596d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6808","nonce":42958}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6809","nonce":42958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6811","nonce":42958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6810","nonce":42958}]},"public_addr":"172.21.15.102:6808/42958","cluster_addr":"172.21.15.102:6809/42958","heartbeat_back_addr":"172.21.15.102:6811/42958","heartbeat_front_addr":"172.21.15.102:6810/42958","state":["exists","up"]},{"osd":2,"uuid":"d524204b-70b2-40a8-8f73-f9e7d44318af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6804","nonce":42954}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6805","nonce":42954}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6807","nonce":42954}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6806","nonce":42954}]},"public_addr":"172.21.15.102:6804/42954","cluster_addr":"172.21.15.102:6805/42954","heartbeat_back_addr":"172.21.15.102:6807/42954","heartbeat_front_addr":"172.21.15.102:6806/42954","state":["exists","up"]},{"osd":3,"uuid":"4fc9d20c-1ad0-484f-aafa-2c9a7574245b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6800","nonce":42953}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6801","nonce":42953}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6803","nonce":42953}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6802","nonce":42953}]},"public_addr":"172.21.15.102:6800/42953","cluster_addr":"172.21.15.102:6801/42953","heartbeat_back_addr":"172.21.15.102:6803/42953","heartbeat_front_addr":"172.21.15.102:6802/42953","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2024-04-17T22:31:28.764 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.778 7fd36ace7700 1 -- 172.21.15.102:0/4155913342 >> v1:172.21.15.102:6816/42360 conn(0x7fd3480145a0 legacy=0x7fd348016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:28.764 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.778 7fd36ace7700 1 -- 172.21.15.102:0/4155913342 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd3641ab050 msgr2=0x7fd3641ab4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:28.764 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.778 7fd36ace7700 1 --2- 172.21.15.102:0/4155913342 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd3641ab050 0x7fd3641ab4a0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 rx=0x7fd3640d6e00 tx=0x7fd34c007f70).stop 2024-04-17T22:31:28.764 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.778 7fd36ace7700 1 -- 172.21.15.102:0/4155913342 shutdown_connections 2024-04-17T22:31:28.764 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.778 7fd36ace7700 1 --2- 172.21.15.102:0/4155913342 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd3641b2110 0x7fd3641b2560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:28.765 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.778 7fd36ace7700 1 --2- 172.21.15.102:0/4155913342 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd3641ab050 0x7fd3641ab4a0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:28.765 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.778 7fd36ace7700 1 --2- 172.21.15.102:0/4155913342 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd3641abad0 0x7fd3641b1ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:28.765 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.778 7fd36ace7700 1 -- 172.21.15.102:0/4155913342 >> 172.21.15.102:0/4155913342 conn(0x7fd3640e0dc0 msgr2=0x7fd3640daae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:28.765 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.778 7fd36ace7700 1 -- 172.21.15.102:0/4155913342 shutdown_connections 2024-04-17T22:31:28.765 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.778 7fd36ace7700 1 -- 172.21.15.102:0/4155913342 wait complete. 2024-04-17T22:31:28.775 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-17T22:31:28.905 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.916 7f2c8d592700 1 Processor -- start 2024-04-17T22:31:28.905 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.917 7f2c8d592700 1 -- start start 2024-04-17T22:31:28.905 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.917 7f2c8d592700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f2c8810e240 0x7f2c8810e690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:28.905 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.917 7f2c8d592700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2c8810ecc0 0x7f2c880f1730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:28.905 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.917 7f2c8d592700 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c880f21b0 con 0x7f2c880f1d60 2024-04-17T22:31:28.905 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.917 7f2c8d592700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c880f9f10 con 0x7f2c8810ecc0 2024-04-17T22:31:28.905 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.917 7f2c8d592700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2c880f9030 con 0x7f2c8810e240 2024-04-17T22:31:28.905 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.917 7f2c85ab0700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2c8810ecc0 0x7f2c880f1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:28.906 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.917 7f2c86ab2700 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f2c880f1d60 0x7f2c881aa4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:33772/0 (socket says 172.21.15.102:33772) 2024-04-17T22:31:28.906 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.917 7f2c86ab2700 1 -- 172.21.15.102:0/3477412442 learned_addr learned my addr 172.21.15.102:0/3477412442 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-17T22:31:28.906 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.917 7f2c862b1700 1 --2- 172.21.15.102:0/3477412442 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f2c8810e240 0x7f2c8810e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:28.906 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.917 7f2c862b1700 1 -- 172.21.15.102:0/3477412442 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2c8810ecc0 msgr2=0x7f2c880f1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:31:28.906 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.917 7f2c862b1700 1 --2- 172.21.15.102:0/3477412442 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2c8810ecc0 0x7f2c880f1730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:28.906 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.917 7f2c862b1700 1 -- 172.21.15.102:0/3477412442 >> v1:172.21.15.102:6789/0 conn(0x7f2c880f1d60 legacy=0x7f2c881aa4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:28.906 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.917 7f2c862b1700 1 -- 172.21.15.102:0/3477412442 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c880f23d0 con 0x7f2c8810e240 2024-04-17T22:31:28.906 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.917 7f2c852af700 1 -- 172.21.15.102:0/3477412442 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4136215097 0 0) 0x7f2c80001490 con 0x7f2c880f1d60 2024-04-17T22:31:28.906 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.917 7f2c85ab0700 1 --2- 172.21.15.102:0/3477412442 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2c8810ecc0 0x7f2c880f1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:28.906 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.918 7f2c862b1700 1 --2- 172.21.15.102:0/3477412442 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f2c8810e240 0x7f2c8810e690 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 rx=0x7f2c7c002f70 tx=0x7f2c7c00b9a0).ready entity=mon.2 client_cookie=bc07b0f649cceb0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:28.906 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.918 7f2c852af700 1 -- 172.21.15.102:0/3477412442 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f2c880f23d0 con 0x7f2c8810e240 2024-04-17T22:31:28.906 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.918 7f2c852af700 1 -- 172.21.15.102:0/3477412442 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c7c00d570 con 0x7f2c8810e240 2024-04-17T22:31:28.906 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.918 7f2c852af700 1 -- 172.21.15.102:0/3477412442 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f2c7c00d940 con 0x7f2c8810e240 2024-04-17T22:31:28.907 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.918 7f2c8d592700 1 -- 172.21.15.102:0/3477412442 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f2c8810e240 msgr2=0x7f2c8810e690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:28.907 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.918 7f2c8d592700 1 --2- 172.21.15.102:0/3477412442 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f2c8810e240 0x7f2c8810e690 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 rx=0x7f2c7c002f70 tx=0x7f2c7c00b9a0).stop 2024-04-17T22:31:28.907 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.918 7f2c8d592700 1 -- 172.21.15.102:0/3477412442 shutdown_connections 2024-04-17T22:31:28.907 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.918 7f2c8d592700 1 --2- 172.21.15.102:0/3477412442 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f2c8810e240 0x7f2c8810e690 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:28.907 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.918 7f2c8d592700 1 --2- 172.21.15.102:0/3477412442 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2c8810ecc0 0x7f2c880f1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:28.907 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.918 7f2c8d592700 1 -- 172.21.15.102:0/3477412442 >> 172.21.15.102:0/3477412442 conn(0x7f2c881138a0 msgr2=0x7f2c880f4f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:28.907 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.918 7f2c8d592700 1 -- 172.21.15.102:0/3477412442 shutdown_connections 2024-04-17T22:31:28.907 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.918 7f2c8d592700 1 -- 172.21.15.102:0/3477412442 wait complete. 2024-04-17T22:31:28.929 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.941 7f2c8d592700 1 Processor -- start 2024-04-17T22:31:28.929 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.941 7f2c8d592700 1 -- start start 2024-04-17T22:31:28.930 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.941 7f2c8d592700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2c880da130 0x7f2c880d51d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:28.930 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.941 7f2c8d592700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f2c880d5800 0x7f2c881b3ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:28.930 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.941 7f2c8d592700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2c880d5c50 0x7f2c881b44d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:28.930 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.941 7f2c8d592700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c880fa1b0 con 0x7f2c880da130 2024-04-17T22:31:28.930 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.941 7f2c8d592700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c880d9ad0 con 0x7f2c880d5c50 2024-04-17T22:31:28.930 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.941 7f2c8d592700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2c880da580 con 0x7f2c880d5800 2024-04-17T22:31:28.930 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.942 7f2c86ab2700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2c880d5c50 0x7f2c881b44d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:28.930 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.942 7f2c85ab0700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f2c880d5800 0x7f2c881b3ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:28.930 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.942 7f2c86ab2700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2c880d5c50 0x7f2c881b44d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:32790/0 (socket says 172.21.15.102:32790) 2024-04-17T22:31:28.930 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.942 7f2c85ab0700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f2c880d5800 0x7f2c881b3ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:33722/0 (socket says 172.21.15.102:33722) 2024-04-17T22:31:28.930 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.942 7f2c86ab2700 1 -- 172.21.15.102:0/1331452774 learned_addr learned my addr 172.21.15.102:0/1331452774 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:31:28.930 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.942 7f2c862b1700 1 --2- 172.21.15.102:0/1331452774 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2c880da130 0x7f2c880d51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:28.930 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.942 7f2c86ab2700 1 -- 172.21.15.102:0/1331452774 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f2c880d5800 msgr2=0x7f2c881b3ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:28.931 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.942 7f2c86ab2700 1 --2- 172.21.15.102:0/1331452774 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f2c880d5800 0x7f2c881b3ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:28.931 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.942 7f2c86ab2700 1 -- 172.21.15.102:0/1331452774 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2c880da130 msgr2=0x7f2c880d51d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:28.931 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.942 7f2c86ab2700 1 --2- 172.21.15.102:0/1331452774 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2c880da130 0x7f2c880d51d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:28.931 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.942 7f2c86ab2700 1 -- 172.21.15.102:0/1331452774 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c881b4b00 con 0x7f2c880d5c50 2024-04-17T22:31:28.931 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.942 7f2c862b1700 1 --2- 172.21.15.102:0/1331452774 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2c880da130 0x7f2c880d51d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:28.931 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.942 7f2c85ab0700 1 --2- 172.21.15.102:0/1331452774 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f2c880d5800 0x7f2c881b3ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:28.931 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.942 7f2c86ab2700 1 --2- 172.21.15.102:0/1331452774 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2c880d5c50 0x7f2c881b44d0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 rx=0x7f2c880c5760 tx=0x7f2c8000cc10).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:28.931 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.942 7f2c777fe700 1 -- 172.21.15.102:0/1331452774 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f2c881b4b00 con 0x7f2c880d5c50 2024-04-17T22:31:28.931 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.942 7f2c777fe700 1 -- 172.21.15.102:0/1331452774 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c800059a0 con 0x7f2c880d5c50 2024-04-17T22:31:28.931 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.942 7f2c8d592700 1 -- 172.21.15.102:0/1331452774 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c881b4e20 con 0x7f2c880d5c50 2024-04-17T22:31:28.931 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.942 7f2c777fe700 1 -- 172.21.15.102:0/1331452774 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f2c80005d70 con 0x7f2c880d5c50 2024-04-17T22:31:28.931 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.942 7f2c8d592700 1 -- 172.21.15.102:0/1331452774 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c881bb7f0 con 0x7f2c880d5c50 2024-04-17T22:31:28.931 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.943 7f2c777fe700 1 -- 172.21.15.102:0/1331452774 <== mon.1 v2:172.21.15.102:3301/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7f2c8001a020 con 0x7f2c880d5c50 2024-04-17T22:31:28.932 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.943 7f2c777fe700 1 -- 172.21.15.102:0/1331452774 <== mon.1 v2:172.21.15.102:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 3194+0+0 (secure 0 0 0) 0x7f2c80009d90 con 0x7f2c880d5c50 2024-04-17T22:31:28.932 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.943 7f2c8d592700 1 -- 172.21.15.102:0/1331452774 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c880f0800 con 0x7f2c880d5c50 2024-04-17T22:31:28.932 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:28.946 7f2c777fe700 1 -- 172.21.15.102:0/1331452774 <== mon.1 v2:172.21.15.102:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+110455 (secure 0 0 0) 0x7f2c8000cd50 con 0x7f2c880d5c50 2024-04-17T22:31:29.068 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:29.082 7f2c8d592700 1 -- 172.21.15.102:0/1331452774 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f2c880efed0 con 0x7f2c880d5c50 2024-04-17T22:31:29.201 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:29.215 7f2c777fe700 1 -- 172.21.15.102:0/1331452774 <== mon.1 v2:172.21.15.102:3301/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v10) v1 ==== 118+0+0 (secure 0 0 0) 0x7f2c88114420 con 0x7f2c880d5c50 2024-04-17T22:31:29.203 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:29.217 7f2c8d592700 1 -- 172.21.15.102:0/1331452774 >> v1:172.21.15.102:6816/42360 conn(0x7f2c64014540 legacy=0x7f2c64016a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:29.203 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:29.217 7f2c8d592700 1 -- 172.21.15.102:0/1331452774 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2c880d5c50 msgr2=0x7f2c881b44d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:29.203 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:29.217 7f2c8d592700 1 --2- 172.21.15.102:0/1331452774 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2c880d5c50 0x7f2c881b44d0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 rx=0x7f2c880c5760 tx=0x7f2c8000cc10).stop 2024-04-17T22:31:29.203 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:29.217 7f2c8d592700 1 -- 172.21.15.102:0/1331452774 shutdown_connections 2024-04-17T22:31:29.203 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:29.217 7f2c8d592700 1 --2- 172.21.15.102:0/1331452774 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2c880d5c50 0x7f2c881b44d0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:29.203 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:29.217 7f2c8d592700 1 --2- 172.21.15.102:0/1331452774 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2c880da130 0x7f2c880d51d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:29.204 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:29.217 7f2c8d592700 1 --2- 172.21.15.102:0/1331452774 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f2c880d5800 0x7f2c881b3ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:29.204 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:29.217 7f2c8d592700 1 -- 172.21.15.102:0/1331452774 >> 172.21.15.102:0/1331452774 conn(0x7f2c880c6ad0 msgr2=0x7f2c881a82e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:29.204 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:29.217 7f2c8d592700 1 -- 172.21.15.102:0/1331452774 shutdown_connections 2024-04-17T22:31:29.204 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:29.217 7f2c8d592700 1 -- 172.21.15.102:0/1331452774 wait complete. 2024-04-17T22:31:29.214 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-17T22:31:29.214 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-17T22:31:29.343 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:29.354 7f6231668700 1 Processor -- start 2024-04-17T22:31:29.343 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:29.355 7f6231668700 1 -- start start 2024-04-17T22:31:29.343 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:29.355 7f6231668700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f622c1038e0 0x7f622c1a8610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:29.343 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:29.355 7f6231668700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f622c1a8c40 0x7f622c1ab0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:29.343 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:29.355 7f6231668700 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f622c0ad030 con 0x7f622c0f95b0 2024-04-17T22:31:29.343 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:29.355 7f6231668700 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f622c0f2390 con 0x7f622c1a8c40 2024-04-17T22:31:29.343 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:29.355 7f6231668700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f622c1042c0 con 0x7f622c1038e0 2024-04-17T22:31:29.344 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:29.355 7f6229ab0700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f622c1038e0 0x7f622c1a8610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:29.344 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:29.355 7f622aab2700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f622c1a8c40 0x7f622c1ab0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:29.344 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:29.355 7f622aab2700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f622c1a8c40 0x7f622c1ab0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:52064/0 (socket says 172.21.15.102:52064) 2024-04-17T22:31:29.344 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:29.355 7f6229ab0700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f622c1038e0 0x7f622c1a8610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:33736/0 (socket says 172.21.15.102:33736) 2024-04-17T22:31:29.344 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:29.355 7f622aab2700 1 -- 172.21.15.102:0/3612244058 learned_addr learned my addr 172.21.15.102:0/3612244058 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:31:29.344 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:29.355 7f622aab2700 1 -- 172.21.15.102:0/3612244058 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f622c1038e0 msgr2=0x7f622c1a8610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:31:29.344 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:29.355 7f622aab2700 1 --2- 172.21.15.102:0/3612244058 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f622c1038e0 0x7f622c1a8610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:29.344 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:29.355 7f622aab2700 1 -- 172.21.15.102:0/3612244058 >> v1:172.21.15.102:6789/0 conn(0x7f622c0f95b0 legacy=0x7f622c1030e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:29.344 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:29.355 7f622aab2700 1 -- 172.21.15.102:0/3612244058 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f622c103d30 con 0x7f622c1a8c40 2024-04-17T22:31:29.344 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:29.355 7f62292af700 1 -- 172.21.15.102:0/3612244058 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1533507972 0 0) 0x7f6214001110 con 0x7f622c0f95b0 2024-04-17T22:31:29.344 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:29.356 7f622aab2700 1 --2- 172.21.15.102:0/3612244058 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f622c1a8c40 0x7f622c1ab0a0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 rx=0x7f62240032f0 tx=0x7f6224009de0).ready entity=mon.0 client_cookie=267bfe3512ed4518 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:29.344 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:29.356 7f62292af700 1 -- 172.21.15.102:0/3612244058 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f622c103d30 con 0x7f622c1a8c40 2024-04-17T22:31:29.345 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:29.356 7f62292af700 1 -- 172.21.15.102:0/3612244058 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f622400b8d0 con 0x7f622c1a8c40 2024-04-17T22:31:29.345 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:29.356 7f62292af700 1 -- 172.21.15.102:0/3612244058 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f622400bab0 con 0x7f622c1a8c40 2024-04-17T22:31:29.345 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:29.356 7f6231668700 1 -- 172.21.15.102:0/3612244058 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f622c1a8c40 msgr2=0x7f622c1ab0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:29.345 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:29.356 7f6231668700 1 --2- 172.21.15.102:0/3612244058 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f622c1a8c40 0x7f622c1ab0a0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 rx=0x7f62240032f0 tx=0x7f6224009de0).stop 2024-04-17T22:31:29.345 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:29.356 7f6231668700 1 -- 172.21.15.102:0/3612244058 shutdown_connections 2024-04-17T22:31:29.345 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:29.356 7f6231668700 1 --2- 172.21.15.102:0/3612244058 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f622c1a8c40 0x7f622c1ab0a0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:29.345 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:29.356 7f6231668700 1 --2- 172.21.15.102:0/3612244058 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f622c1038e0 0x7f622c1a8610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:29.345 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:29.356 7f6231668700 1 -- 172.21.15.102:0/3612244058 >> 172.21.15.102:0/3612244058 conn(0x7f622c113000 msgr2=0x7f622c113450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:29.345 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:29.356 7f6231668700 1 -- 172.21.15.102:0/3612244058 shutdown_connections 2024-04-17T22:31:29.345 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:29.356 7f6231668700 1 -- 172.21.15.102:0/3612244058 wait complete. 2024-04-17T22:31:29.367 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:29.379 7f6231668700 1 Processor -- start 2024-04-17T22:31:29.367 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:29.379 7f6231668700 1 -- start start 2024-04-17T22:31:29.367 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:29.380 7f6231668700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f622c1a0210 0x7f622c1a0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:29.367 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:29.380 7f6231668700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f622c1b3d20 0x7f622c1b4170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:29.367 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:29.380 7f6231668700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f622c1b47a0 0x7f622c1bb200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:29.367 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:29.380 7f6231668700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f622c0e4100 con 0x7f622c1b47a0 2024-04-17T22:31:29.367 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:29.380 7f6231668700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f622c0de2f0 con 0x7f622c1a0210 2024-04-17T22:31:29.367 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:29.380 7f6231668700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f622c0d0a20 con 0x7f622c1b3d20 2024-04-17T22:31:29.367 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:29.380 7f622aab2700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f622c1b47a0 0x7f622c1bb200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:29.368 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:29.380 7f6229ab0700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f622c1b3d20 0x7f622c1b4170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:29.368 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:29.380 7f622aab2700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f622c1b47a0 0x7f622c1bb200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:52072/0 (socket says 172.21.15.102:52072) 2024-04-17T22:31:29.368 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:29.380 7f622aab2700 1 -- 172.21.15.102:0/2630496585 learned_addr learned my addr 172.21.15.102:0/2630496585 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:31:29.368 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:29.380 7f622a2b1700 1 --2- 172.21.15.102:0/2630496585 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f622c1a0210 0x7f622c1a0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:29.368 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:29.380 7f622aab2700 1 -- 172.21.15.102:0/2630496585 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f622c1b3d20 msgr2=0x7f622c1b4170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:29.368 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:29.380 7f622aab2700 1 --2- 172.21.15.102:0/2630496585 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f622c1b3d20 0x7f622c1b4170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:29.368 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:29.380 7f622aab2700 1 -- 172.21.15.102:0/2630496585 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f622c1a0210 msgr2=0x7f622c1a0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:29.368 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:29.380 7f622aab2700 1 --2- 172.21.15.102:0/2630496585 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f622c1a0210 0x7f622c1a0660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:29.368 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:29.380 7f622aab2700 1 -- 172.21.15.102:0/2630496585 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f622c0ddc50 con 0x7f622c1b47a0 2024-04-17T22:31:29.368 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:29.380 7f622aab2700 1 --2- 172.21.15.102:0/2630496585 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f622c1b47a0 0x7f622c1bb200 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 rx=0x7f622c0de4a0 tx=0x7f6224006c40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:29.368 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:29.380 7f62237fe700 1 -- 172.21.15.102:0/2630496585 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f622c0ddc50 con 0x7f622c1b47a0 2024-04-17T22:31:29.368 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:29.380 7f62237fe700 1 -- 172.21.15.102:0/2630496585 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6224011d90 con 0x7f622c1b47a0 2024-04-17T22:31:29.368 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:29.380 7f6231668700 1 -- 172.21.15.102:0/2630496585 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f622c1b4bf0 con 0x7f622c1b47a0 2024-04-17T22:31:29.368 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:29.380 7f62237fe700 1 -- 172.21.15.102:0/2630496585 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f6224017040 con 0x7f622c1b47a0 2024-04-17T22:31:29.368 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:29.380 7f6231668700 1 -- 172.21.15.102:0/2630496585 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f622c0deb00 con 0x7f622c1b47a0 2024-04-17T22:31:29.369 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:29.381 7f62237fe700 1 -- 172.21.15.102:0/2630496585 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7f622401c660 con 0x7f622c1b47a0 2024-04-17T22:31:29.369 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:29.381 7f62237fe700 1 -- 172.21.15.102:0/2630496585 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 3194+0+0 (secure 0 0 0) 0x7f6224018030 con 0x7f622c1b47a0 2024-04-17T22:31:29.369 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:29.381 7f6231668700 1 -- 172.21.15.102:0/2630496585 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f622c0f09f0 con 0x7f622c1b47a0 2024-04-17T22:31:29.370 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:29.385 7f62237fe700 1 -- 172.21.15.102:0/2630496585 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+110455 (secure 0 0 0) 0x7f6224011b80 con 0x7f622c1b47a0 2024-04-17T22:31:29.510 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:29.524 7f6231668700 1 -- 172.21.15.102:0/2630496585 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f622c0a64e0 con 0x7f622c1b47a0 2024-04-17T22:31:30.205 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.219 7f62237fe700 1 -- 172.21.15.102:0/2630496585 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v11) v1 ==== 122+0+0 (secure 0 0 0) 0x7f622400b6c0 con 0x7f622c1b47a0 2024-04-17T22:31:30.207 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.220 7f6231668700 1 -- 172.21.15.102:0/2630496585 >> v1:172.21.15.102:6816/42360 conn(0x7f62100145a0 legacy=0x7f6210016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:30.207 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.221 7f6231668700 1 -- 172.21.15.102:0/2630496585 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f622c1b47a0 msgr2=0x7f622c1bb200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:30.207 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.221 7f6231668700 1 --2- 172.21.15.102:0/2630496585 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f622c1b47a0 0x7f622c1bb200 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 rx=0x7f622c0de4a0 tx=0x7f6224006c40).stop 2024-04-17T22:31:30.207 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.221 7f6231668700 1 -- 172.21.15.102:0/2630496585 shutdown_connections 2024-04-17T22:31:30.207 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.221 7f6231668700 1 --2- 172.21.15.102:0/2630496585 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f622c1b47a0 0x7f622c1bb200 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:30.207 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.221 7f6231668700 1 --2- 172.21.15.102:0/2630496585 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f622c1a0210 0x7f622c1a0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:30.207 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.221 7f6231668700 1 --2- 172.21.15.102:0/2630496585 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f622c1b3d20 0x7f622c1b4170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:30.207 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.221 7f6231668700 1 -- 172.21.15.102:0/2630496585 >> 172.21.15.102:0/2630496585 conn(0x7f622c1a71b0 msgr2=0x7f622c0d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:30.208 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.221 7f6231668700 1 -- 172.21.15.102:0/2630496585 shutdown_connections 2024-04-17T22:31:30.208 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.221 7f6231668700 1 -- 172.21.15.102:0/2630496585 wait complete. 2024-04-17T22:31:30.222 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-17T22:31:30.222 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T22:31:30.352 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.363 7fe68efaf700 1 Processor -- start 2024-04-17T22:31:30.352 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.363 7fe68efaf700 1 -- start start 2024-04-17T22:31:30.352 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.364 7fe68efaf700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fe6881a6440 0x7fe6881a88a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:30.352 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.364 7fe68efaf700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fe6881a8ed0 0x7fe6881ab330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:30.352 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.364 7fe68efaf700 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe68810ee20 con 0x7fe68810e1d0 2024-04-17T22:31:30.352 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.364 7fe68efaf700 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6880f9ea0 con 0x7fe6881a8ed0 2024-04-17T22:31:30.352 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.364 7fe68efaf700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe6880f8fc0 con 0x7fe6881a6440 2024-04-17T22:31:30.352 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.364 7fe686ffd700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fe6881a6440 0x7fe6881a88a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:30.352 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.364 7fe687fff700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fe6881a8ed0 0x7fe6881ab330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:30.352 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.364 7fe6877fe700 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7fe68810e1d0 0x7fe68810e620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:33792/0 (socket says 172.21.15.102:33792) 2024-04-17T22:31:30.352 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.364 7fe6877fe700 1 -- 172.21.15.102:0/1576835992 learned_addr learned my addr 172.21.15.102:0/1576835992 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-17T22:31:30.353 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.364 7fe686ffd700 1 -- 172.21.15.102:0/1576835992 >> v2:172.21.15.102:3300/0 conn(0x7fe6881a8ed0 msgr2=0x7fe6881ab330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:31:30.353 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.364 7fe686ffd700 1 --2- 172.21.15.102:0/1576835992 >> v2:172.21.15.102:3300/0 conn(0x7fe6881a8ed0 0x7fe6881ab330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:30.353 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.364 7fe686ffd700 1 -- 172.21.15.102:0/1576835992 >> v1:172.21.15.102:6789/0 conn(0x7fe68810e1d0 legacy=0x7fe68810e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:30.353 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.364 7fe686ffd700 1 -- 172.21.15.102:0/1576835992 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6881ac970 con 0x7fe6881a6440 2024-04-17T22:31:30.353 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.364 7fe6867fc700 1 -- 172.21.15.102:0/1576835992 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3976859774 0 0) 0x7fe67c001110 con 0x7fe68810e1d0 2024-04-17T22:31:30.353 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.364 7fe687fff700 1 --2- 172.21.15.102:0/1576835992 >> v2:172.21.15.102:3300/0 conn(0x7fe6881a8ed0 0x7fe6881ab330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:30.353 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.364 7fe686ffd700 1 --2- 172.21.15.102:0/1576835992 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fe6881a6440 0x7fe6881a88a0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 rx=0x7fe678002f70 tx=0x7fe67800b970).ready entity=mon.2 client_cookie=2fb29c0caf7c3556 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:30.353 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.364 7fe6867fc700 1 -- 172.21.15.102:0/1576835992 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe6881ac970 con 0x7fe6881a6440 2024-04-17T22:31:30.353 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.365 7fe6867fc700 1 -- 172.21.15.102:0/1576835992 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe67800d610 con 0x7fe6881a6440 2024-04-17T22:31:30.353 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.365 7fe6867fc700 1 -- 172.21.15.102:0/1576835992 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe67800d9e0 con 0x7fe6881a6440 2024-04-17T22:31:30.353 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.365 7fe68efaf700 1 -- 172.21.15.102:0/1576835992 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fe6881a6440 msgr2=0x7fe6881a88a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:30.353 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.365 7fe68efaf700 1 --2- 172.21.15.102:0/1576835992 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fe6881a6440 0x7fe6881a88a0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 rx=0x7fe678002f70 tx=0x7fe67800b970).stop 2024-04-17T22:31:30.354 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.365 7fe68efaf700 1 -- 172.21.15.102:0/1576835992 shutdown_connections 2024-04-17T22:31:30.354 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.365 7fe68efaf700 1 --2- 172.21.15.102:0/1576835992 >> v2:172.21.15.102:3300/0 conn(0x7fe6881a8ed0 0x7fe6881ab330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:30.354 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.365 7fe68efaf700 1 --2- 172.21.15.102:0/1576835992 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fe6881a6440 0x7fe6881a88a0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:30.354 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.365 7fe68efaf700 1 -- 172.21.15.102:0/1576835992 >> 172.21.15.102:0/1576835992 conn(0x7fe688113830 msgr2=0x7fe6880f4ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:30.354 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.365 7fe68efaf700 1 -- 172.21.15.102:0/1576835992 shutdown_connections 2024-04-17T22:31:30.354 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.365 7fe68efaf700 1 -- 172.21.15.102:0/1576835992 wait complete. 2024-04-17T22:31:30.376 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.387 7fe68efaf700 1 Processor -- start 2024-04-17T22:31:30.376 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.388 7fe68efaf700 1 -- start start 2024-04-17T22:31:30.376 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.388 7fe68efaf700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe6880d5120 0x7fe6880d5570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:30.376 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.388 7fe68efaf700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe6880d5ba0 0x7fe6880cbb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:30.376 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.388 7fe68efaf700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fe6880cc150 0x7fe6881bb8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:30.376 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.388 7fe68efaf700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe688113a40 con 0x7fe6880d5120 2024-04-17T22:31:30.376 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.388 7fe68efaf700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6880fa140 con 0x7fe6880d5ba0 2024-04-17T22:31:30.376 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.388 7fe68efaf700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe6880c4660 con 0x7fe6880cc150 2024-04-17T22:31:30.376 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.388 7fe6877fe700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe6880d5120 0x7fe6880d5570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:30.376 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.388 7fe686ffd700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe6880d5ba0 0x7fe6880cbb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:30.377 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.388 7fe6877fe700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe6880d5120 0x7fe6880d5570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:52104/0 (socket says 172.21.15.102:52104) 2024-04-17T22:31:30.377 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.388 7fe6877fe700 1 -- 172.21.15.102:0/537414020 learned_addr learned my addr 172.21.15.102:0/537414020 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:31:30.377 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.388 7fe687fff700 1 --2- 172.21.15.102:0/537414020 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fe6880cc150 0x7fe6881bb8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:30.377 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.389 7fe686ffd700 1 -- 172.21.15.102:0/537414020 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fe6880cc150 msgr2=0x7fe6881bb8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:30.377 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.389 7fe686ffd700 1 --2- 172.21.15.102:0/537414020 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fe6880cc150 0x7fe6881bb8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:30.377 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.389 7fe686ffd700 1 -- 172.21.15.102:0/537414020 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe6880d5120 msgr2=0x7fe6880d5570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:30.377 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.389 7fe686ffd700 1 --2- 172.21.15.102:0/537414020 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe6880d5120 0x7fe6880d5570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:30.377 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.389 7fe686ffd700 1 -- 172.21.15.102:0/537414020 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6880cb4f0 con 0x7fe6880d5ba0 2024-04-17T22:31:30.377 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.389 7fe6877fe700 1 --2- 172.21.15.102:0/537414020 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe6880d5120 0x7fe6880d5570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:30.377 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.389 7fe686ffd700 1 --2- 172.21.15.102:0/537414020 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe6880d5ba0 0x7fe6880cbb20 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 rx=0x7fe6880c4f50 tx=0x7fe67800a040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:30.377 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.389 7fe684ff9700 1 -- 172.21.15.102:0/537414020 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe6880cb4f0 con 0x7fe6880d5ba0 2024-04-17T22:31:30.378 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.389 7fe684ff9700 1 -- 172.21.15.102:0/537414020 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe678013850 con 0x7fe6880d5ba0 2024-04-17T22:31:30.378 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.389 7fe684ff9700 1 -- 172.21.15.102:0/537414020 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe678013c20 con 0x7fe6880d5ba0 2024-04-17T22:31:30.378 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.389 7fe68efaf700 1 -- 172.21.15.102:0/537414020 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6880cc5a0 con 0x7fe6880d5ba0 2024-04-17T22:31:30.378 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.389 7fe68efaf700 1 -- 172.21.15.102:0/537414020 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6881bbf10 con 0x7fe6880d5ba0 2024-04-17T22:31:30.378 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.389 7fe684ff9700 1 -- 172.21.15.102:0/537414020 <== mon.1 v2:172.21.15.102:3301/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7fe678027070 con 0x7fe6880d5ba0 2024-04-17T22:31:30.378 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.389 7fe684ff9700 1 -- 172.21.15.102:0/537414020 <== mon.1 v2:172.21.15.102:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 3194+0+0 (secure 0 0 0) 0x7fe678008820 con 0x7fe6880d5ba0 2024-04-17T22:31:30.378 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.390 7fe68efaf700 1 -- 172.21.15.102:0/537414020 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6880f0790 con 0x7fe6880d5ba0 2024-04-17T22:31:30.379 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.393 7fe684ff9700 1 -- 172.21.15.102:0/537414020 <== mon.1 v2:172.21.15.102:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+110455 (secure 0 0 0) 0x7fe678013640 con 0x7fe6880d5ba0 2024-04-17T22:31:30.514 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.528 7fe68efaf700 1 -- 172.21.15.102:0/537414020 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe6881ae910 con 0x7fe6880d5ba0 2024-04-17T22:31:30.514 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.528 7fe684ff9700 1 -- 172.21.15.102:0/537414020 <== mon.1 v2:172.21.15.102:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) v1 ==== 94+0+1704 (secure 0 0 0) 0x7fe67800d400 con 0x7fe6880d5ba0 2024-04-17T22:31:30.515 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-17T22:31:30.515 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":11,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[],"filesystems":[{"mdsmap":{"epoch":11,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17 22:31:28.204771","modified":"2024-04-17 22:31:30.213118","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"-1 (unspecified)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5319},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5319":{"gid":5319,"name":"a","rank":0,"incarnation":9,"state":"up:active","state_seq":3,"addr":"172.21.15.102:6817/479367134","addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6817","nonce":479367134}]},"export_targets":[],"features":4611087854035861503,"flags":0}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2024-04-17T22:31:30.516 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.529 7fe68efaf700 1 -- 172.21.15.102:0/537414020 >> v1:172.21.15.102:6816/42360 conn(0x7fe6680145a0 legacy=0x7fe668016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:30.516 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.529 7fe68efaf700 1 -- 172.21.15.102:0/537414020 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe6880d5ba0 msgr2=0x7fe6880cbb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:30.516 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.529 7fe68efaf700 1 --2- 172.21.15.102:0/537414020 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe6880d5ba0 0x7fe6880cbb20 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 rx=0x7fe6880c4f50 tx=0x7fe67800a040).stop 2024-04-17T22:31:30.516 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.529 7fe68efaf700 1 -- 172.21.15.102:0/537414020 shutdown_connections 2024-04-17T22:31:30.516 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.529 7fe68efaf700 1 --2- 172.21.15.102:0/537414020 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fe6880cc150 0x7fe6881bb8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:30.516 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.529 7fe68efaf700 1 --2- 172.21.15.102:0/537414020 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe6880d5120 0x7fe6880d5570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:30.516 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.529 7fe68efaf700 1 --2- 172.21.15.102:0/537414020 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe6880d5ba0 0x7fe6880cbb20 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:30.517 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.529 7fe68efaf700 1 -- 172.21.15.102:0/537414020 >> 172.21.15.102:0/537414020 conn(0x7fe6880c6a60 msgr2=0x7fe6880d6200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:30.517 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.529 7fe68efaf700 1 -- 172.21.15.102:0/537414020 shutdown_connections 2024-04-17T22:31:30.517 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.529 7fe68efaf700 1 -- 172.21.15.102:0/537414020 wait complete. 2024-04-17T22:31:30.517 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 11 2024-04-17T22:31:30.527 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T22:31:30.655 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.666 7efe8208f700 1 Processor -- start 2024-04-17T22:31:30.656 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.666 7efe8208f700 1 -- start start 2024-04-17T22:31:30.656 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.666 7efe8208f700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7efe7c10b7a0 0x7efe7c1a5250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:30.656 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.666 7efe8208f700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7efe7c10bbf0 0x7efe7c1a7890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:30.656 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.666 7efe8208f700 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe7c103ad0 con 0x7efe7c0f57d0 2024-04-17T22:31:30.656 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.666 7efe8208f700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe7c0fa300 con 0x7efe7c10b7a0 2024-04-17T22:31:30.656 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.666 7efe8208f700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7efe7c1a0a90 con 0x7efe7c10bbf0 2024-04-17T22:31:30.656 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.666 7efe7a5c5700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7efe7c10b7a0 0x7efe7c1a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:30.656 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.666 7efe7b5c7700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7efe7c10bbf0 0x7efe7c1a7890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:30.656 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.667 7efe7adc6700 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7efe7c0f57d0 0x7efe7c10afa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:33798/0 (socket says 172.21.15.102:33798) 2024-04-17T22:31:30.657 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.667 7efe7a5c5700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7efe7c10b7a0 0x7efe7c1a5250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:32816/0 (socket says 172.21.15.102:32816) 2024-04-17T22:31:30.657 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.667 7efe7adc6700 1 -- 172.21.15.102:0/3182878993 learned_addr learned my addr 172.21.15.102:0/3182878993 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-17T22:31:30.657 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.667 7efe79dc4700 1 -- 172.21.15.102:0/3182878993 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3655478535 0 0) 0x7efe70001110 con 0x7efe7c0f57d0 2024-04-17T22:31:30.657 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.667 7efe7b5c7700 1 -- 172.21.15.102:0/3182878993 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7efe7c10b7a0 msgr2=0x7efe7c1a5250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:31:30.657 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.667 7efe7b5c7700 1 --2- 172.21.15.102:0/3182878993 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7efe7c10b7a0 0x7efe7c1a5250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:30.657 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.667 7efe7b5c7700 1 -- 172.21.15.102:0/3182878993 >> v1:172.21.15.102:6789/0 conn(0x7efe7c0f57d0 legacy=0x7efe7c10afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:30.657 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.667 7efe7b5c7700 1 -- 172.21.15.102:0/3182878993 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe7c1a8ed0 con 0x7efe7c10bbf0 2024-04-17T22:31:30.658 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.667 7efe7a5c5700 1 --2- 172.21.15.102:0/3182878993 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7efe7c10b7a0 0x7efe7c1a5250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:30.658 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.667 7efe7b5c7700 1 --2- 172.21.15.102:0/3182878993 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7efe7c10bbf0 0x7efe7c1a7890 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 rx=0x7efe7c0f4630 tx=0x7efe74009cf0).ready entity=mon.2 client_cookie=9c6709966fb334fa server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:30.658 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.667 7efe79dc4700 1 -- 172.21.15.102:0/3182878993 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7efe7c1a8ed0 con 0x7efe7c10bbf0 2024-04-17T22:31:30.658 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.667 7efe79dc4700 1 -- 172.21.15.102:0/3182878993 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe7400b840 con 0x7efe7c10bbf0 2024-04-17T22:31:30.658 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.667 7efe79dc4700 1 -- 172.21.15.102:0/3182878993 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7efe7400ba20 con 0x7efe7c10bbf0 2024-04-17T22:31:30.658 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.667 7efe8208f700 1 -- 172.21.15.102:0/3182878993 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7efe7c10bbf0 msgr2=0x7efe7c1a7890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:30.658 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.667 7efe8208f700 1 --2- 172.21.15.102:0/3182878993 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7efe7c10bbf0 0x7efe7c1a7890 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 rx=0x7efe7c0f4630 tx=0x7efe74009cf0).stop 2024-04-17T22:31:30.658 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.668 7efe8208f700 1 -- 172.21.15.102:0/3182878993 shutdown_connections 2024-04-17T22:31:30.659 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.668 7efe8208f700 1 --2- 172.21.15.102:0/3182878993 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7efe7c10bbf0 0x7efe7c1a7890 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:30.659 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.668 7efe8208f700 1 --2- 172.21.15.102:0/3182878993 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7efe7c10b7a0 0x7efe7c1a5250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:30.659 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.668 7efe8208f700 1 -- 172.21.15.102:0/3182878993 >> 172.21.15.102:0/3182878993 conn(0x7efe7c113830 msgr2=0x7efe7c0f4ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:30.659 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.668 7efe8208f700 1 -- 172.21.15.102:0/3182878993 shutdown_connections 2024-04-17T22:31:30.659 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.668 7efe8208f700 1 -- 172.21.15.102:0/3182878993 wait complete. 2024-04-17T22:31:30.679 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.691 7efe8208f700 1 Processor -- start 2024-04-17T22:31:30.679 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.691 7efe8208f700 1 -- start start 2024-04-17T22:31:30.679 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.691 7efe8208f700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7efe7c1b1150 0x7efe7c1b15a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:30.679 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.691 7efe8208f700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7efe7c1b1bd0 0x7efe7c1b4460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:30.679 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.691 7efe8208f700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efe7c1b4a90 0x7efe7c1b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:30.679 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.691 7efe8208f700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe7c0d2030 con 0x7efe7c1b4a90 2024-04-17T22:31:30.679 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.691 7efe8208f700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe7c0d66c0 con 0x7efe7c1b1150 2024-04-17T22:31:30.679 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.691 7efe8208f700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7efe7c1bbc00 con 0x7efe7c1b1bd0 2024-04-17T22:31:30.679 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.691 7efe7adc6700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7efe7c1b1150 0x7efe7c1b15a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:30.680 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.691 7efe7a5c5700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7efe7c1b1bd0 0x7efe7c1b4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:30.680 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.691 7efe7b5c7700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efe7c1b4a90 0x7efe7c1b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:30.680 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.691 7efe7adc6700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7efe7c1b1150 0x7efe7c1b15a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:32820/0 (socket says 172.21.15.102:32820) 2024-04-17T22:31:30.680 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.691 7efe7adc6700 1 -- 172.21.15.102:0/1555488237 learned_addr learned my addr 172.21.15.102:0/1555488237 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:31:30.680 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.691 7efe7a5c5700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7efe7c1b1bd0 0x7efe7c1b4460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:33786/0 (socket says 172.21.15.102:33786) 2024-04-17T22:31:30.680 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.691 7efe7b5c7700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efe7c1b4a90 0x7efe7c1b4ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:52120/0 (socket says 172.21.15.102:52120) 2024-04-17T22:31:30.680 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.691 7efe7a5c5700 1 -- 172.21.15.102:0/1555488237 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7efe7c1b1150 msgr2=0x7efe7c1b15a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:30.680 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.691 7efe7a5c5700 1 --2- 172.21.15.102:0/1555488237 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7efe7c1b1150 0x7efe7c1b15a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:30.680 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.691 7efe7a5c5700 1 -- 172.21.15.102:0/1555488237 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efe7c1b4a90 msgr2=0x7efe7c1b4ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:30.680 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.691 7efe7a5c5700 1 --2- 172.21.15.102:0/1555488237 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efe7c1b4a90 0x7efe7c1b4ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:30.680 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.691 7efe7a5c5700 1 -- 172.21.15.102:0/1555488237 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe7c1bbdb0 con 0x7efe7c1b1bd0 2024-04-17T22:31:30.680 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.691 7efe7adc6700 1 --2- 172.21.15.102:0/1555488237 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7efe7c1b1150 0x7efe7c1b15a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:30.680 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.691 7efe7b5c7700 1 --2- 172.21.15.102:0/1555488237 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efe7c1b4a90 0x7efe7c1b4ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:30.680 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.691 7efe7a5c5700 1 --2- 172.21.15.102:0/1555488237 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7efe7c1b1bd0 0x7efe7c1b4460 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 rx=0x7efe7c0e0dc0 tx=0x7efe6c00ec50).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:30.681 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.692 7efe6bfff700 1 -- 172.21.15.102:0/1555488237 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7efe7c1bbdb0 con 0x7efe7c1b1bd0 2024-04-17T22:31:30.681 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.692 7efe6bfff700 1 -- 172.21.15.102:0/1555488237 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe6c00c810 con 0x7efe7c1b1bd0 2024-04-17T22:31:30.681 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.692 7efe8208f700 1 -- 172.21.15.102:0/1555488237 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe7c1bbfb0 con 0x7efe7c1b1bd0 2024-04-17T22:31:30.681 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.692 7efe6bfff700 1 -- 172.21.15.102:0/1555488237 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7efe6c00f040 con 0x7efe7c1b1bd0 2024-04-17T22:31:30.681 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.692 7efe8208f700 1 -- 172.21.15.102:0/1555488237 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe7c1bc390 con 0x7efe7c1b1bd0 2024-04-17T22:31:30.681 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.692 7efe6bfff700 1 -- 172.21.15.102:0/1555488237 <== mon.2 v2:172.21.15.102:3302/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7efe6c011070 con 0x7efe7c1b1bd0 2024-04-17T22:31:30.681 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.692 7efe6bfff700 1 -- 172.21.15.102:0/1555488237 <== mon.2 v2:172.21.15.102:3302/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 3194+0+0 (secure 0 0 0) 0x7efe6c010040 con 0x7efe7c1b1bd0 2024-04-17T22:31:30.681 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.693 7efe8208f700 1 -- 172.21.15.102:0/1555488237 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe7c0d5ea0 con 0x7efe7c1b1bd0 2024-04-17T22:31:30.682 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.696 7efe6bfff700 1 -- 172.21.15.102:0/1555488237 <== mon.2 v2:172.21.15.102:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+110455 (secure 0 0 0) 0x7efe6c00ed90 con 0x7efe7c1b1bd0 2024-04-17T22:31:30.786 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.800 7efe8208f700 1 -- 172.21.15.102:0/1555488237 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efe7c0de370 con 0x7efe7c1b1bd0 2024-04-17T22:31:30.787 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.801 7efe6bfff700 1 -- 172.21.15.102:0/1555488237 <== mon.2 v2:172.21.15.102:3302/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+8764 (secure 0 0 0) 0x7efe6c000c30 con 0x7efe7c1b1bd0 2024-04-17T22:31:30.787 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-17T22:31:30.787 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":25,"fsid":"cb63e789-091d-49b1-b120-55f9db526871","created":"2024-04-17 22:28:48.209635","modified":"2024-04-17 22:31:29.205644","last_up_change":"2024-04-17 22:28:54.236215","last_in_change":"2024-04-17 22:28:54.236215","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":4,"require_min_compat_client":"nautilus","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-04-17 22:28:57.478545","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-17 22:31:26.214102","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-17 22:31:27.183113","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3351ef85-ade4-4b90-a8f4-213f4e00ba01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6812","nonce":42955}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6813","nonce":42955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6815","nonce":42955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6814","nonce":42955}]},"public_addr":"172.21.15.102:6812/42955","cluster_addr":"172.21.15.102:6813/42955","heartbeat_back_addr":"172.21.15.102:6815/42955","heartbeat_front_addr":"172.21.15.102:6814/42955","state":["exists","up"]},{"osd":1,"uuid":"04f67eac-f36b-48f5-982d-4cfc776596d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6808","nonce":42958}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6809","nonce":42958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6811","nonce":42958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6810","nonce":42958}]},"public_addr":"172.21.15.102:6808/42958","cluster_addr":"172.21.15.102:6809/42958","heartbeat_back_addr":"172.21.15.102:6811/42958","heartbeat_front_addr":"172.21.15.102:6810/42958","state":["exists","up"]},{"osd":2,"uuid":"d524204b-70b2-40a8-8f73-f9e7d44318af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6804","nonce":42954}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6805","nonce":42954}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6807","nonce":42954}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6806","nonce":42954}]},"public_addr":"172.21.15.102:6804/42954","cluster_addr":"172.21.15.102:6805/42954","heartbeat_back_addr":"172.21.15.102:6807/42954","heartbeat_front_addr":"172.21.15.102:6806/42954","state":["exists","up"]},{"osd":3,"uuid":"4fc9d20c-1ad0-484f-aafa-2c9a7574245b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6800","nonce":42953}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6801","nonce":42953}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6803","nonce":42953}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6802","nonce":42953}]},"public_addr":"172.21.15.102:6800/42953","cluster_addr":"172.21.15.102:6801/42953","heartbeat_back_addr":"172.21.15.102:6803/42953","heartbeat_front_addr":"172.21.15.102:6802/42953","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2024-04-17T22:31:30.789 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.802 7efe8208f700 1 -- 172.21.15.102:0/1555488237 >> v1:172.21.15.102:6816/42360 conn(0x7efe580145a0 legacy=0x7efe58016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:30.789 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.802 7efe8208f700 1 -- 172.21.15.102:0/1555488237 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7efe7c1b1bd0 msgr2=0x7efe7c1b4460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:30.789 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.802 7efe8208f700 1 --2- 172.21.15.102:0/1555488237 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7efe7c1b1bd0 0x7efe7c1b4460 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 rx=0x7efe7c0e0dc0 tx=0x7efe6c00ec50).stop 2024-04-17T22:31:30.789 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.802 7efe8208f700 1 -- 172.21.15.102:0/1555488237 shutdown_connections 2024-04-17T22:31:30.789 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.802 7efe8208f700 1 --2- 172.21.15.102:0/1555488237 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efe7c1b4a90 0x7efe7c1b4ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:30.789 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.802 7efe8208f700 1 --2- 172.21.15.102:0/1555488237 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7efe7c1b1150 0x7efe7c1b15a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:30.789 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.802 7efe8208f700 1 --2- 172.21.15.102:0/1555488237 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7efe7c1b1bd0 0x7efe7c1b4460 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:30.789 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.802 7efe8208f700 1 -- 172.21.15.102:0/1555488237 >> 172.21.15.102:0/1555488237 conn(0x7efe7c0c6a60 msgr2=0x7efe7c0bf5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:30.789 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.803 7efe8208f700 1 -- 172.21.15.102:0/1555488237 shutdown_connections 2024-04-17T22:31:30.790 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.803 7efe8208f700 1 -- 172.21.15.102:0/1555488237 wait complete. 2024-04-17T22:31:30.799 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-17T22:31:30.800 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-17T22:31:30.927 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.938 7fb12e5dd700 1 Processor -- start 2024-04-17T22:31:30.927 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.938 7fb12e5dd700 1 -- start start 2024-04-17T22:31:30.927 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.938 7fb12e5dd700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb1280f5770 0x7fb12810c110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:30.927 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.938 7fb12e5dd700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb12810c740 0x7fb1281a0c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:30.927 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.938 7fb12e5dd700 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1281a1a30 con 0x7fb12810cb90 2024-04-17T22:31:30.927 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.938 7fb12e5dd700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1280f93e0 con 0x7fb12810c740 2024-04-17T22:31:30.927 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.938 7fb12e5dd700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb1281a1c50 con 0x7fb1280f5770 2024-04-17T22:31:30.927 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.938 7fb1273b0700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb1280f5770 0x7fb12810c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:30.928 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.938 7fb11ffff700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb12810c740 0x7fb1281a0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:30.928 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.938 7fb127bb1700 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7fb12810cb90 0x7fb1281a1230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:33812/0 (socket says 172.21.15.102:33812) 2024-04-17T22:31:30.928 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.938 7fb127bb1700 1 -- 172.21.15.102:0/565793436 learned_addr learned my addr 172.21.15.102:0/565793436 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-17T22:31:30.928 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.939 7fb1273b0700 1 -- 172.21.15.102:0/565793436 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb12810c740 msgr2=0x7fb1281a0c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:31:30.928 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.939 7fb1273b0700 1 --2- 172.21.15.102:0/565793436 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb12810c740 0x7fb1281a0c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:30.928 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.939 7fb126baf700 1 -- 172.21.15.102:0/565793436 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 562131980 0 0) 0x7fb120001490 con 0x7fb12810cb90 2024-04-17T22:31:30.928 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.939 7fb1273b0700 1 -- 172.21.15.102:0/565793436 >> v1:172.21.15.102:6789/0 conn(0x7fb12810cb90 legacy=0x7fb1281a1230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:30.928 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.939 7fb1273b0700 1 -- 172.21.15.102:0/565793436 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1281aa330 con 0x7fb1280f5770 2024-04-17T22:31:30.928 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.939 7fb11ffff700 1 --2- 172.21.15.102:0/565793436 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb12810c740 0x7fb1281a0c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:30.928 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.939 7fb1273b0700 1 --2- 172.21.15.102:0/565793436 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb1280f5770 0x7fb12810c110 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 rx=0x7fb118002f70 tx=0x7fb11800b9a0).ready entity=mon.2 client_cookie=79843c3fcb0fef64 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:30.928 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.939 7fb126baf700 1 -- 172.21.15.102:0/565793436 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb1281aa330 con 0x7fb1280f5770 2024-04-17T22:31:30.928 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.939 7fb126baf700 1 -- 172.21.15.102:0/565793436 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb11800d570 con 0x7fb1280f5770 2024-04-17T22:31:30.929 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.939 7fb126baf700 1 -- 172.21.15.102:0/565793436 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb11800d940 con 0x7fb1280f5770 2024-04-17T22:31:30.929 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.939 7fb12e5dd700 1 -- 172.21.15.102:0/565793436 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb1280f5770 msgr2=0x7fb12810c110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:30.929 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.939 7fb12e5dd700 1 --2- 172.21.15.102:0/565793436 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb1280f5770 0x7fb12810c110 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 rx=0x7fb118002f70 tx=0x7fb11800b9a0).stop 2024-04-17T22:31:30.929 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.940 7fb12e5dd700 1 -- 172.21.15.102:0/565793436 shutdown_connections 2024-04-17T22:31:30.929 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.940 7fb12e5dd700 1 --2- 172.21.15.102:0/565793436 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb1280f5770 0x7fb12810c110 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:30.929 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.940 7fb12e5dd700 1 --2- 172.21.15.102:0/565793436 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb12810c740 0x7fb1281a0c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:30.929 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.940 7fb12e5dd700 1 -- 172.21.15.102:0/565793436 >> 172.21.15.102:0/565793436 conn(0x7fb1281138c0 msgr2=0x7fb1280f4f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:30.929 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.940 7fb12e5dd700 1 -- 172.21.15.102:0/565793436 shutdown_connections 2024-04-17T22:31:30.929 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.940 7fb12e5dd700 1 -- 172.21.15.102:0/565793436 wait complete. 2024-04-17T22:31:30.950 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.962 7fb12e5dd700 1 Processor -- start 2024-04-17T22:31:30.951 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.962 7fb12e5dd700 1 -- start start 2024-04-17T22:31:30.951 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.963 7fb12e5dd700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb1280cdf10 0x7fb1280d6010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:30.951 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.963 7fb12e5dd700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb1280d6640 0x7fb1281b1dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:30.951 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.963 7fb12e5dd700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb1280d6a90 0x7fb1281b23f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:30.951 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.963 7fb12e5dd700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1280cd670 con 0x7fb1280d6640 2024-04-17T22:31:30.951 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.963 7fb12e5dd700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1280c4560 con 0x7fb1280d6a90 2024-04-17T22:31:30.951 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.963 7fb12e5dd700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb1281b2a20 con 0x7fb1280cdf10 2024-04-17T22:31:30.951 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.963 7fb1273b0700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb1280cdf10 0x7fb1280d6010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:30.951 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.963 7fb127bb1700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb1280d6a90 0x7fb1281b23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:30.951 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.963 7fb1273b0700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb1280cdf10 0x7fb1280d6010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:33802/0 (socket says 172.21.15.102:33802) 2024-04-17T22:31:30.951 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.963 7fb127bb1700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb1280d6a90 0x7fb1281b23f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:32830/0 (socket says 172.21.15.102:32830) 2024-04-17T22:31:30.951 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.963 7fb1273b0700 1 -- 172.21.15.102:0/2513704952 learned_addr learned my addr 172.21.15.102:0/2513704952 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:31:30.952 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.963 7fb11ffff700 1 --2- 172.21.15.102:0/2513704952 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb1280d6640 0x7fb1281b1dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:30.952 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.963 7fb1273b0700 1 -- 172.21.15.102:0/2513704952 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb1280d6a90 msgr2=0x7fb1281b23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:30.952 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.963 7fb1273b0700 1 --2- 172.21.15.102:0/2513704952 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb1280d6a90 0x7fb1281b23f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:30.952 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.963 7fb1273b0700 1 -- 172.21.15.102:0/2513704952 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb1280d6640 msgr2=0x7fb1281b1dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:30.952 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.963 7fb1273b0700 1 --2- 172.21.15.102:0/2513704952 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb1280d6640 0x7fb1281b1dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:30.952 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.963 7fb1273b0700 1 -- 172.21.15.102:0/2513704952 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1281b2c50 con 0x7fb1280cdf10 2024-04-17T22:31:30.952 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.963 7fb1273b0700 1 --2- 172.21.15.102:0/2513704952 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb1280cdf10 0x7fb1280d6010 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 rx=0x7fb1280d74c0 tx=0x7fb1180068c0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:30.952 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.963 7fb1253ac700 1 -- 172.21.15.102:0/2513704952 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb1281b2c50 con 0x7fb1280cdf10 2024-04-17T22:31:30.952 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.963 7fb1253ac700 1 -- 172.21.15.102:0/2513704952 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1180119b0 con 0x7fb1280cdf10 2024-04-17T22:31:30.952 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.963 7fb12e5dd700 1 -- 172.21.15.102:0/2513704952 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1281b9530 con 0x7fb1280cdf10 2024-04-17T22:31:30.953 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.963 7fb1253ac700 1 -- 172.21.15.102:0/2513704952 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb118022020 con 0x7fb1280cdf10 2024-04-17T22:31:30.953 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.963 7fb12e5dd700 1 -- 172.21.15.102:0/2513704952 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1281b9910 con 0x7fb1280cdf10 2024-04-17T22:31:30.953 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.964 7fb1253ac700 1 -- 172.21.15.102:0/2513704952 <== mon.2 v2:172.21.15.102:3302/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7fb11801d450 con 0x7fb1280cdf10 2024-04-17T22:31:30.953 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.964 7fb1253ac700 1 -- 172.21.15.102:0/2513704952 <== mon.2 v2:172.21.15.102:3302/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 3194+0+0 (secure 0 0 0) 0x7fb11800c9f0 con 0x7fb1280cdf10 2024-04-17T22:31:30.953 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.964 7fb12e5dd700 1 -- 172.21.15.102:0/2513704952 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1280f0820 con 0x7fb1280cdf10 2024-04-17T22:31:30.953 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:30.968 7fb1253ac700 1 -- 172.21.15.102:0/2513704952 <== mon.2 v2:172.21.15.102:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+110455 (secure 0 0 0) 0x7fb1180117a0 con 0x7fb1280cdf10 2024-04-17T22:31:31.055 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.069 7fb12e5dd700 1 -- 172.21.15.102:0/2513704952 --> v1:172.21.15.102:6816/42360 -- command(tid 0: {"prefix": "pg dump", "target": ["mgr", ""], "format": "json"}) v1 -- 0x7fb1280cd940 con 0x7fb10c0145a0 2024-04-17T22:31:31.062 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.076 7fb1253ac700 1 -- 172.21.15.102:0/2513704952 <== mgr.4104 v1:172.21.15.102:6816/42360 1 ==== command_reply(tid 0: 0 dumped all) v1 ==== 18+0+304823 (unknown 2965378022 0 1163813529) 0x7fb11000b740 con 0x7fb10c0145a0 2024-04-17T22:31:31.067 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-17T22:31:31.070 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.079 7fb12e5dd700 1 -- 172.21.15.102:0/2513704952 >> v1:172.21.15.102:6816/42360 conn(0x7fb10c0145a0 legacy=0x7fb10c016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:31.070 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.079 7fb12e5dd700 1 -- 172.21.15.102:0/2513704952 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb1280cdf10 msgr2=0x7fb1280d6010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:31.070 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.079 7fb12e5dd700 1 --2- 172.21.15.102:0/2513704952 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb1280cdf10 0x7fb1280d6010 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 rx=0x7fb1280d74c0 tx=0x7fb1180068c0).stop 2024-04-17T22:31:31.070 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.079 7fb12e5dd700 1 -- 172.21.15.102:0/2513704952 shutdown_connections 2024-04-17T22:31:31.070 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.079 7fb12e5dd700 1 --2- 172.21.15.102:0/2513704952 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb1280d6a90 0x7fb1281b23f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:31.070 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.079 7fb12e5dd700 1 --2- 172.21.15.102:0/2513704952 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb1280cdf10 0x7fb1280d6010 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:31.071 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.079 7fb12e5dd700 1 --2- 172.21.15.102:0/2513704952 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb1280d6640 0x7fb1281b1dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:31.071 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.079 7fb12e5dd700 1 -- 172.21.15.102:0/2513704952 >> 172.21.15.102:0/2513704952 conn(0x7fb1280c3930 msgr2=0x7fb1280dae90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:31.071 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.079 7fb12e5dd700 1 -- 172.21.15.102:0/2513704952 shutdown_connections 2024-04-17T22:31:31.071 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.079 7fb12e5dd700 1 -- 172.21.15.102:0/2513704952 wait complete. 2024-04-17T22:31:31.071 INFO:teuthology.orchestra.run.smithi102.stderr:dumped all 2024-04-17T22:31:31.076 INFO:teuthology.orchestra.run.smithi102.stdout:{"pg_ready":true,"pg_map":{"version":99,"stamp":"2024-04-17 22:31:29.219417","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":4,"num_per_pool_osds":4,"kb":377487360,"kb_used":4195080,"kb_used_data":736,"kb_used_omap":0,"kb_used_meta":4194304,"kb_avail":373292280,"statfs":{"total":386547056640,"available":382251294720,"internally_reserved":4294967296,"allocated":753664,"data_stored":220700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4294967296},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-4007,"num_objects":-22,"num_object_clones":0,"num_object_copies":-44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-51,"num_write_kb":-19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.398188"},"pg_stats":[{"pgid":"5.3e","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:31:27.861194","last_change":"2024-04-17 22:31:27.861194","last_active":"2024-04-17 22:31:27.861194","last_peered":"2024-04-17 22:31:27.861194","last_clean":"2024-04-17 22:31:27.861194","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:31:27.861194","last_undegraded":"2024-04-17 22:31:27.861194","last_fullsized":"2024-04-17 22:31:27.861194","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:27.861194","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:27.861194","last_clean_scrub_stamp":"2024-04-17 22:31:27.861194","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3f","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:31:26.859642","last_change":"2024-04-17 22:31:26.859642","last_active":"2024-04-17 22:31:26.859642","last_peered":"2024-04-17 22:31:26.859642","last_clean":"2024-04-17 22:31:26.859642","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:31:26.859642","last_undegraded":"2024-04-17 22:31:26.859642","last_fullsized":"2024-04-17 22:31:26.859642","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:26.859642","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:26.859642","last_clean_scrub_stamp":"2024-04-17 22:31:26.859642","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3f","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:31:27.861194","last_change":"2024-04-17 22:31:27.861194","last_active":"2024-04-17 22:31:27.861194","last_peered":"2024-04-17 22:31:27.861194","last_clean":"2024-04-17 22:31:27.861194","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:31:27.861194","last_undegraded":"2024-04-17 22:31:27.861194","last_fullsized":"2024-04-17 22:31:27.861194","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:27.861194","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:27.861194","last_clean_scrub_stamp":"2024-04-17 22:31:27.861194","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:31:26.859642","last_change":"2024-04-17 22:31:26.859642","last_active":"2024-04-17 22:31:26.859642","last_peered":"2024-04-17 22:31:26.859642","last_clean":"2024-04-17 22:31:26.859642","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:31:26.859642","last_undegraded":"2024-04-17 22:31:26.859642","last_fullsized":"2024-04-17 22:31:26.859642","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:26.859642","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:26.859642","last_clean_scrub_stamp":"2024-04-17 22:31:26.859642","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:31:27.861194","last_change":"2024-04-17 22:31:27.861194","last_active":"2024-04-17 22:31:27.861194","last_peered":"2024-04-17 22:31:27.861194","last_clean":"2024-04-17 22:31:27.861194","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:31:27.861194","last_undegraded":"2024-04-17 22:31:27.861194","last_fullsized":"2024-04-17 22:31:27.861194","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:27.861194","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:27.861194","last_clean_scrub_stamp":"2024-04-17 22:31:27.861194","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:31:26.859642","last_change":"2024-04-17 22:31:26.859642","last_active":"2024-04-17 22:31:26.859642","last_peered":"2024-04-17 22:31:26.859642","last_clean":"2024-04-17 22:31:26.859642","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:31:26.859642","last_undegraded":"2024-04-17 22:31:26.859642","last_fullsized":"2024-04-17 22:31:26.859642","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:26.859642","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:26.859642","last_clean_scrub_stamp":"2024-04-17 22:31:26.859642","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:31:27.861194","last_change":"2024-04-17 22:31:27.861194","last_active":"2024-04-17 22:31:27.861194","last_peered":"2024-04-17 22:31:27.861194","last_clean":"2024-04-17 22:31:27.861194","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:31:27.861194","last_undegraded":"2024-04-17 22:31:27.861194","last_fullsized":"2024-04-17 22:31:27.861194","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:27.861194","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:27.861194","last_clean_scrub_stamp":"2024-04-17 22:31:27.861194","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:31:26.859642","last_change":"2024-04-17 22:31:26.859642","last_active":"2024-04-17 22:31:26.859642","last_peered":"2024-04-17 22:31:26.859642","last_clean":"2024-04-17 22:31:26.859642","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:31:26.859642","last_undegraded":"2024-04-17 22:31:26.859642","last_fullsized":"2024-04-17 22:31:26.859642","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:26.859642","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:26.859642","last_clean_scrub_stamp":"2024-04-17 22:31:26.859642","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:31:27.861194","last_change":"2024-04-17 22:31:27.861194","last_active":"2024-04-17 22:31:27.861194","last_peered":"2024-04-17 22:31:27.861194","last_clean":"2024-04-17 22:31:27.861194","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:31:27.861194","last_undegraded":"2024-04-17 22:31:27.861194","last_fullsized":"2024-04-17 22:31:27.861194","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:27.861194","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:27.861194","last_clean_scrub_stamp":"2024-04-17 22:31:27.861194","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:31:26.859642","last_change":"2024-04-17 22:31:26.859642","last_active":"2024-04-17 22:31:26.859642","last_peered":"2024-04-17 22:31:26.859642","last_clean":"2024-04-17 22:31:26.859642","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:31:26.859642","last_undegraded":"2024-04-17 22:31:26.859642","last_fullsized":"2024-04-17 22:31:26.859642","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:26.859642","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:26.859642","last_clean_scrub_stamp":"2024-04-17 22:31:26.859642","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:31:27.861194","last_change":"2024-04-17 22:31:27.861194","last_active":"2024-04-17 22:31:27.861194","last_peered":"2024-04-17 22:31:27.861194","last_clean":"2024-04-17 22:31:27.861194","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:31:27.861194","last_undegraded":"2024-04-17 22:31:27.861194","last_fullsized":"2024-04-17 22:31:27.861194","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:27.861194","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:27.861194","last_clean_scrub_stamp":"2024-04-17 22:31:27.861194","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:31:26.859642","last_change":"2024-04-17 22:31:26.859642","last_active":"2024-04-17 22:31:26.859642","last_peered":"2024-04-17 22:31:26.859642","last_clean":"2024-04-17 22:31:26.859642","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:31:26.859642","last_undegraded":"2024-04-17 22:31:26.859642","last_fullsized":"2024-04-17 22:31:26.859642","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:26.859642","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:26.859642","last_clean_scrub_stamp":"2024-04-17 22:31:26.859642","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:31:27.861194","last_change":"2024-04-17 22:31:27.861194","last_active":"2024-04-17 22:31:27.861194","last_peered":"2024-04-17 22:31:27.861194","last_clean":"2024-04-17 22:31:27.861194","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:31:27.861194","last_undegraded":"2024-04-17 22:31:27.861194","last_fullsized":"2024-04-17 22:31:27.861194","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:27.861194","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:27.861194","last_clean_scrub_stamp":"2024-04-17 22:31:27.861194","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:31:26.859642","last_change":"2024-04-17 22:31:26.859642","last_active":"2024-04-17 22:31:26.859642","last_peered":"2024-04-17 22:31:26.859642","last_clean":"2024-04-17 22:31:26.859642","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:31:26.859642","last_undegraded":"2024-04-17 22:31:26.859642","last_fullsized":"2024-04-17 22:31:26.859642","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:26.859642","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:26.859642","last_clean_scrub_stamp":"2024-04-17 22:31:26.859642","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:31:27.861194","last_change":"2024-04-17 22:31:27.861194","last_active":"2024-04-17 22:31:27.861194","last_peered":"2024-04-17 22:31:27.861194","last_clean":"2024-04-17 22:31:27.861194","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:31:27.861194","last_undegraded":"2024-04-17 22:31:27.861194","last_fullsized":"2024-04-17 22:31:27.861194","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:27.861194","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:27.861194","last_clean_scrub_stamp":"2024-04-17 22:31:27.861194","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:31:26.859642","last_change":"2024-04-17 22:31:26.859642","last_active":"2024-04-17 22:31:26.859642","last_peered":"2024-04-17 22:31:26.859642","last_clean":"2024-04-17 22:31:26.859642","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:31:26.859642","last_undegraded":"2024-04-17 22:31:26.859642","last_fullsized":"2024-04-17 22:31:26.859642","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:26.859642","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:26.859642","last_clean_scrub_stamp":"2024-04-17 22:31:26.859642","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:31:27.861194","last_change":"2024-04-17 22:31:27.861194","last_active":"2024-04-17 22:31:27.861194","last_peered":"2024-04-17 22:31:27.861194","last_clean":"2024-04-17 22:31:27.861194","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:31:27.861194","last_undegraded":"2024-04-17 22:31:27.861194","last_fullsized":"2024-04-17 22:31:27.861194","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:27.861194","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:27.861194","last_clean_scrub_stamp":"2024-04-17 22:31:27.861194","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:31:26.859642","last_change":"2024-04-17 22:31:26.859642","last_active":"2024-04-17 22:31:26.859642","last_peered":"2024-04-17 22:31:26.859642","last_clean":"2024-04-17 22:31:26.859642","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:31:26.859642","last_undegraded":"2024-04-17 22:31:26.859642","last_fullsized":"2024-04-17 22:31:26.859642","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:26.859642","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:26.859642","last_clean_scrub_stamp":"2024-04-17 22:31:26.859642","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:31:27.861194","last_change":"2024-04-17 22:31:27.861194","last_active":"2024-04-17 22:31:27.861194","last_peered":"2024-04-17 22:31:27.861194","last_clean":"2024-04-17 22:31:27.861194","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:31:27.861194","last_undegraded":"2024-04-17 22:31:27.861194","last_fullsized":"2024-04-17 22:31:27.861194","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:27.861194","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:27.861194","last_clean_scrub_stamp":"2024-04-17 22:31:27.861194","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:31:26.859642","last_change":"2024-04-17 22:31:26.859642","last_active":"2024-04-17 22:31:26.859642","last_peered":"2024-04-17 22:31:26.859642","last_clean":"2024-04-17 22:31:26.859642","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:31:26.859642","last_undegraded":"2024-04-17 22:31:26.859642","last_fullsized":"2024-04-17 22:31:26.859642","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:26.859642","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:26.859642","last_clean_scrub_stamp":"2024-04-17 22:31:26.859642","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:31:27.861194","last_change":"2024-04-17 22:31:27.861194","last_active":"2024-04-17 22:31:27.861194","last_peered":"2024-04-17 22:31:27.861194","last_clean":"2024-04-17 22:31:27.861194","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:31:27.861194","last_undegraded":"2024-04-17 22:31:27.861194","last_fullsized":"2024-04-17 22:31:27.861194","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:27.861194","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:27.861194","last_clean_scrub_stamp":"2024-04-17 22:31:27.861194","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:31:26.859642","last_change":"2024-04-17 22:31:26.859642","last_active":"2024-04-17 22:31:26.859642","last_peered":"2024-04-17 22:31:26.859642","last_clean":"2024-04-17 22:31:26.859642","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:31:26.859642","last_undegraded":"2024-04-17 22:31:26.859642","last_fullsized":"2024-04-17 22:31:26.859642","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:26.859642","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:26.859642","last_clean_scrub_stamp":"2024-04-17 22:31:26.859642","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:31:27.861194","last_change":"2024-04-17 22:31:27.861194","last_active":"2024-04-17 22:31:27.861194","last_peered":"2024-04-17 22:31:27.861194","last_clean":"2024-04-17 22:31:27.861194","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:31:27.861194","last_undegraded":"2024-04-17 22:31:27.861194","last_fullsized":"2024-04-17 22:31:27.861194","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:27.861194","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:27.861194","last_clean_scrub_stamp":"2024-04-17 22:31:27.861194","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:31:26.859642","last_change":"2024-04-17 22:31:26.859642","last_active":"2024-04-17 22:31:26.859642","last_peered":"2024-04-17 22:31:26.859642","last_clean":"2024-04-17 22:31:26.859642","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:31:26.859642","last_undegraded":"2024-04-17 22:31:26.859642","last_fullsized":"2024-04-17 22:31:26.859642","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:26.859642","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:26.859642","last_clean_scrub_stamp":"2024-04-17 22:31:26.859642","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:31:27.861194","last_change":"2024-04-17 22:31:27.861194","last_active":"2024-04-17 22:31:27.861194","last_peered":"2024-04-17 22:31:27.861194","last_clean":"2024-04-17 22:31:27.861194","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:31:27.861194","last_undegraded":"2024-04-17 22:31:27.861194","last_fullsized":"2024-04-17 22:31:27.861194","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:27.861194","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:27.861194","last_clean_scrub_stamp":"2024-04-17 22:31:27.861194","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:31:26.859642","last_change":"2024-04-17 22:31:26.859642","last_active":"2024-04-17 22:31:26.859642","last_peered":"2024-04-17 22:31:26.859642","last_clean":"2024-04-17 22:31:26.859642","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:31:26.859642","last_undegraded":"2024-04-17 22:31:26.859642","last_fullsized":"2024-04-17 22:31:26.859642","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:26.859642","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:26.859642","last_clean_scrub_stamp":"2024-04-17 22:31:26.859642","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:31:27.861194","last_change":"2024-04-17 22:31:27.861194","last_active":"2024-04-17 22:31:27.861194","last_peered":"2024-04-17 22:31:27.861194","last_clean":"2024-04-17 22:31:27.861194","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:31:27.861194","last_undegraded":"2024-04-17 22:31:27.861194","last_fullsized":"2024-04-17 22:31:27.861194","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:27.861194","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:27.861194","last_clean_scrub_stamp":"2024-04-17 22:31:27.861194","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:31:26.859642","last_change":"2024-04-17 22:31:26.859642","last_active":"2024-04-17 22:31:26.859642","last_peered":"2024-04-17 22:31:26.859642","last_clean":"2024-04-17 22:31:26.859642","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:31:26.859642","last_undegraded":"2024-04-17 22:31:26.859642","last_fullsized":"2024-04-17 22:31:26.859642","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:26.859642","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:26.859642","last_clean_scrub_stamp":"2024-04-17 22:31:26.859642","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:31:27.861194","last_change":"2024-04-17 22:31:27.861194","last_active":"2024-04-17 22:31:27.861194","last_peered":"2024-04-17 22:31:27.861194","last_clean":"2024-04-17 22:31:27.861194","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:31:27.861194","last_undegraded":"2024-04-17 22:31:27.861194","last_fullsized":"2024-04-17 22:31:27.861194","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:27.861194","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:27.861194","last_clean_scrub_stamp":"2024-04-17 22:31:27.861194","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:31:26.859642","last_change":"2024-04-17 22:31:26.859642","last_active":"2024-04-17 22:31:26.859642","last_peered":"2024-04-17 22:31:26.859642","last_clean":"2024-04-17 22:31:26.859642","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:31:26.859642","last_undegraded":"2024-04-17 22:31:26.859642","last_fullsized":"2024-04-17 22:31:26.859642","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:26.859642","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:26.859642","last_clean_scrub_stamp":"2024-04-17 22:31:26.859642","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:31:27.861194","last_change":"2024-04-17 22:31:27.861194","last_active":"2024-04-17 22:31:27.861194","last_peered":"2024-04-17 22:31:27.861194","last_clean":"2024-04-17 22:31:27.861194","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:31:27.861194","last_undegraded":"2024-04-17 22:31:27.861194","last_fullsized":"2024-04-17 22:31:27.861194","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:27.861194","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:27.861194","last_clean_scrub_stamp":"2024-04-17 22:31:27.861194","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:31:26.859642","last_change":"2024-04-17 22:31:26.859642","last_active":"2024-04-17 22:31:26.859642","last_peered":"2024-04-17 22:31:26.859642","last_clean":"2024-04-17 22:31:26.859642","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:31:26.859642","last_undegraded":"2024-04-17 22:31:26.859642","last_fullsized":"2024-04-17 22:31:26.859642","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:26.859642","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:26.859642","last_clean_scrub_stamp":"2024-04-17 22:31:26.859642","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:31:27.861194","last_change":"2024-04-17 22:31:27.861194","last_active":"2024-04-17 22:31:27.861194","last_peered":"2024-04-17 22:31:27.861194","last_clean":"2024-04-17 22:31:27.861194","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:31:27.861194","last_undegraded":"2024-04-17 22:31:27.861194","last_fullsized":"2024-04-17 22:31:27.861194","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:27.861194","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:27.861194","last_clean_scrub_stamp":"2024-04-17 22:31:27.861194","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:31:26.859642","last_change":"2024-04-17 22:31:26.859642","last_active":"2024-04-17 22:31:26.859642","last_peered":"2024-04-17 22:31:26.859642","last_clean":"2024-04-17 22:31:26.859642","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:31:26.859642","last_undegraded":"2024-04-17 22:31:26.859642","last_fullsized":"2024-04-17 22:31:26.859642","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:26.859642","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:26.859642","last_clean_scrub_stamp":"2024-04-17 22:31:26.859642","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:31:27.861194","last_change":"2024-04-17 22:31:27.861194","last_active":"2024-04-17 22:31:27.861194","last_peered":"2024-04-17 22:31:27.861194","last_clean":"2024-04-17 22:31:27.861194","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:31:27.861194","last_undegraded":"2024-04-17 22:31:27.861194","last_fullsized":"2024-04-17 22:31:27.861194","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:27.861194","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:27.861194","last_clean_scrub_stamp":"2024-04-17 22:31:27.861194","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:31:26.859642","last_change":"2024-04-17 22:31:26.859642","last_active":"2024-04-17 22:31:26.859642","last_peered":"2024-04-17 22:31:26.859642","last_clean":"2024-04-17 22:31:26.859642","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:31:26.859642","last_undegraded":"2024-04-17 22:31:26.859642","last_fullsized":"2024-04-17 22:31:26.859642","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:26.859642","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:26.859642","last_clean_scrub_stamp":"2024-04-17 22:31:26.859642","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:31:27.861194","last_change":"2024-04-17 22:31:27.861194","last_active":"2024-04-17 22:31:27.861194","last_peered":"2024-04-17 22:31:27.861194","last_clean":"2024-04-17 22:31:27.861194","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:31:27.861194","last_undegraded":"2024-04-17 22:31:27.861194","last_fullsized":"2024-04-17 22:31:27.861194","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:27.861194","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:27.861194","last_clean_scrub_stamp":"2024-04-17 22:31:27.861194","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:31:26.859642","last_change":"2024-04-17 22:31:26.859642","last_active":"2024-04-17 22:31:26.859642","last_peered":"2024-04-17 22:31:26.859642","last_clean":"2024-04-17 22:31:26.859642","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:31:26.859642","last_undegraded":"2024-04-17 22:31:26.859642","last_fullsized":"2024-04-17 22:31:26.859642","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:26.859642","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:26.859642","last_clean_scrub_stamp":"2024-04-17 22:31:26.859642","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:31:27.861194","last_change":"2024-04-17 22:31:27.861194","last_active":"2024-04-17 22:31:27.861194","last_peered":"2024-04-17 22:31:27.861194","last_clean":"2024-04-17 22:31:27.861194","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:31:27.861194","last_undegraded":"2024-04-17 22:31:27.861194","last_fullsized":"2024-04-17 22:31:27.861194","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:27.861194","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:27.861194","last_clean_scrub_stamp":"2024-04-17 22:31:27.861194","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:31:26.859642","last_change":"2024-04-17 22:31:26.859642","last_active":"2024-04-17 22:31:26.859642","last_peered":"2024-04-17 22:31:26.859642","last_clean":"2024-04-17 22:31:26.859642","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:31:26.859642","last_undegraded":"2024-04-17 22:31:26.859642","last_fullsized":"2024-04-17 22:31:26.859642","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:26.859642","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:26.859642","last_clean_scrub_stamp":"2024-04-17 22:31:26.859642","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:31:27.861194","last_change":"2024-04-17 22:31:27.861194","last_active":"2024-04-17 22:31:27.861194","last_peered":"2024-04-17 22:31:27.861194","last_clean":"2024-04-17 22:31:27.861194","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:31:27.861194","last_undegraded":"2024-04-17 22:31:27.861194","last_fullsized":"2024-04-17 22:31:27.861194","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:27.861194","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:27.861194","last_clean_scrub_stamp":"2024-04-17 22:31:27.861194","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:31:26.859642","last_change":"2024-04-17 22:31:26.859642","last_active":"2024-04-17 22:31:26.859642","last_peered":"2024-04-17 22:31:26.859642","last_clean":"2024-04-17 22:31:26.859642","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:31:26.859642","last_undegraded":"2024-04-17 22:31:26.859642","last_fullsized":"2024-04-17 22:31:26.859642","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:26.859642","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:26.859642","last_clean_scrub_stamp":"2024-04-17 22:31:26.859642","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:31:27.861194","last_change":"2024-04-17 22:31:27.861194","last_active":"2024-04-17 22:31:27.861194","last_peered":"2024-04-17 22:31:27.861194","last_clean":"2024-04-17 22:31:27.861194","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:31:27.861194","last_undegraded":"2024-04-17 22:31:27.861194","last_fullsized":"2024-04-17 22:31:27.861194","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:27.861194","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:27.861194","last_clean_scrub_stamp":"2024-04-17 22:31:27.861194","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:31:26.859642","last_change":"2024-04-17 22:31:26.859642","last_active":"2024-04-17 22:31:26.859642","last_peered":"2024-04-17 22:31:26.859642","last_clean":"2024-04-17 22:31:26.859642","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:31:26.859642","last_undegraded":"2024-04-17 22:31:26.859642","last_fullsized":"2024-04-17 22:31:26.859642","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:26.859642","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:26.859642","last_clean_scrub_stamp":"2024-04-17 22:31:26.859642","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:31:27.861194","last_change":"2024-04-17 22:31:27.861194","last_active":"2024-04-17 22:31:27.861194","last_peered":"2024-04-17 22:31:27.861194","last_clean":"2024-04-17 22:31:27.861194","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:31:27.861194","last_undegraded":"2024-04-17 22:31:27.861194","last_fullsized":"2024-04-17 22:31:27.861194","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:27.861194","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:27.861194","last_clean_scrub_stamp":"2024-04-17 22:31:27.861194","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:31:26.859642","last_change":"2024-04-17 22:31:26.859642","last_active":"2024-04-17 22:31:26.859642","last_peered":"2024-04-17 22:31:26.859642","last_clean":"2024-04-17 22:31:26.859642","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:31:26.859642","last_undegraded":"2024-04-17 22:31:26.859642","last_fullsized":"2024-04-17 22:31:26.859642","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:26.859642","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:26.859642","last_clean_scrub_stamp":"2024-04-17 22:31:26.859642","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:31:27.861194","last_change":"2024-04-17 22:31:27.861194","last_active":"2024-04-17 22:31:27.861194","last_peered":"2024-04-17 22:31:27.861194","last_clean":"2024-04-17 22:31:27.861194","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:31:27.861194","last_undegraded":"2024-04-17 22:31:27.861194","last_fullsized":"2024-04-17 22:31:27.861194","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:27.861194","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:27.861194","last_clean_scrub_stamp":"2024-04-17 22:31:27.861194","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:31:26.859642","last_change":"2024-04-17 22:31:26.859642","last_active":"2024-04-17 22:31:26.859642","last_peered":"2024-04-17 22:31:26.859642","last_clean":"2024-04-17 22:31:26.859642","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:31:26.859642","last_undegraded":"2024-04-17 22:31:26.859642","last_fullsized":"2024-04-17 22:31:26.859642","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:26.859642","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:26.859642","last_clean_scrub_stamp":"2024-04-17 22:31:26.859642","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:31:27.861194","last_change":"2024-04-17 22:31:27.861194","last_active":"2024-04-17 22:31:27.861194","last_peered":"2024-04-17 22:31:27.861194","last_clean":"2024-04-17 22:31:27.861194","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:31:27.861194","last_undegraded":"2024-04-17 22:31:27.861194","last_fullsized":"2024-04-17 22:31:27.861194","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:27.861194","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:27.861194","last_clean_scrub_stamp":"2024-04-17 22:31:27.861194","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:31:26.859642","last_change":"2024-04-17 22:31:26.859642","last_active":"2024-04-17 22:31:26.859642","last_peered":"2024-04-17 22:31:26.859642","last_clean":"2024-04-17 22:31:26.859642","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:31:26.859642","last_undegraded":"2024-04-17 22:31:26.859642","last_fullsized":"2024-04-17 22:31:26.859642","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:26.859642","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:26.859642","last_clean_scrub_stamp":"2024-04-17 22:31:26.859642","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:31:27.861194","last_change":"2024-04-17 22:31:27.861194","last_active":"2024-04-17 22:31:27.861194","last_peered":"2024-04-17 22:31:27.861194","last_clean":"2024-04-17 22:31:27.861194","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:31:27.861194","last_undegraded":"2024-04-17 22:31:27.861194","last_fullsized":"2024-04-17 22:31:27.861194","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:27.861194","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:27.861194","last_clean_scrub_stamp":"2024-04-17 22:31:27.861194","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:31:26.859642","last_change":"2024-04-17 22:31:26.859642","last_active":"2024-04-17 22:31:26.859642","last_peered":"2024-04-17 22:31:26.859642","last_clean":"2024-04-17 22:31:26.859642","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:31:26.859642","last_undegraded":"2024-04-17 22:31:26.859642","last_fullsized":"2024-04-17 22:31:26.859642","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:26.859642","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:26.859642","last_clean_scrub_stamp":"2024-04-17 22:31:26.859642","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:31:27.861194","last_change":"2024-04-17 22:31:27.861194","last_active":"2024-04-17 22:31:27.861194","last_peered":"2024-04-17 22:31:27.861194","last_clean":"2024-04-17 22:31:27.861194","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:31:27.861194","last_undegraded":"2024-04-17 22:31:27.861194","last_fullsized":"2024-04-17 22:31:27.861194","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:27.861194","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:27.861194","last_clean_scrub_stamp":"2024-04-17 22:31:27.861194","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:31:26.859642","last_change":"2024-04-17 22:31:26.859642","last_active":"2024-04-17 22:31:26.859642","last_peered":"2024-04-17 22:31:26.859642","last_clean":"2024-04-17 22:31:26.859642","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:31:26.859642","last_undegraded":"2024-04-17 22:31:26.859642","last_fullsized":"2024-04-17 22:31:26.859642","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:26.859642","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:26.859642","last_clean_scrub_stamp":"2024-04-17 22:31:26.859642","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:31:27.861194","last_change":"2024-04-17 22:31:27.861194","last_active":"2024-04-17 22:31:27.861194","last_peered":"2024-04-17 22:31:27.861194","last_clean":"2024-04-17 22:31:27.861194","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:31:27.861194","last_undegraded":"2024-04-17 22:31:27.861194","last_fullsized":"2024-04-17 22:31:27.861194","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:27.861194","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:27.861194","last_clean_scrub_stamp":"2024-04-17 22:31:27.861194","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:31:26.859642","last_change":"2024-04-17 22:31:26.859642","last_active":"2024-04-17 22:31:26.859642","last_peered":"2024-04-17 22:31:26.859642","last_clean":"2024-04-17 22:31:26.859642","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:31:26.859642","last_undegraded":"2024-04-17 22:31:26.859642","last_fullsized":"2024-04-17 22:31:26.859642","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:26.859642","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:26.859642","last_clean_scrub_stamp":"2024-04-17 22:31:26.859642","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:31:27.861194","last_change":"2024-04-17 22:31:27.861194","last_active":"2024-04-17 22:31:27.861194","last_peered":"2024-04-17 22:31:27.861194","last_clean":"2024-04-17 22:31:27.861194","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:31:27.861194","last_undegraded":"2024-04-17 22:31:27.861194","last_fullsized":"2024-04-17 22:31:27.861194","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:27.861194","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:27.861194","last_clean_scrub_stamp":"2024-04-17 22:31:27.861194","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:31:26.859642","last_change":"2024-04-17 22:31:26.859642","last_active":"2024-04-17 22:31:26.859642","last_peered":"2024-04-17 22:31:26.859642","last_clean":"2024-04-17 22:31:26.859642","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:31:26.859642","last_undegraded":"2024-04-17 22:31:26.859642","last_fullsized":"2024-04-17 22:31:26.859642","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:26.859642","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:26.859642","last_clean_scrub_stamp":"2024-04-17 22:31:26.859642","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:31:27.861194","last_change":"2024-04-17 22:31:27.861194","last_active":"2024-04-17 22:31:27.861194","last_peered":"2024-04-17 22:31:27.861194","last_clean":"2024-04-17 22:31:27.861194","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:31:27.861194","last_undegraded":"2024-04-17 22:31:27.861194","last_fullsized":"2024-04-17 22:31:27.861194","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:27.861194","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:27.861194","last_clean_scrub_stamp":"2024-04-17 22:31:27.861194","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:31:26.859642","last_change":"2024-04-17 22:31:26.859642","last_active":"2024-04-17 22:31:26.859642","last_peered":"2024-04-17 22:31:26.859642","last_clean":"2024-04-17 22:31:26.859642","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:31:26.859642","last_undegraded":"2024-04-17 22:31:26.859642","last_fullsized":"2024-04-17 22:31:26.859642","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:26.859642","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:26.859642","last_clean_scrub_stamp":"2024-04-17 22:31:26.859642","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:31:27.861194","last_change":"2024-04-17 22:31:27.861194","last_active":"2024-04-17 22:31:27.861194","last_peered":"2024-04-17 22:31:27.861194","last_clean":"2024-04-17 22:31:27.861194","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:31:27.861194","last_undegraded":"2024-04-17 22:31:27.861194","last_fullsized":"2024-04-17 22:31:27.861194","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:27.861194","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:27.861194","last_clean_scrub_stamp":"2024-04-17 22:31:27.861194","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:31:26.859642","last_change":"2024-04-17 22:31:26.859642","last_active":"2024-04-17 22:31:26.859642","last_peered":"2024-04-17 22:31:26.859642","last_clean":"2024-04-17 22:31:26.859642","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:31:26.859642","last_undegraded":"2024-04-17 22:31:26.859642","last_fullsized":"2024-04-17 22:31:26.859642","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:26.859642","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:26.859642","last_clean_scrub_stamp":"2024-04-17 22:31:26.859642","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:31:27.861194","last_change":"2024-04-17 22:31:27.861194","last_active":"2024-04-17 22:31:27.861194","last_peered":"2024-04-17 22:31:27.861194","last_clean":"2024-04-17 22:31:27.861194","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:31:27.861194","last_undegraded":"2024-04-17 22:31:27.861194","last_fullsized":"2024-04-17 22:31:27.861194","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:27.861194","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:27.861194","last_clean_scrub_stamp":"2024-04-17 22:31:27.861194","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:31:26.859642","last_change":"2024-04-17 22:31:26.859642","last_active":"2024-04-17 22:31:26.859642","last_peered":"2024-04-17 22:31:26.859642","last_clean":"2024-04-17 22:31:26.859642","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:31:26.859642","last_undegraded":"2024-04-17 22:31:26.859642","last_fullsized":"2024-04-17 22:31:26.859642","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:26.859642","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:26.859642","last_clean_scrub_stamp":"2024-04-17 22:31:26.859642","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:31:27.861194","last_change":"2024-04-17 22:31:27.861194","last_active":"2024-04-17 22:31:27.861194","last_peered":"2024-04-17 22:31:27.861194","last_clean":"2024-04-17 22:31:27.861194","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:31:27.861194","last_undegraded":"2024-04-17 22:31:27.861194","last_fullsized":"2024-04-17 22:31:27.861194","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:27.861194","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:27.861194","last_clean_scrub_stamp":"2024-04-17 22:31:27.861194","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:31:26.859642","last_change":"2024-04-17 22:31:26.859642","last_active":"2024-04-17 22:31:26.859642","last_peered":"2024-04-17 22:31:26.859642","last_clean":"2024-04-17 22:31:26.859642","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:31:26.859642","last_undegraded":"2024-04-17 22:31:26.859642","last_fullsized":"2024-04-17 22:31:26.859642","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:26.859642","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:26.859642","last_clean_scrub_stamp":"2024-04-17 22:31:26.859642","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:31:27.861194","last_change":"2024-04-17 22:31:27.861194","last_active":"2024-04-17 22:31:27.861194","last_peered":"2024-04-17 22:31:27.861194","last_clean":"2024-04-17 22:31:27.861194","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:31:27.861194","last_undegraded":"2024-04-17 22:31:27.861194","last_fullsized":"2024-04-17 22:31:27.861194","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:27.861194","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:27.861194","last_clean_scrub_stamp":"2024-04-17 22:31:27.861194","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:31:26.859642","last_change":"2024-04-17 22:31:26.859642","last_active":"2024-04-17 22:31:26.859642","last_peered":"2024-04-17 22:31:26.859642","last_clean":"2024-04-17 22:31:26.859642","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:31:26.859642","last_undegraded":"2024-04-17 22:31:26.859642","last_fullsized":"2024-04-17 22:31:26.859642","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:26.859642","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:26.859642","last_clean_scrub_stamp":"2024-04-17 22:31:26.859642","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:31:27.861194","last_change":"2024-04-17 22:31:27.861194","last_active":"2024-04-17 22:31:27.861194","last_peered":"2024-04-17 22:31:27.861194","last_clean":"2024-04-17 22:31:27.861194","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:31:27.861194","last_undegraded":"2024-04-17 22:31:27.861194","last_fullsized":"2024-04-17 22:31:27.861194","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:27.861194","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:27.861194","last_clean_scrub_stamp":"2024-04-17 22:31:27.861194","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:31:26.859642","last_change":"2024-04-17 22:31:26.859642","last_active":"2024-04-17 22:31:26.859642","last_peered":"2024-04-17 22:31:26.859642","last_clean":"2024-04-17 22:31:26.859642","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:31:26.859642","last_undegraded":"2024-04-17 22:31:26.859642","last_fullsized":"2024-04-17 22:31:26.859642","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:26.859642","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:26.859642","last_clean_scrub_stamp":"2024-04-17 22:31:26.859642","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:31:27.861194","last_change":"2024-04-17 22:31:27.861194","last_active":"2024-04-17 22:31:27.861194","last_peered":"2024-04-17 22:31:27.861194","last_clean":"2024-04-17 22:31:27.861194","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:31:27.861194","last_undegraded":"2024-04-17 22:31:27.861194","last_fullsized":"2024-04-17 22:31:27.861194","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:27.861194","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:27.861194","last_clean_scrub_stamp":"2024-04-17 22:31:27.861194","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:31:26.859642","last_change":"2024-04-17 22:31:26.859642","last_active":"2024-04-17 22:31:26.859642","last_peered":"2024-04-17 22:31:26.859642","last_clean":"2024-04-17 22:31:26.859642","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:31:26.859642","last_undegraded":"2024-04-17 22:31:26.859642","last_fullsized":"2024-04-17 22:31:26.859642","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:26.859642","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:26.859642","last_clean_scrub_stamp":"2024-04-17 22:31:26.859642","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:31:27.861194","last_change":"2024-04-17 22:31:27.861194","last_active":"2024-04-17 22:31:27.861194","last_peered":"2024-04-17 22:31:27.861194","last_clean":"2024-04-17 22:31:27.861194","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:31:27.861194","last_undegraded":"2024-04-17 22:31:27.861194","last_fullsized":"2024-04-17 22:31:27.861194","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:27.861194","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:27.861194","last_clean_scrub_stamp":"2024-04-17 22:31:27.861194","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:31:26.859642","last_change":"2024-04-17 22:31:26.859642","last_active":"2024-04-17 22:31:26.859642","last_peered":"2024-04-17 22:31:26.859642","last_clean":"2024-04-17 22:31:26.859642","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:31:26.859642","last_undegraded":"2024-04-17 22:31:26.859642","last_fullsized":"2024-04-17 22:31:26.859642","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:26.859642","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:26.859642","last_clean_scrub_stamp":"2024-04-17 22:31:26.859642","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:31:27.861194","last_change":"2024-04-17 22:31:27.861194","last_active":"2024-04-17 22:31:27.861194","last_peered":"2024-04-17 22:31:27.861194","last_clean":"2024-04-17 22:31:27.861194","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:31:27.861194","last_undegraded":"2024-04-17 22:31:27.861194","last_fullsized":"2024-04-17 22:31:27.861194","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:27.861194","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:27.861194","last_clean_scrub_stamp":"2024-04-17 22:31:27.861194","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:31:26.859642","last_change":"2024-04-17 22:31:26.859642","last_active":"2024-04-17 22:31:26.859642","last_peered":"2024-04-17 22:31:26.859642","last_clean":"2024-04-17 22:31:26.859642","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:31:26.859642","last_undegraded":"2024-04-17 22:31:26.859642","last_fullsized":"2024-04-17 22:31:26.859642","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:26.859642","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:26.859642","last_clean_scrub_stamp":"2024-04-17 22:31:26.859642","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:31:27.861194","last_change":"2024-04-17 22:31:27.861194","last_active":"2024-04-17 22:31:27.861194","last_peered":"2024-04-17 22:31:27.861194","last_clean":"2024-04-17 22:31:27.861194","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:31:27.861194","last_undegraded":"2024-04-17 22:31:27.861194","last_fullsized":"2024-04-17 22:31:27.861194","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:27.861194","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:27.861194","last_clean_scrub_stamp":"2024-04-17 22:31:27.861194","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:31:26.859642","last_change":"2024-04-17 22:31:26.859642","last_active":"2024-04-17 22:31:26.859642","last_peered":"2024-04-17 22:31:26.859642","last_clean":"2024-04-17 22:31:26.859642","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:31:26.859642","last_undegraded":"2024-04-17 22:31:26.859642","last_fullsized":"2024-04-17 22:31:26.859642","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:26.859642","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:26.859642","last_clean_scrub_stamp":"2024-04-17 22:31:26.859642","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:31:27.861194","last_change":"2024-04-17 22:31:27.861194","last_active":"2024-04-17 22:31:27.861194","last_peered":"2024-04-17 22:31:27.861194","last_clean":"2024-04-17 22:31:27.861194","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:31:27.861194","last_undegraded":"2024-04-17 22:31:27.861194","last_fullsized":"2024-04-17 22:31:27.861194","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:27.861194","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:27.861194","last_clean_scrub_stamp":"2024-04-17 22:31:27.861194","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:31:26.859642","last_change":"2024-04-17 22:31:26.859642","last_active":"2024-04-17 22:31:26.859642","last_peered":"2024-04-17 22:31:26.859642","last_clean":"2024-04-17 22:31:26.859642","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:31:26.859642","last_undegraded":"2024-04-17 22:31:26.859642","last_fullsized":"2024-04-17 22:31:26.859642","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:26.859642","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:26.859642","last_clean_scrub_stamp":"2024-04-17 22:31:26.859642","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:31:27.861194","last_change":"2024-04-17 22:31:27.861194","last_active":"2024-04-17 22:31:27.861194","last_peered":"2024-04-17 22:31:27.861194","last_clean":"2024-04-17 22:31:27.861194","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:31:27.861194","last_undegraded":"2024-04-17 22:31:27.861194","last_fullsized":"2024-04-17 22:31:27.861194","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:27.861194","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:27.861194","last_clean_scrub_stamp":"2024-04-17 22:31:27.861194","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:31:26.859642","last_change":"2024-04-17 22:31:26.859642","last_active":"2024-04-17 22:31:26.859642","last_peered":"2024-04-17 22:31:26.859642","last_clean":"2024-04-17 22:31:26.859642","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:31:26.859642","last_undegraded":"2024-04-17 22:31:26.859642","last_fullsized":"2024-04-17 22:31:26.859642","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:26.859642","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:26.859642","last_clean_scrub_stamp":"2024-04-17 22:31:26.859642","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:31:27.861194","last_change":"2024-04-17 22:31:27.861194","last_active":"2024-04-17 22:31:27.861194","last_peered":"2024-04-17 22:31:27.861194","last_clean":"2024-04-17 22:31:27.861194","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:31:27.861194","last_undegraded":"2024-04-17 22:31:27.861194","last_fullsized":"2024-04-17 22:31:27.861194","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:27.861194","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:27.861194","last_clean_scrub_stamp":"2024-04-17 22:31:27.861194","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:31:26.859642","last_change":"2024-04-17 22:31:26.859642","last_active":"2024-04-17 22:31:26.859642","last_peered":"2024-04-17 22:31:26.859642","last_clean":"2024-04-17 22:31:26.859642","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:31:26.859642","last_undegraded":"2024-04-17 22:31:26.859642","last_fullsized":"2024-04-17 22:31:26.859642","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:26.859642","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:26.859642","last_clean_scrub_stamp":"2024-04-17 22:31:26.859642","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:31:27.861194","last_change":"2024-04-17 22:31:27.861194","last_active":"2024-04-17 22:31:27.861194","last_peered":"2024-04-17 22:31:27.861194","last_clean":"2024-04-17 22:31:27.861194","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:31:27.861194","last_undegraded":"2024-04-17 22:31:27.861194","last_fullsized":"2024-04-17 22:31:27.861194","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:27.861194","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:27.861194","last_clean_scrub_stamp":"2024-04-17 22:31:27.861194","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:31:26.859642","last_change":"2024-04-17 22:31:26.859642","last_active":"2024-04-17 22:31:26.859642","last_peered":"2024-04-17 22:31:26.859642","last_clean":"2024-04-17 22:31:26.859642","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:31:26.859642","last_undegraded":"2024-04-17 22:31:26.859642","last_fullsized":"2024-04-17 22:31:26.859642","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:26.859642","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:26.859642","last_clean_scrub_stamp":"2024-04-17 22:31:26.859642","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:31:27.861194","last_change":"2024-04-17 22:31:27.861194","last_active":"2024-04-17 22:31:27.861194","last_peered":"2024-04-17 22:31:27.861194","last_clean":"2024-04-17 22:31:27.861194","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:31:27.861194","last_undegraded":"2024-04-17 22:31:27.861194","last_fullsized":"2024-04-17 22:31:27.861194","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:27.861194","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:27.861194","last_clean_scrub_stamp":"2024-04-17 22:31:27.861194","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:31:26.859642","last_change":"2024-04-17 22:31:26.859642","last_active":"2024-04-17 22:31:26.859642","last_peered":"2024-04-17 22:31:26.859642","last_clean":"2024-04-17 22:31:26.859642","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:31:26.859642","last_undegraded":"2024-04-17 22:31:26.859642","last_fullsized":"2024-04-17 22:31:26.859642","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:26.859642","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:26.859642","last_clean_scrub_stamp":"2024-04-17 22:31:26.859642","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:31:27.861194","last_change":"2024-04-17 22:31:27.861194","last_active":"2024-04-17 22:31:27.861194","last_peered":"2024-04-17 22:31:27.861194","last_clean":"2024-04-17 22:31:27.861194","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:31:27.861194","last_undegraded":"2024-04-17 22:31:27.861194","last_fullsized":"2024-04-17 22:31:27.861194","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:27.861194","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:27.861194","last_clean_scrub_stamp":"2024-04-17 22:31:27.861194","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:31:26.859642","last_change":"2024-04-17 22:31:26.859642","last_active":"2024-04-17 22:31:26.859642","last_peered":"2024-04-17 22:31:26.859642","last_clean":"2024-04-17 22:31:26.859642","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:31:26.859642","last_undegraded":"2024-04-17 22:31:26.859642","last_fullsized":"2024-04-17 22:31:26.859642","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:26.859642","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:26.859642","last_clean_scrub_stamp":"2024-04-17 22:31:26.859642","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:31:27.861194","last_change":"2024-04-17 22:31:27.861194","last_active":"2024-04-17 22:31:27.861194","last_peered":"2024-04-17 22:31:27.861194","last_clean":"2024-04-17 22:31:27.861194","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:31:27.861194","last_undegraded":"2024-04-17 22:31:27.861194","last_fullsized":"2024-04-17 22:31:27.861194","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:27.861194","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:27.861194","last_clean_scrub_stamp":"2024-04-17 22:31:27.861194","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:31:26.859642","last_change":"2024-04-17 22:31:26.859642","last_active":"2024-04-17 22:31:26.859642","last_peered":"2024-04-17 22:31:26.859642","last_clean":"2024-04-17 22:31:26.859642","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:31:26.859642","last_undegraded":"2024-04-17 22:31:26.859642","last_fullsized":"2024-04-17 22:31:26.859642","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:26.859642","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:26.859642","last_clean_scrub_stamp":"2024-04-17 22:31:26.859642","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:31:27.861194","last_change":"2024-04-17 22:31:27.861194","last_active":"2024-04-17 22:31:27.861194","last_peered":"2024-04-17 22:31:27.861194","last_clean":"2024-04-17 22:31:27.861194","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:31:27.861194","last_undegraded":"2024-04-17 22:31:27.861194","last_fullsized":"2024-04-17 22:31:27.861194","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:27.861194","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:27.861194","last_clean_scrub_stamp":"2024-04-17 22:31:27.861194","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:31:26.859642","last_change":"2024-04-17 22:31:26.859642","last_active":"2024-04-17 22:31:26.859642","last_peered":"2024-04-17 22:31:26.859642","last_clean":"2024-04-17 22:31:26.859642","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:31:26.859642","last_undegraded":"2024-04-17 22:31:26.859642","last_fullsized":"2024-04-17 22:31:26.859642","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:26.859642","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:26.859642","last_clean_scrub_stamp":"2024-04-17 22:31:26.859642","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:31:27.861194","last_change":"2024-04-17 22:31:27.861194","last_active":"2024-04-17 22:31:27.861194","last_peered":"2024-04-17 22:31:27.861194","last_clean":"2024-04-17 22:31:27.861194","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:31:27.861194","last_undegraded":"2024-04-17 22:31:27.861194","last_fullsized":"2024-04-17 22:31:27.861194","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:27.861194","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:27.861194","last_clean_scrub_stamp":"2024-04-17 22:31:27.861194","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:31:26.859642","last_change":"2024-04-17 22:31:26.859642","last_active":"2024-04-17 22:31:26.859642","last_peered":"2024-04-17 22:31:26.859642","last_clean":"2024-04-17 22:31:26.859642","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:31:26.859642","last_undegraded":"2024-04-17 22:31:26.859642","last_fullsized":"2024-04-17 22:31:26.859642","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:26.859642","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:26.859642","last_clean_scrub_stamp":"2024-04-17 22:31:26.859642","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:31:27.861194","last_change":"2024-04-17 22:31:27.861194","last_active":"2024-04-17 22:31:27.861194","last_peered":"2024-04-17 22:31:27.861194","last_clean":"2024-04-17 22:31:27.861194","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:31:27.861194","last_undegraded":"2024-04-17 22:31:27.861194","last_fullsized":"2024-04-17 22:31:27.861194","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:27.861194","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:27.861194","last_clean_scrub_stamp":"2024-04-17 22:31:27.861194","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:31:26.859642","last_change":"2024-04-17 22:31:26.859642","last_active":"2024-04-17 22:31:26.859642","last_peered":"2024-04-17 22:31:26.859642","last_clean":"2024-04-17 22:31:26.859642","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:31:26.859642","last_undegraded":"2024-04-17 22:31:26.859642","last_fullsized":"2024-04-17 22:31:26.859642","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:26.859642","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:26.859642","last_clean_scrub_stamp":"2024-04-17 22:31:26.859642","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:31:27.861194","last_change":"2024-04-17 22:31:27.861194","last_active":"2024-04-17 22:31:27.861194","last_peered":"2024-04-17 22:31:27.861194","last_clean":"2024-04-17 22:31:27.861194","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:31:27.861194","last_undegraded":"2024-04-17 22:31:27.861194","last_fullsized":"2024-04-17 22:31:27.861194","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:27.861194","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:27.861194","last_clean_scrub_stamp":"2024-04-17 22:31:27.861194","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:31:26.859642","last_change":"2024-04-17 22:31:26.859642","last_active":"2024-04-17 22:31:26.859642","last_peered":"2024-04-17 22:31:26.859642","last_clean":"2024-04-17 22:31:26.859642","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:31:26.859642","last_undegraded":"2024-04-17 22:31:26.859642","last_fullsized":"2024-04-17 22:31:26.859642","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:26.859642","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:26.859642","last_clean_scrub_stamp":"2024-04-17 22:31:26.859642","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:31:27.861194","last_change":"2024-04-17 22:31:27.861194","last_active":"2024-04-17 22:31:27.861194","last_peered":"2024-04-17 22:31:27.861194","last_clean":"2024-04-17 22:31:27.861194","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:31:27.861194","last_undegraded":"2024-04-17 22:31:27.861194","last_fullsized":"2024-04-17 22:31:27.861194","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:27.861194","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:27.861194","last_clean_scrub_stamp":"2024-04-17 22:31:27.861194","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:31:26.859642","last_change":"2024-04-17 22:31:26.859642","last_active":"2024-04-17 22:31:26.859642","last_peered":"2024-04-17 22:31:26.859642","last_clean":"2024-04-17 22:31:26.859642","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:31:26.859642","last_undegraded":"2024-04-17 22:31:26.859642","last_fullsized":"2024-04-17 22:31:26.859642","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:26.859642","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:26.859642","last_clean_scrub_stamp":"2024-04-17 22:31:26.859642","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:31:27.861194","last_change":"2024-04-17 22:31:27.861194","last_active":"2024-04-17 22:31:27.861194","last_peered":"2024-04-17 22:31:27.861194","last_clean":"2024-04-17 22:31:27.861194","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:31:27.861194","last_undegraded":"2024-04-17 22:31:27.861194","last_fullsized":"2024-04-17 22:31:27.861194","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:27.861194","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:27.861194","last_clean_scrub_stamp":"2024-04-17 22:31:27.861194","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:31:26.859642","last_change":"2024-04-17 22:31:26.859642","last_active":"2024-04-17 22:31:26.859642","last_peered":"2024-04-17 22:31:26.859642","last_clean":"2024-04-17 22:31:26.859642","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:31:26.859642","last_undegraded":"2024-04-17 22:31:26.859642","last_fullsized":"2024-04-17 22:31:26.859642","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:26.859642","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:26.859642","last_clean_scrub_stamp":"2024-04-17 22:31:26.859642","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:31:27.861194","last_change":"2024-04-17 22:31:27.861194","last_active":"2024-04-17 22:31:27.861194","last_peered":"2024-04-17 22:31:27.861194","last_clean":"2024-04-17 22:31:27.861194","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:31:27.861194","last_undegraded":"2024-04-17 22:31:27.861194","last_fullsized":"2024-04-17 22:31:27.861194","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:27.861194","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:27.861194","last_clean_scrub_stamp":"2024-04-17 22:31:27.861194","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:31:26.859642","last_change":"2024-04-17 22:31:26.859642","last_active":"2024-04-17 22:31:26.859642","last_peered":"2024-04-17 22:31:26.859642","last_clean":"2024-04-17 22:31:26.859642","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:31:26.859642","last_undegraded":"2024-04-17 22:31:26.859642","last_fullsized":"2024-04-17 22:31:26.859642","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:26.859642","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:26.859642","last_clean_scrub_stamp":"2024-04-17 22:31:26.859642","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:31:27.861194","last_change":"2024-04-17 22:31:27.861194","last_active":"2024-04-17 22:31:27.861194","last_peered":"2024-04-17 22:31:27.861194","last_clean":"2024-04-17 22:31:27.861194","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:31:27.861194","last_undegraded":"2024-04-17 22:31:27.861194","last_fullsized":"2024-04-17 22:31:27.861194","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:27.861194","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:27.861194","last_clean_scrub_stamp":"2024-04-17 22:31:27.861194","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:31:26.859642","last_change":"2024-04-17 22:31:26.859642","last_active":"2024-04-17 22:31:26.859642","last_peered":"2024-04-17 22:31:26.859642","last_clean":"2024-04-17 22:31:26.859642","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:31:26.859642","last_undegraded":"2024-04-17 22:31:26.859642","last_fullsized":"2024-04-17 22:31:26.859642","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:26.859642","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:26.859642","last_clean_scrub_stamp":"2024-04-17 22:31:26.859642","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:31:27.861194","last_change":"2024-04-17 22:31:27.861194","last_active":"2024-04-17 22:31:27.861194","last_peered":"2024-04-17 22:31:27.861194","last_clean":"2024-04-17 22:31:27.861194","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:31:27.861194","last_undegraded":"2024-04-17 22:31:27.861194","last_fullsized":"2024-04-17 22:31:27.861194","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:27.861194","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:27.861194","last_clean_scrub_stamp":"2024-04-17 22:31:27.861194","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:31:26.859642","last_change":"2024-04-17 22:31:26.859642","last_active":"2024-04-17 22:31:26.859642","last_peered":"2024-04-17 22:31:26.859642","last_clean":"2024-04-17 22:31:26.859642","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:31:26.859642","last_undegraded":"2024-04-17 22:31:26.859642","last_fullsized":"2024-04-17 22:31:26.859642","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:26.859642","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:26.859642","last_clean_scrub_stamp":"2024-04-17 22:31:26.859642","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:31:27.861194","last_change":"2024-04-17 22:31:27.861194","last_active":"2024-04-17 22:31:27.861194","last_peered":"2024-04-17 22:31:27.861194","last_clean":"2024-04-17 22:31:27.861194","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:31:27.861194","last_undegraded":"2024-04-17 22:31:27.861194","last_fullsized":"2024-04-17 22:31:27.861194","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:27.861194","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:27.861194","last_clean_scrub_stamp":"2024-04-17 22:31:27.861194","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:31:26.859642","last_change":"2024-04-17 22:31:26.859642","last_active":"2024-04-17 22:31:26.859642","last_peered":"2024-04-17 22:31:26.859642","last_clean":"2024-04-17 22:31:26.859642","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:31:26.859642","last_undegraded":"2024-04-17 22:31:26.859642","last_fullsized":"2024-04-17 22:31:26.859642","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:26.859642","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:26.859642","last_clean_scrub_stamp":"2024-04-17 22:31:26.859642","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:31:27.861194","last_change":"2024-04-17 22:31:27.861194","last_active":"2024-04-17 22:31:27.861194","last_peered":"2024-04-17 22:31:27.861194","last_clean":"2024-04-17 22:31:27.861194","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:31:27.861194","last_undegraded":"2024-04-17 22:31:27.861194","last_fullsized":"2024-04-17 22:31:27.861194","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:27.861194","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:27.861194","last_clean_scrub_stamp":"2024-04-17 22:31:27.861194","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:31:26.859642","last_change":"2024-04-17 22:31:26.859642","last_active":"2024-04-17 22:31:26.859642","last_peered":"2024-04-17 22:31:26.859642","last_clean":"2024-04-17 22:31:26.859642","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:31:26.859642","last_undegraded":"2024-04-17 22:31:26.859642","last_fullsized":"2024-04-17 22:31:26.859642","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:26.859642","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:26.859642","last_clean_scrub_stamp":"2024-04-17 22:31:26.859642","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":"18","reported_epoch":"20","state":"active+clean","last_fresh":"2024-04-17 22:31:20.621398","last_change":"2024-04-17 22:28:59.253977","last_active":"2024-04-17 22:31:20.621398","last_peered":"2024-04-17 22:31:20.621398","last_clean":"2024-04-17 22:31:20.621398","last_became_active":"2024-04-17 22:28:59.253641","last_became_peered":"2024-04-17 22:28:59.253641","last_unstale":"2024-04-17 22:31:20.621398","last_undegraded":"2024-04-17 22:31:20.621398","last_fullsized":"2024-04-17 22:31:20.621398","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:28:58.239187","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:28:58.239187","last_clean_scrub_stamp":"2024-04-17 22:28:58.239187","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:31:27.861194","last_change":"2024-04-17 22:31:27.861194","last_active":"2024-04-17 22:31:27.861194","last_peered":"2024-04-17 22:31:27.861194","last_clean":"2024-04-17 22:31:27.861194","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:31:27.861194","last_undegraded":"2024-04-17 22:31:27.861194","last_fullsized":"2024-04-17 22:31:27.861194","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:27.861194","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:27.861194","last_clean_scrub_stamp":"2024-04-17 22:31:27.861194","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:31:26.859642","last_change":"2024-04-17 22:31:26.859642","last_active":"2024-04-17 22:31:26.859642","last_peered":"2024-04-17 22:31:26.859642","last_clean":"2024-04-17 22:31:26.859642","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:31:26.859642","last_undegraded":"2024-04-17 22:31:26.859642","last_fullsized":"2024-04-17 22:31:26.859642","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:26.859642","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:26.859642","last_clean_scrub_stamp":"2024-04-17 22:31:26.859642","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":"18","reported_epoch":"20","state":"active+clean","last_fresh":"2024-04-17 22:31:20.720715","last_change":"2024-04-17 22:28:59.255733","last_active":"2024-04-17 22:31:20.720715","last_peered":"2024-04-17 22:31:20.720715","last_clean":"2024-04-17 22:31:20.720715","last_became_active":"2024-04-17 22:28:59.255411","last_became_peered":"2024-04-17 22:28:59.255411","last_unstale":"2024-04-17 22:31:20.720715","last_undegraded":"2024-04-17 22:31:20.720715","last_fullsized":"2024-04-17 22:31:20.720715","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:28:58.239187","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:28:58.239187","last_clean_scrub_stamp":"2024-04-17 22:28:58.239187","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:31:27.861194","last_change":"2024-04-17 22:31:27.861194","last_active":"2024-04-17 22:31:27.861194","last_peered":"2024-04-17 22:31:27.861194","last_clean":"2024-04-17 22:31:27.861194","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:31:27.861194","last_undegraded":"2024-04-17 22:31:27.861194","last_fullsized":"2024-04-17 22:31:27.861194","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:27.861194","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:27.861194","last_clean_scrub_stamp":"2024-04-17 22:31:27.861194","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:31:26.859642","last_change":"2024-04-17 22:31:26.859642","last_active":"2024-04-17 22:31:26.859642","last_peered":"2024-04-17 22:31:26.859642","last_clean":"2024-04-17 22:31:26.859642","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:31:26.859642","last_undegraded":"2024-04-17 22:31:26.859642","last_fullsized":"2024-04-17 22:31:26.859642","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:26.859642","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:26.859642","last_clean_scrub_stamp":"2024-04-17 22:31:26.859642","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":"18","reported_epoch":"20","state":"active+clean","last_fresh":"2024-04-17 22:31:20.621122","last_change":"2024-04-17 22:28:59.252728","last_active":"2024-04-17 22:31:20.621122","last_peered":"2024-04-17 22:31:20.621122","last_clean":"2024-04-17 22:31:20.621122","last_became_active":"2024-04-17 22:28:59.252345","last_became_peered":"2024-04-17 22:28:59.252345","last_unstale":"2024-04-17 22:31:20.621122","last_undegraded":"2024-04-17 22:31:20.621122","last_fullsized":"2024-04-17 22:31:20.621122","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:28:58.239187","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:28:58.239187","last_clean_scrub_stamp":"2024-04-17 22:28:58.239187","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:31:27.861194","last_change":"2024-04-17 22:31:27.861194","last_active":"2024-04-17 22:31:27.861194","last_peered":"2024-04-17 22:31:27.861194","last_clean":"2024-04-17 22:31:27.861194","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:31:27.861194","last_undegraded":"2024-04-17 22:31:27.861194","last_fullsized":"2024-04-17 22:31:27.861194","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:27.861194","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:27.861194","last_clean_scrub_stamp":"2024-04-17 22:31:27.861194","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:31:26.859642","last_change":"2024-04-17 22:31:26.859642","last_active":"2024-04-17 22:31:26.859642","last_peered":"2024-04-17 22:31:26.859642","last_clean":"2024-04-17 22:31:26.859642","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:31:26.859642","last_undegraded":"2024-04-17 22:31:26.859642","last_fullsized":"2024-04-17 22:31:26.859642","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:26.859642","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:26.859642","last_clean_scrub_stamp":"2024-04-17 22:31:26.859642","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":"18","reported_epoch":"20","state":"active+clean","last_fresh":"2024-04-17 22:31:20.620247","last_change":"2024-04-17 22:28:59.253392","last_active":"2024-04-17 22:31:20.620247","last_peered":"2024-04-17 22:31:20.620247","last_clean":"2024-04-17 22:31:20.620247","last_became_active":"2024-04-17 22:28:59.253049","last_became_peered":"2024-04-17 22:28:59.253049","last_unstale":"2024-04-17 22:31:20.620247","last_undegraded":"2024-04-17 22:31:20.620247","last_fullsized":"2024-04-17 22:31:20.620247","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:28:58.239187","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:28:58.239187","last_clean_scrub_stamp":"2024-04-17 22:28:58.239187","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:31:27.861194","last_change":"2024-04-17 22:31:27.861194","last_active":"2024-04-17 22:31:27.861194","last_peered":"2024-04-17 22:31:27.861194","last_clean":"2024-04-17 22:31:27.861194","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:31:27.861194","last_undegraded":"2024-04-17 22:31:27.861194","last_fullsized":"2024-04-17 22:31:27.861194","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:27.861194","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:27.861194","last_clean_scrub_stamp":"2024-04-17 22:31:27.861194","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:31:26.859642","last_change":"2024-04-17 22:31:26.859642","last_active":"2024-04-17 22:31:26.859642","last_peered":"2024-04-17 22:31:26.859642","last_clean":"2024-04-17 22:31:26.859642","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:31:26.859642","last_undegraded":"2024-04-17 22:31:26.859642","last_fullsized":"2024-04-17 22:31:26.859642","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:26.859642","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:26.859642","last_clean_scrub_stamp":"2024-04-17 22:31:26.859642","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":"18","reported_epoch":"20","state":"active+clean","last_fresh":"2024-04-17 22:31:20.624163","last_change":"2024-04-17 22:28:59.254907","last_active":"2024-04-17 22:31:20.624163","last_peered":"2024-04-17 22:31:20.624163","last_clean":"2024-04-17 22:31:20.624163","last_became_active":"2024-04-17 22:28:59.254633","last_became_peered":"2024-04-17 22:28:59.254633","last_unstale":"2024-04-17 22:31:20.624163","last_undegraded":"2024-04-17 22:31:20.624163","last_fullsized":"2024-04-17 22:31:20.624163","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:28:58.239187","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:28:58.239187","last_clean_scrub_stamp":"2024-04-17 22:28:58.239187","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:31:27.861194","last_change":"2024-04-17 22:31:27.861194","last_active":"2024-04-17 22:31:27.861194","last_peered":"2024-04-17 22:31:27.861194","last_clean":"2024-04-17 22:31:27.861194","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:31:27.861194","last_undegraded":"2024-04-17 22:31:27.861194","last_fullsized":"2024-04-17 22:31:27.861194","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:27.861194","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:27.861194","last_clean_scrub_stamp":"2024-04-17 22:31:27.861194","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:31:26.859642","last_change":"2024-04-17 22:31:26.859642","last_active":"2024-04-17 22:31:26.859642","last_peered":"2024-04-17 22:31:26.859642","last_clean":"2024-04-17 22:31:26.859642","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:31:26.859642","last_undegraded":"2024-04-17 22:31:26.859642","last_fullsized":"2024-04-17 22:31:26.859642","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:26.859642","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:26.859642","last_clean_scrub_stamp":"2024-04-17 22:31:26.859642","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":"18","reported_epoch":"20","state":"active+clean","last_fresh":"2024-04-17 22:31:20.720929","last_change":"2024-04-17 22:28:59.254217","last_active":"2024-04-17 22:31:20.720929","last_peered":"2024-04-17 22:31:20.720929","last_clean":"2024-04-17 22:31:20.720929","last_became_active":"2024-04-17 22:28:59.253877","last_became_peered":"2024-04-17 22:28:59.253877","last_unstale":"2024-04-17 22:31:20.720929","last_undegraded":"2024-04-17 22:31:20.720929","last_fullsized":"2024-04-17 22:31:20.720929","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:28:58.239187","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:28:58.239187","last_clean_scrub_stamp":"2024-04-17 22:28:58.239187","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:31:27.861194","last_change":"2024-04-17 22:31:27.861194","last_active":"2024-04-17 22:31:27.861194","last_peered":"2024-04-17 22:31:27.861194","last_clean":"2024-04-17 22:31:27.861194","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:31:27.861194","last_undegraded":"2024-04-17 22:31:27.861194","last_fullsized":"2024-04-17 22:31:27.861194","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:27.861194","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:27.861194","last_clean_scrub_stamp":"2024-04-17 22:31:27.861194","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:31:26.859642","last_change":"2024-04-17 22:31:26.859642","last_active":"2024-04-17 22:31:26.859642","last_peered":"2024-04-17 22:31:26.859642","last_clean":"2024-04-17 22:31:26.859642","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:31:26.859642","last_undegraded":"2024-04-17 22:31:26.859642","last_fullsized":"2024-04-17 22:31:26.859642","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:26.859642","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:26.859642","last_clean_scrub_stamp":"2024-04-17 22:31:26.859642","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":"18","reported_epoch":"20","state":"active+clean","last_fresh":"2024-04-17 22:31:20.722439","last_change":"2024-04-17 22:28:59.254830","last_active":"2024-04-17 22:31:20.722439","last_peered":"2024-04-17 22:31:20.722439","last_clean":"2024-04-17 22:31:20.722439","last_became_active":"2024-04-17 22:28:59.254505","last_became_peered":"2024-04-17 22:28:59.254505","last_unstale":"2024-04-17 22:31:20.722439","last_undegraded":"2024-04-17 22:31:20.722439","last_fullsized":"2024-04-17 22:31:20.722439","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:28:58.239187","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:28:58.239187","last_clean_scrub_stamp":"2024-04-17 22:28:58.239187","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:31:27.861194","last_change":"2024-04-17 22:31:27.861194","last_active":"2024-04-17 22:31:27.861194","last_peered":"2024-04-17 22:31:27.861194","last_clean":"2024-04-17 22:31:27.861194","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:31:27.861194","last_undegraded":"2024-04-17 22:31:27.861194","last_fullsized":"2024-04-17 22:31:27.861194","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:27.861194","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:27.861194","last_clean_scrub_stamp":"2024-04-17 22:31:27.861194","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2024-04-17 22:31:26.859642","last_change":"2024-04-17 22:31:26.859642","last_active":"2024-04-17 22:31:26.859642","last_peered":"2024-04-17 22:31:26.859642","last_clean":"2024-04-17 22:31:26.859642","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17 22:31:26.859642","last_undegraded":"2024-04-17 22:31:26.859642","last_fullsized":"2024-04-17 22:31:26.859642","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:26.859642","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:26.859642","last_clean_scrub_stamp":"2024-04-17 22:31:26.859642","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":"18","reported_epoch":"20","state":"active+clean","last_fresh":"2024-04-17 22:31:20.624777","last_change":"2024-04-17 22:28:59.254055","last_active":"2024-04-17 22:31:20.624777","last_peered":"2024-04-17 22:31:20.624777","last_clean":"2024-04-17 22:31:20.624777","last_became_active":"2024-04-17 22:28:59.253750","last_became_peered":"2024-04-17 22:28:59.253750","last_unstale":"2024-04-17 22:31:20.624777","last_undegraded":"2024-04-17 22:31:20.624777","last_fullsized":"2024-04-17 22:31:20.624777","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:28:58.239187","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:28:58.239187","last_clean_scrub_stamp":"2024-04-17 22:28:58.239187","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705698,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"kb":94371840,"kb_used":1048776,"kb_used_data":184,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323064,"statfs":{"total":96636764160,"available":95562817536,"internally_reserved":1073741824,"allocated":188416,"data_stored":55175,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 17 22:30:59 2024","interfaces":[{"interface":"back","average":{"1min":0.308,"5min":0.30399999999999999,"15min":0.30399999999999999},"min":{"1min":0.20399999999999999,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"last":0.24299999999999999},{"interface":"front","average":{"1min":0.30199999999999999,"5min":0.314,"15min":0.316},"min":{"1min":0.21099999999999999,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.44600000000000001,"5min":0.52400000000000002,"15min":0.52400000000000002},"last":0.218}]},{"osd":1,"last update":"Wed Apr 17 22:30:59 2024","interfaces":[{"interface":"back","average":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"min":{"1min":0.223,"5min":0.19600000000000001,"15min":0.19600000000000001},"max":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"last":0.23499999999999999},{"interface":"front","average":{"1min":0.33400000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"min":{"1min":0.249,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.438,"5min":0.55300000000000005,"15min":0.55300000000000005},"last":0.222}]},{"osd":2,"last update":"Wed Apr 17 22:30:59 2024","interfaces":[{"interface":"back","average":{"1min":0.33800000000000002,"5min":0.34799999999999998,"15min":0.34999999999999998},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.46200000000000002,"5min":0.50600000000000001,"15min":0.50600000000000001},"last":0.26200000000000001},{"interface":"front","average":{"1min":0.32800000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"min":{"1min":0.23200000000000001,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.46899999999999997,"5min":0.47699999999999998,"15min":0.47699999999999998},"last":0.20300000000000001}]}]},{"osd":2,"up_from":9,"seq":38654705698,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"kb":94371840,"kb_used":1048768,"kb_used_data":184,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323072,"statfs":{"total":96636764160,"available":95562825728,"internally_reserved":1073741824,"allocated":188416,"data_stored":55175,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 17 22:30:58 2024","interfaces":[{"interface":"back","average":{"1min":0.30099999999999999,"5min":0.28499999999999998,"15min":0.28299999999999997},"min":{"1min":0.22600000000000001,"5min":0.112,"15min":0.112},"max":{"1min":0.41499999999999998,"5min":0.46400000000000002,"15min":0.46400000000000002},"last":0.28199999999999997},{"interface":"front","average":{"1min":0.33600000000000002,"5min":0.314,"15min":0.31},"min":{"1min":0.27600000000000002,"5min":0.155,"15min":0.155},"max":{"1min":0.40400000000000003,"5min":0.49199999999999999,"15min":0.49199999999999999},"last":0.29099999999999998}]},{"osd":1,"last update":"Wed Apr 17 22:30:58 2024","interfaces":[{"interface":"back","average":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"min":{"1min":0.214,"5min":0.17899999999999999,"15min":0.17899999999999999},"max":{"1min":0.377,"5min":0.52900000000000003,"15min":0.52900000000000003},"last":0.23899999999999999},{"interface":"front","average":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"min":{"1min":0.245,"5min":0.192,"15min":0.192},"max":{"1min":0.38800000000000001,"5min":0.47199999999999998,"15min":0.47199999999999998},"last":0.30199999999999999}]},{"osd":3,"last update":"Wed Apr 17 22:30:58 2024","interfaces":[{"interface":"back","average":{"1min":0.30399999999999999,"5min":0.29599999999999999,"15min":0.29499999999999998},"min":{"1min":0.24199999999999999,"5min":0.191,"15min":0.191},"max":{"1min":0.42599999999999999,"5min":0.45600000000000002,"15min":0.45600000000000002},"last":0.25700000000000001},{"interface":"front","average":{"1min":0.31900000000000001,"5min":0.311,"15min":0.31},"min":{"1min":0.26200000000000001,"5min":0.214,"15min":0.214},"max":{"1min":0.41299999999999998,"5min":0.48699999999999999,"15min":0.48699999999999999},"last":0.27200000000000002}]}]},{"osd":0,"up_from":9,"seq":38654705698,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"kb":94371840,"kb_used":1048768,"kb_used_data":184,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323072,"statfs":{"total":96636764160,"available":95562825728,"internally_reserved":1073741824,"allocated":188416,"data_stored":55175,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Apr 17 22:30:56 2024","interfaces":[{"interface":"back","average":{"1min":0.248,"5min":0.28299999999999997,"15min":0.28899999999999998},"min":{"1min":0.159,"5min":0.159,"15min":0.159},"max":{"1min":0.33200000000000002,"5min":0.40899999999999997,"15min":0.40899999999999997},"last":0.21099999999999999},{"interface":"front","average":{"1min":0.26000000000000001,"5min":0.29199999999999998,"15min":0.29699999999999999},"min":{"1min":0.189,"5min":0.16600000000000001,"15min":0.16600000000000001},"max":{"1min":0.35399999999999998,"5min":0.42999999999999999,"15min":0.42999999999999999},"last":0.23400000000000001}]},{"osd":2,"last update":"Wed Apr 17 22:30:56 2024","interfaces":[{"interface":"back","average":{"1min":0.28899999999999998,"5min":0.32100000000000001,"15min":0.32700000000000001},"min":{"1min":0.218,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.39500000000000002,"5min":0.48599999999999999,"15min":0.48599999999999999},"last":0.26700000000000002},{"interface":"front","average":{"1min":0.27800000000000002,"5min":0.317,"15min":0.32300000000000001},"min":{"1min":0.19400000000000001,"5min":0.19400000000000001,"15min":0.19400000000000001},"max":{"1min":0.371,"5min":0.60699999999999998,"15min":0.60699999999999998},"last":0.222}]},{"osd":3,"last update":"Wed Apr 17 22:30:56 2024","interfaces":[{"interface":"back","average":{"1min":0.311,"5min":0.33400000000000002,"15min":0.33800000000000002},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.40600000000000003,"5min":0.505,"15min":0.505},"last":0.25900000000000001},{"interface":"front","average":{"1min":0.28599999999999998,"5min":0.32000000000000001,"15min":0.32600000000000001},"min":{"1min":0.20000000000000001,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.38300000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"last":0.253}]}]},{"osd":1,"up_from":9,"seq":38654705697,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"kb":94371840,"kb_used":1048768,"kb_used_data":184,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323072,"statfs":{"total":96636764160,"available":95562825728,"internally_reserved":1073741824,"allocated":188416,"data_stored":55175,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 17 22:31:00 2024","interfaces":[{"interface":"back","average":{"1min":0.26900000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"min":{"1min":0.128,"5min":0.128,"15min":0.128},"max":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"last":0.20100000000000001},{"interface":"front","average":{"1min":0.28000000000000003,"5min":0.27600000000000002,"15min":0.27600000000000002},"min":{"1min":0.19400000000000001,"5min":0.191,"15min":0.191},"max":{"1min":0.441,"5min":0.441,"15min":0.441},"last":0.28799999999999998}]},{"osd":2,"last update":"Wed Apr 17 22:31:00 2024","interfaces":[{"interface":"back","average":{"1min":0.28599999999999998,"5min":0.27900000000000003,"15min":0.27800000000000002},"min":{"1min":0.153,"5min":0.153,"15min":0.153},"max":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"last":0.27800000000000002},{"interface":"front","average":{"1min":0.28599999999999998,"5min":0.29099999999999998,"15min":0.29199999999999998},"min":{"1min":0.18099999999999999,"5min":0.18099999999999999,"15min":0.18099999999999999},"max":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"last":0.20999999999999999}]},{"osd":3,"last update":"Wed Apr 17 22:31:00 2024","interfaces":[{"interface":"back","average":{"1min":0.30499999999999999,"5min":0.29699999999999999,"15min":0.29499999999999998},"min":{"1min":0.23300000000000001,"5min":0.218,"15min":0.218},"max":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"last":0.26900000000000002},{"interface":"front","average":{"1min":0.309,"5min":0.315,"15min":0.316},"min":{"1min":0.245,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"last":0.318}]}]}]}} 2024-04-17T22:31:31.081 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-17T22:31:31.247 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.256 7f1fe96c0700 1 Processor -- start 2024-04-17T22:31:31.247 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.256 7f1fe96c0700 1 -- start start 2024-04-17T22:31:31.247 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.256 7f1fe96c0700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f1fe40f5770 0x7f1fe410c110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:31.247 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.256 7f1fe96c0700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f1fe410cb90 0x7f1fe41a1400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:31.248 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.256 7f1fe96c0700 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1fe41a1a30 con 0x7f1fe410c740 2024-04-17T22:31:31.248 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.256 7f1fe96c0700 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1fe40f93e0 con 0x7f1fe40f5770 2024-04-17T22:31:31.248 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.256 7f1fe96c0700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fe41a1c50 con 0x7f1fe410cb90 2024-04-17T22:31:31.248 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.257 7f1fe22b1700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f1fe40f5770 0x7f1fe410c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:31.248 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.257 7f1fe2ab2700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f1fe410cb90 0x7f1fe41a1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:31.248 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.257 7f1fe1ab0700 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f1fe410c740 0x7f1fe41a0c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:33820/0 (socket says 172.21.15.102:33820) 2024-04-17T22:31:31.248 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.257 7f1fe1ab0700 1 -- 172.21.15.102:0/3447715083 learned_addr learned my addr 172.21.15.102:0/3447715083 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-17T22:31:31.248 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.257 7f1fe22b1700 1 -- 172.21.15.102:0/3447715083 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f1fe410cb90 msgr2=0x7f1fe41a1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:31:31.248 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.257 7f1fe22b1700 1 --2- 172.21.15.102:0/3447715083 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f1fe410cb90 0x7f1fe41a1400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:31.248 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.257 7f1fe22b1700 1 -- 172.21.15.102:0/3447715083 >> v1:172.21.15.102:6789/0 conn(0x7f1fe410c740 legacy=0x7f1fe41a0c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:31.248 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.257 7f1fe12af700 1 -- 172.21.15.102:0/3447715083 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1475051000 0 0) 0x7f1fcc001110 con 0x7f1fe410c740 2024-04-17T22:31:31.248 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.257 7f1fe22b1700 1 -- 172.21.15.102:0/3447715083 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fe41aa330 con 0x7f1fe40f5770 2024-04-17T22:31:31.249 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.257 7f1fe2ab2700 1 --2- 172.21.15.102:0/3447715083 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f1fe410cb90 0x7f1fe41a1400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:31.249 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.257 7f1fe22b1700 1 --2- 172.21.15.102:0/3447715083 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1fe40f5770 0x7f1fe410c110 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 rx=0x7f1fe4089f10 tx=0x7f1fd400ba20).ready entity=mon.0 client_cookie=631ee2acf4edbaed server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:31.249 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.257 7f1fe12af700 1 -- 172.21.15.102:0/3447715083 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f1fe41aa330 con 0x7f1fe40f5770 2024-04-17T22:31:31.249 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.257 7f1fe12af700 1 -- 172.21.15.102:0/3447715083 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fd400d5c0 con 0x7f1fe40f5770 2024-04-17T22:31:31.249 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.257 7f1fe12af700 1 -- 172.21.15.102:0/3447715083 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f1fd400d990 con 0x7f1fe40f5770 2024-04-17T22:31:31.249 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.257 7f1fe96c0700 1 -- 172.21.15.102:0/3447715083 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1fe40f5770 msgr2=0x7f1fe410c110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:31.249 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.257 7f1fe96c0700 1 --2- 172.21.15.102:0/3447715083 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1fe40f5770 0x7f1fe410c110 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 rx=0x7f1fe4089f10 tx=0x7f1fd400ba20).stop 2024-04-17T22:31:31.249 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.258 7f1fe96c0700 1 -- 172.21.15.102:0/3447715083 shutdown_connections 2024-04-17T22:31:31.249 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.258 7f1fe96c0700 1 --2- 172.21.15.102:0/3447715083 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f1fe410cb90 0x7f1fe41a1400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:31.249 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.258 7f1fe96c0700 1 --2- 172.21.15.102:0/3447715083 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1fe40f5770 0x7f1fe410c110 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:31.249 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.258 7f1fe96c0700 1 -- 172.21.15.102:0/3447715083 >> 172.21.15.102:0/3447715083 conn(0x7f1fe41138c0 msgr2=0x7f1fe40f4f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:31.249 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.258 7f1fe96c0700 1 -- 172.21.15.102:0/3447715083 shutdown_connections 2024-04-17T22:31:31.249 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.258 7f1fe96c0700 1 -- 172.21.15.102:0/3447715083 wait complete. 2024-04-17T22:31:31.269 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.280 7f1fe96c0700 1 Processor -- start 2024-04-17T22:31:31.269 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.280 7f1fe96c0700 1 -- start start 2024-04-17T22:31:31.269 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.281 7f1fe96c0700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f1fe41a6540 0x7f1fe40d6450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:31.269 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.281 7f1fe96c0700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1fe40d6a80 0x7f1fe41b42c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:31.269 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.281 7f1fe96c0700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f1fe41b48f0 0x7f1fe41b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:31.269 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.281 7f1fe96c0700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fe40dda40 con 0x7f1fe40d6a80 2024-04-17T22:31:31.269 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.281 7f1fe96c0700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fe41b5370 con 0x7f1fe41b48f0 2024-04-17T22:31:31.269 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.281 7f1fe96c0700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1fe40ddf70 con 0x7f1fe41a6540 2024-04-17T22:31:31.269 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.281 7f1fe22b1700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f1fe41a6540 0x7f1fe40d6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:31.270 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.281 7f1fe22b1700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f1fe41a6540 0x7f1fe40d6450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:33818/0 (socket says 172.21.15.102:33818) 2024-04-17T22:31:31.270 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.281 7f1fe22b1700 1 -- 172.21.15.102:0/2227382007 learned_addr learned my addr 172.21.15.102:0/2227382007 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:31:31.270 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.281 7f1fe1ab0700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1fe40d6a80 0x7f1fe41b42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:31.270 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.281 7f1fe2ab2700 1 --2- 172.21.15.102:0/2227382007 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f1fe41b48f0 0x7f1fe41b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:31.270 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.281 7f1fe22b1700 1 -- 172.21.15.102:0/2227382007 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f1fe41b48f0 msgr2=0x7f1fe41b4d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:31.270 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.281 7f1fe22b1700 1 --2- 172.21.15.102:0/2227382007 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f1fe41b48f0 0x7f1fe41b4d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:31.270 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.281 7f1fe22b1700 1 -- 172.21.15.102:0/2227382007 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1fe40d6a80 msgr2=0x7f1fe41b42c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:31.270 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.281 7f1fe22b1700 1 --2- 172.21.15.102:0/2227382007 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1fe40d6a80 0x7f1fe41b42c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:31.270 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.281 7f1fe22b1700 1 -- 172.21.15.102:0/2227382007 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fe40d6ed0 con 0x7f1fe41a6540 2024-04-17T22:31:31.270 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.281 7f1fe1ab0700 1 --2- 172.21.15.102:0/2227382007 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1fe40d6a80 0x7f1fe41b42c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:31.271 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.281 7f1fe22b1700 1 --2- 172.21.15.102:0/2227382007 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f1fe41a6540 0x7f1fe40d6450 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 rx=0x7f1fe40d7690 tx=0x7f1fd40068c0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:31.271 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.282 7f1fdb7fe700 1 -- 172.21.15.102:0/2227382007 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f1fe40d6ed0 con 0x7f1fe41a6540 2024-04-17T22:31:31.271 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.282 7f1fdb7fe700 1 -- 172.21.15.102:0/2227382007 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fd40117e0 con 0x7f1fe41a6540 2024-04-17T22:31:31.271 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.282 7f1fe96c0700 1 -- 172.21.15.102:0/2227382007 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fe41bb7a0 con 0x7f1fe41a6540 2024-04-17T22:31:31.271 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.282 7f1fdb7fe700 1 -- 172.21.15.102:0/2227382007 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f1fd4011bb0 con 0x7f1fe41a6540 2024-04-17T22:31:31.271 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.282 7f1fe96c0700 1 -- 172.21.15.102:0/2227382007 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fe41bbb80 con 0x7f1fe41a6540 2024-04-17T22:31:31.271 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.282 7f1fdb7fe700 1 -- 172.21.15.102:0/2227382007 <== mon.2 v2:172.21.15.102:3302/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7f1fd4027460 con 0x7f1fe41a6540 2024-04-17T22:31:31.271 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.282 7f1fdb7fe700 1 -- 172.21.15.102:0/2227382007 <== mon.2 v2:172.21.15.102:3302/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 3194+0+0 (secure 0 0 0) 0x7f1fd4021180 con 0x7f1fe41a6540 2024-04-17T22:31:31.271 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.283 7f1fe96c0700 1 -- 172.21.15.102:0/2227382007 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1fe40f0820 con 0x7f1fe41a6540 2024-04-17T22:31:31.272 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.286 7f1fdb7fe700 1 -- 172.21.15.102:0/2227382007 <== mon.2 v2:172.21.15.102:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+110455 (secure 0 0 0) 0x7f1fd40115d0 con 0x7f1fe41a6540 2024-04-17T22:31:31.373 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.387 7f1fe96c0700 1 -- 172.21.15.102:0/2227382007 --> v1:172.21.15.102:6816/42360 -- command(tid 0: {"prefix": "pg dump", "target": ["mgr", ""], "format": "json"}) v1 -- 0x7f1fe408a300 con 0x7f1fc80145a0 2024-04-17T22:31:31.380 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.395 7f1fdb7fe700 1 -- 172.21.15.102:0/2227382007 <== mgr.4104 v1:172.21.15.102:6816/42360 1 ==== command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311223 (unknown 2965378022 0 1077297877) 0x7f1fcc001580 con 0x7f1fc80145a0 2024-04-17T22:31:31.385 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-17T22:31:31.387 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.397 7f1fe96c0700 1 -- 172.21.15.102:0/2227382007 >> v1:172.21.15.102:6816/42360 conn(0x7f1fc80145a0 legacy=0x7f1fc8016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:31.388 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.397 7f1fe96c0700 1 -- 172.21.15.102:0/2227382007 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f1fe41a6540 msgr2=0x7f1fe40d6450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:31.388 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.397 7f1fe96c0700 1 --2- 172.21.15.102:0/2227382007 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f1fe41a6540 0x7f1fe40d6450 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 rx=0x7f1fe40d7690 tx=0x7f1fd40068c0).stop 2024-04-17T22:31:31.388 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.397 7f1fe96c0700 1 -- 172.21.15.102:0/2227382007 shutdown_connections 2024-04-17T22:31:31.388 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.397 7f1fe96c0700 1 --2- 172.21.15.102:0/2227382007 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f1fe41b48f0 0x7f1fe41b4d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:31.388 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.397 7f1fe96c0700 1 --2- 172.21.15.102:0/2227382007 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f1fe41a6540 0x7f1fe40d6450 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:31.388 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.397 7f1fe96c0700 1 --2- 172.21.15.102:0/2227382007 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1fe40d6a80 0x7f1fe41b42c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:31.388 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.397 7f1fe96c0700 1 -- 172.21.15.102:0/2227382007 >> 172.21.15.102:0/2227382007 conn(0x7f1fe40c6a60 msgr2=0x7f1fe40cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:31.388 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.397 7f1fe96c0700 1 -- 172.21.15.102:0/2227382007 shutdown_connections 2024-04-17T22:31:31.388 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.397 7f1fe96c0700 1 -- 172.21.15.102:0/2227382007 wait complete. 2024-04-17T22:31:31.388 INFO:teuthology.orchestra.run.smithi102.stderr:dumped all 2024-04-17T22:31:31.394 INFO:teuthology.orchestra.run.smithi102.stdout:{"pg_ready":true,"pg_map":{"version":100,"stamp":"2024-04-17 22:31:31.219680","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2286,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":272,"acting":272,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":249,"num_osds":4,"num_per_pool_osds":4,"kb":377487360,"kb_used":4195272,"kb_used_data":928,"kb_used_omap":0,"kb_used_meta":4194304,"kb_avail":373292088,"statfs":{"total":386547056640,"available":382251098112,"internally_reserved":4294967296,"allocated":950272,"data_stored":284868,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4294967296},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,4,4,4],"upper_bound":32},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-4007,"num_objects":-22,"num_object_clones":0,"num_object_copies":-44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-51,"num_write_kb":-19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.398188"},"pg_stats":[{"pgid":"5.3e","version":"0'0","reported_seq":"9","reported_epoch":"24","state":"active+clean","last_fresh":"2024-04-17 22:31:28.248894","last_change":"2024-04-17 22:31:28.240382","last_active":"2024-04-17 22:31:28.248894","last_peered":"2024-04-17 22:31:28.248894","last_clean":"2024-04-17 22:31:28.248894","last_became_active":"2024-04-17 22:31:28.240063","last_became_peered":"2024-04-17 22:31:28.240063","last_unstale":"2024-04-17 22:31:28.248894","last_undegraded":"2024-04-17 22:31:28.248894","last_fullsized":"2024-04-17 22:31:28.248894","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:27.861194","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:27.861194","last_clean_scrub_stamp":"2024-04-17 22:31:27.861194","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3f","version":"24'3","reported_seq":"13","reported_epoch":"24","state":"active+clean","last_fresh":"2024-04-17 22:31:28.244406","last_change":"2024-04-17 22:31:27.892292","last_active":"2024-04-17 22:31:28.244406","last_peered":"2024-04-17 22:31:28.244406","last_clean":"2024-04-17 22:31:28.244406","last_became_active":"2024-04-17 22:31:27.891945","last_became_peered":"2024-04-17 22:31:27.891945","last_unstale":"2024-04-17 22:31:28.244406","last_undegraded":"2024-04-17 22:31:28.244406","last_fullsized":"2024-04-17 22:31:28.244406","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:26.859642","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:26.859642","last_clean_scrub_stamp":"2024-04-17 22:31:26.859642","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":544,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3f","version":"0'0","reported_seq":"9","reported_epoch":"24","state":"active+clean","last_fresh":"2024-04-17 22:31:28.236538","last_change":"2024-04-17 22:31:28.219218","last_active":"2024-04-17 22:31:28.236538","last_peered":"2024-04-17 22:31:28.236538","last_clean":"2024-04-17 22:31:28.236538","last_became_active":"2024-04-17 22:31:28.218896","last_became_peered":"2024-04-17 22:31:28.218896","last_unstale":"2024-04-17 22:31:28.236538","last_undegraded":"2024-04-17 22:31:28.236538","last_fullsized":"2024-04-17 22:31:28.236538","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:27.861194","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:27.861194","last_clean_scrub_stamp":"2024-04-17 22:31:27.861194","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":"10","reported_epoch":"24","state":"active+clean","last_fresh":"2024-04-17 22:31:28.216660","last_change":"2024-04-17 22:31:27.893734","last_active":"2024-04-17 22:31:28.216660","last_peered":"2024-04-17 22:31:28.216660","last_clean":"2024-04-17 22:31:28.216660","last_became_active":"2024-04-17 22:31:27.893413","last_became_peered":"2024-04-17 22:31:27.893413","last_unstale":"2024-04-17 22:31:28.216660","last_undegraded":"2024-04-17 22:31:28.216660","last_fullsized":"2024-04-17 22:31:28.216660","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:26.859642","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:26.859642","last_clean_scrub_stamp":"2024-04-17 22:31:26.859642","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":"9","reported_epoch":"24","state":"active+clean","last_fresh":"2024-04-17 22:31:28.229957","last_change":"2024-04-17 22:31:28.223880","last_active":"2024-04-17 22:31:28.229957","last_peered":"2024-04-17 22:31:28.229957","last_clean":"2024-04-17 22:31:28.229957","last_became_active":"2024-04-17 22:31:28.223624","last_became_peered":"2024-04-17 22:31:28.223624","last_unstale":"2024-04-17 22:31:28.229957","last_undegraded":"2024-04-17 22:31:28.229957","last_fullsized":"2024-04-17 22:31:28.229957","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:27.861194","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:27.861194","last_clean_scrub_stamp":"2024-04-17 22:31:27.861194","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":"10","reported_epoch":"24","state":"active+clean","last_fresh":"2024-04-17 22:31:28.215210","last_change":"2024-04-17 22:31:27.892907","last_active":"2024-04-17 22:31:28.215210","last_peered":"2024-04-17 22:31:28.215210","last_clean":"2024-04-17 22:31:28.215210","last_became_active":"2024-04-17 22:31:27.892599","last_became_peered":"2024-04-17 22:31:27.892599","last_unstale":"2024-04-17 22:31:28.215210","last_undegraded":"2024-04-17 22:31:28.215210","last_fullsized":"2024-04-17 22:31:28.215210","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:26.859642","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:26.859642","last_clean_scrub_stamp":"2024-04-17 22:31:26.859642","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":"9","reported_epoch":"24","state":"active+clean","last_fresh":"2024-04-17 22:31:28.245086","last_change":"2024-04-17 22:31:28.239988","last_active":"2024-04-17 22:31:28.245086","last_peered":"2024-04-17 22:31:28.245086","last_clean":"2024-04-17 22:31:28.245086","last_became_active":"2024-04-17 22:31:28.239592","last_became_peered":"2024-04-17 22:31:28.239592","last_unstale":"2024-04-17 22:31:28.245086","last_undegraded":"2024-04-17 22:31:28.245086","last_fullsized":"2024-04-17 22:31:28.245086","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:27.861194","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:27.861194","last_clean_scrub_stamp":"2024-04-17 22:31:27.861194","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3c","version":"24'2","reported_seq":"12","reported_epoch":"24","state":"active+clean","last_fresh":"2024-04-17 22:31:28.228496","last_change":"2024-04-17 22:31:27.890933","last_active":"2024-04-17 22:31:28.228496","last_peered":"2024-04-17 22:31:28.228496","last_clean":"2024-04-17 22:31:28.228496","last_became_active":"2024-04-17 22:31:27.890571","last_became_peered":"2024-04-17 22:31:27.890571","last_unstale":"2024-04-17 22:31:28.228496","last_undegraded":"2024-04-17 22:31:28.228496","last_fullsized":"2024-04-17 22:31:28.228496","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:26.859642","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:26.859642","last_clean_scrub_stamp":"2024-04-17 22:31:26.859642","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":"9","reported_epoch":"24","state":"active+clean","last_fresh":"2024-04-17 22:31:28.248395","last_change":"2024-04-17 22:31:28.239981","last_active":"2024-04-17 22:31:28.248395","last_peered":"2024-04-17 22:31:28.248395","last_clean":"2024-04-17 22:31:28.248395","last_became_active":"2024-04-17 22:31:28.239680","last_became_peered":"2024-04-17 22:31:28.239680","last_unstale":"2024-04-17 22:31:28.248395","last_undegraded":"2024-04-17 22:31:28.248395","last_fullsized":"2024-04-17 22:31:28.248395","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:27.861194","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:27.861194","last_clean_scrub_stamp":"2024-04-17 22:31:27.861194","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3b","version":"24'1","reported_seq":"11","reported_epoch":"24","state":"active+clean","last_fresh":"2024-04-17 22:31:28.244633","last_change":"2024-04-17 22:31:27.906353","last_active":"2024-04-17 22:31:28.244633","last_peered":"2024-04-17 22:31:28.244633","last_clean":"2024-04-17 22:31:28.244633","last_became_active":"2024-04-17 22:31:27.906038","last_became_peered":"2024-04-17 22:31:27.906038","last_unstale":"2024-04-17 22:31:28.244633","last_undegraded":"2024-04-17 22:31:28.244633","last_fullsized":"2024-04-17 22:31:28.244633","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:26.859642","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:26.859642","last_clean_scrub_stamp":"2024-04-17 22:31:26.859642","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":"9","reported_epoch":"24","state":"active+clean","last_fresh":"2024-04-17 22:31:28.241612","last_change":"2024-04-17 22:31:28.230260","last_active":"2024-04-17 22:31:28.241612","last_peered":"2024-04-17 22:31:28.241612","last_clean":"2024-04-17 22:31:28.241612","last_became_active":"2024-04-17 22:31:28.229940","last_became_peered":"2024-04-17 22:31:28.229940","last_unstale":"2024-04-17 22:31:28.241612","last_undegraded":"2024-04-17 22:31:28.241612","last_fullsized":"2024-04-17 22:31:28.241612","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:27.861194","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:27.861194","last_clean_scrub_stamp":"2024-04-17 22:31:27.861194","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3a","version":"24'1","reported_seq":"11","reported_epoch":"24","state":"active+clean","last_fresh":"2024-04-17 22:31:28.242570","last_change":"2024-04-17 22:31:27.895105","last_active":"2024-04-17 22:31:28.242570","last_peered":"2024-04-17 22:31:28.242570","last_clean":"2024-04-17 22:31:28.242570","last_became_active":"2024-04-17 22:31:27.894795","last_became_peered":"2024-04-17 22:31:27.894795","last_unstale":"2024-04-17 22:31:28.242570","last_undegraded":"2024-04-17 22:31:28.242570","last_fullsized":"2024-04-17 22:31:28.242570","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:26.859642","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:26.859642","last_clean_scrub_stamp":"2024-04-17 22:31:26.859642","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":"9","reported_epoch":"24","state":"active+clean","last_fresh":"2024-04-17 22:31:28.242303","last_change":"2024-04-17 22:31:28.233783","last_active":"2024-04-17 22:31:28.242303","last_peered":"2024-04-17 22:31:28.242303","last_clean":"2024-04-17 22:31:28.242303","last_became_active":"2024-04-17 22:31:28.230971","last_became_peered":"2024-04-17 22:31:28.230971","last_unstale":"2024-04-17 22:31:28.242303","last_undegraded":"2024-04-17 22:31:28.242303","last_fullsized":"2024-04-17 22:31:28.242303","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:27.861194","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:27.861194","last_clean_scrub_stamp":"2024-04-17 22:31:27.861194","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":"10","reported_epoch":"24","state":"active+clean","last_fresh":"2024-04-17 22:31:28.212469","last_change":"2024-04-17 22:31:27.903787","last_active":"2024-04-17 22:31:28.212469","last_peered":"2024-04-17 22:31:28.212469","last_clean":"2024-04-17 22:31:28.212469","last_became_active":"2024-04-17 22:31:27.903355","last_became_peered":"2024-04-17 22:31:27.903355","last_unstale":"2024-04-17 22:31:28.212469","last_undegraded":"2024-04-17 22:31:28.212469","last_fullsized":"2024-04-17 22:31:28.212469","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:26.859642","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:26.859642","last_clean_scrub_stamp":"2024-04-17 22:31:26.859642","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":"9","reported_epoch":"24","state":"active+clean","last_fresh":"2024-04-17 22:31:28.242379","last_change":"2024-04-17 22:31:28.234920","last_active":"2024-04-17 22:31:28.242379","last_peered":"2024-04-17 22:31:28.242379","last_clean":"2024-04-17 22:31:28.242379","last_became_active":"2024-04-17 22:31:28.233052","last_became_peered":"2024-04-17 22:31:28.233052","last_unstale":"2024-04-17 22:31:28.242379","last_undegraded":"2024-04-17 22:31:28.242379","last_fullsized":"2024-04-17 22:31:28.242379","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:27.861194","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:27.861194","last_clean_scrub_stamp":"2024-04-17 22:31:27.861194","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":"10","reported_epoch":"24","state":"active+clean","last_fresh":"2024-04-17 22:31:28.220881","last_change":"2024-04-17 22:31:27.900038","last_active":"2024-04-17 22:31:28.220881","last_peered":"2024-04-17 22:31:28.220881","last_clean":"2024-04-17 22:31:28.220881","last_became_active":"2024-04-17 22:31:27.899388","last_became_peered":"2024-04-17 22:31:27.899388","last_unstale":"2024-04-17 22:31:28.220881","last_undegraded":"2024-04-17 22:31:28.220881","last_fullsized":"2024-04-17 22:31:28.220881","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:26.859642","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:26.859642","last_clean_scrub_stamp":"2024-04-17 22:31:26.859642","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":"9","reported_epoch":"24","state":"active+clean","last_fresh":"2024-04-17 22:31:28.232605","last_change":"2024-04-17 22:31:28.227545","last_active":"2024-04-17 22:31:28.232605","last_peered":"2024-04-17 22:31:28.232605","last_clean":"2024-04-17 22:31:28.232605","last_became_active":"2024-04-17 22:31:28.224979","last_became_peered":"2024-04-17 22:31:28.224979","last_unstale":"2024-04-17 22:31:28.232605","last_undegraded":"2024-04-17 22:31:28.232605","last_fullsized":"2024-04-17 22:31:28.232605","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:27.861194","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:27.861194","last_clean_scrub_stamp":"2024-04-17 22:31:27.861194","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":"10","reported_epoch":"24","state":"active+clean","last_fresh":"2024-04-17 22:31:28.215255","last_change":"2024-04-17 22:31:27.905889","last_active":"2024-04-17 22:31:28.215255","last_peered":"2024-04-17 22:31:28.215255","last_clean":"2024-04-17 22:31:28.215255","last_became_active":"2024-04-17 22:31:27.905363","last_became_peered":"2024-04-17 22:31:27.905363","last_unstale":"2024-04-17 22:31:28.215255","last_undegraded":"2024-04-17 22:31:28.215255","last_fullsized":"2024-04-17 22:31:28.215255","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:26.859642","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:26.859642","last_clean_scrub_stamp":"2024-04-17 22:31:26.859642","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":"9","reported_epoch":"24","state":"active+clean","last_fresh":"2024-04-17 22:31:28.237878","last_change":"2024-04-17 22:31:28.234681","last_active":"2024-04-17 22:31:28.237878","last_peered":"2024-04-17 22:31:28.237878","last_clean":"2024-04-17 22:31:28.237878","last_became_active":"2024-04-17 22:31:28.233297","last_became_peered":"2024-04-17 22:31:28.233297","last_unstale":"2024-04-17 22:31:28.237878","last_undegraded":"2024-04-17 22:31:28.237878","last_fullsized":"2024-04-17 22:31:28.237878","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:27.861194","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:27.861194","last_clean_scrub_stamp":"2024-04-17 22:31:27.861194","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":"10","reported_epoch":"24","state":"active+clean","last_fresh":"2024-04-17 22:31:28.220088","last_change":"2024-04-17 22:31:27.893600","last_active":"2024-04-17 22:31:28.220088","last_peered":"2024-04-17 22:31:28.220088","last_clean":"2024-04-17 22:31:28.220088","last_became_active":"2024-04-17 22:31:27.893041","last_became_peered":"2024-04-17 22:31:27.893041","last_unstale":"2024-04-17 22:31:28.220088","last_undegraded":"2024-04-17 22:31:28.220088","last_fullsized":"2024-04-17 22:31:28.220088","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:26.859642","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:26.859642","last_clean_scrub_stamp":"2024-04-17 22:31:26.859642","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":"9","reported_epoch":"24","state":"active+clean","last_fresh":"2024-04-17 22:31:28.247044","last_change":"2024-04-17 22:31:28.244433","last_active":"2024-04-17 22:31:28.247044","last_peered":"2024-04-17 22:31:28.247044","last_clean":"2024-04-17 22:31:28.247044","last_became_active":"2024-04-17 22:31:28.240254","last_became_peered":"2024-04-17 22:31:28.240254","last_unstale":"2024-04-17 22:31:28.247044","last_undegraded":"2024-04-17 22:31:28.247044","last_fullsized":"2024-04-17 22:31:28.247044","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:27.861194","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:27.861194","last_clean_scrub_stamp":"2024-04-17 22:31:27.861194","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":"10","reported_epoch":"24","state":"active+clean","last_fresh":"2024-04-17 22:31:28.221537","last_change":"2024-04-17 22:31:27.897998","last_active":"2024-04-17 22:31:28.221537","last_peered":"2024-04-17 22:31:28.221537","last_clean":"2024-04-17 22:31:28.221537","last_became_active":"2024-04-17 22:31:27.897639","last_became_peered":"2024-04-17 22:31:27.897639","last_unstale":"2024-04-17 22:31:28.221537","last_undegraded":"2024-04-17 22:31:28.221537","last_fullsized":"2024-04-17 22:31:28.221537","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:26.859642","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:26.859642","last_clean_scrub_stamp":"2024-04-17 22:31:26.859642","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":"9","reported_epoch":"24","state":"active+clean","last_fresh":"2024-04-17 22:31:28.246919","last_change":"2024-04-17 22:31:28.239190","last_active":"2024-04-17 22:31:28.246919","last_peered":"2024-04-17 22:31:28.246919","last_clean":"2024-04-17 22:31:28.246919","last_became_active":"2024-04-17 22:31:28.238932","last_became_peered":"2024-04-17 22:31:28.238932","last_unstale":"2024-04-17 22:31:28.246919","last_undegraded":"2024-04-17 22:31:28.246919","last_fullsized":"2024-04-17 22:31:28.246919","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:27.861194","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:27.861194","last_clean_scrub_stamp":"2024-04-17 22:31:27.861194","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.34","version":"24'4","reported_seq":"14","reported_epoch":"24","state":"active+clean","last_fresh":"2024-04-17 22:31:28.250273","last_change":"2024-04-17 22:31:27.890064","last_active":"2024-04-17 22:31:28.250273","last_peered":"2024-04-17 22:31:28.250273","last_clean":"2024-04-17 22:31:28.250273","last_became_active":"2024-04-17 22:31:27.889734","last_became_peered":"2024-04-17 22:31:27.889734","last_unstale":"2024-04-17 22:31:28.250273","last_undegraded":"2024-04-17 22:31:28.250273","last_fullsized":"2024-04-17 22:31:28.250273","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:26.859642","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:26.859642","last_clean_scrub_stamp":"2024-04-17 22:31:26.859642","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":892,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":"9","reported_epoch":"24","state":"active+clean","last_fresh":"2024-04-17 22:31:28.248618","last_change":"2024-04-17 22:31:28.240509","last_active":"2024-04-17 22:31:28.248618","last_peered":"2024-04-17 22:31:28.248618","last_clean":"2024-04-17 22:31:28.248618","last_became_active":"2024-04-17 22:31:28.239870","last_became_peered":"2024-04-17 22:31:28.239870","last_unstale":"2024-04-17 22:31:28.248618","last_undegraded":"2024-04-17 22:31:28.248618","last_fullsized":"2024-04-17 22:31:28.248618","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:27.861194","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:27.861194","last_clean_scrub_stamp":"2024-04-17 22:31:27.861194","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.33","version":"24'1","reported_seq":"11","reported_epoch":"24","state":"active+clean","last_fresh":"2024-04-17 22:31:28.244832","last_change":"2024-04-17 22:31:27.896276","last_active":"2024-04-17 22:31:28.244832","last_peered":"2024-04-17 22:31:28.244832","last_clean":"2024-04-17 22:31:28.244832","last_became_active":"2024-04-17 22:31:27.895637","last_became_peered":"2024-04-17 22:31:27.895637","last_unstale":"2024-04-17 22:31:28.244832","last_undegraded":"2024-04-17 22:31:28.244832","last_fullsized":"2024-04-17 22:31:28.244832","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:26.859642","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:26.859642","last_clean_scrub_stamp":"2024-04-17 22:31:26.859642","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":"9","reported_epoch":"24","state":"active+clean","last_fresh":"2024-04-17 22:31:28.241589","last_change":"2024-04-17 22:31:28.226759","last_active":"2024-04-17 22:31:28.241589","last_peered":"2024-04-17 22:31:28.241589","last_clean":"2024-04-17 22:31:28.241589","last_became_active":"2024-04-17 22:31:28.226426","last_became_peered":"2024-04-17 22:31:28.226426","last_unstale":"2024-04-17 22:31:28.241589","last_undegraded":"2024-04-17 22:31:28.241589","last_fullsized":"2024-04-17 22:31:28.241589","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:27.861194","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:27.861194","last_clean_scrub_stamp":"2024-04-17 22:31:27.861194","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":"10","reported_epoch":"24","state":"active+clean","last_fresh":"2024-04-17 22:31:28.219766","last_change":"2024-04-17 22:31:27.900866","last_active":"2024-04-17 22:31:28.219766","last_peered":"2024-04-17 22:31:28.219766","last_clean":"2024-04-17 22:31:28.219766","last_became_active":"2024-04-17 22:31:27.900037","last_became_peered":"2024-04-17 22:31:27.900037","last_unstale":"2024-04-17 22:31:28.219766","last_undegraded":"2024-04-17 22:31:28.219766","last_fullsized":"2024-04-17 22:31:28.219766","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:26.859642","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:26.859642","last_clean_scrub_stamp":"2024-04-17 22:31:26.859642","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":"9","reported_epoch":"24","state":"active+clean","last_fresh":"2024-04-17 22:31:28.251363","last_change":"2024-04-17 22:31:28.249454","last_active":"2024-04-17 22:31:28.251363","last_peered":"2024-04-17 22:31:28.251363","last_clean":"2024-04-17 22:31:28.251363","last_became_active":"2024-04-17 22:31:28.249125","last_became_peered":"2024-04-17 22:31:28.249125","last_unstale":"2024-04-17 22:31:28.251363","last_undegraded":"2024-04-17 22:31:28.251363","last_fullsized":"2024-04-17 22:31:28.251363","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:27.861194","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:27.861194","last_clean_scrub_stamp":"2024-04-17 22:31:27.861194","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":"10","reported_epoch":"24","state":"active+clean","last_fresh":"2024-04-17 22:31:28.214060","last_change":"2024-04-17 22:31:27.887895","last_active":"2024-04-17 22:31:28.214060","last_peered":"2024-04-17 22:31:28.214060","last_clean":"2024-04-17 22:31:28.214060","last_became_active":"2024-04-17 22:31:27.887536","last_became_peered":"2024-04-17 22:31:27.887536","last_unstale":"2024-04-17 22:31:28.214060","last_undegraded":"2024-04-17 22:31:28.214060","last_fullsized":"2024-04-17 22:31:28.214060","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:26.859642","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:26.859642","last_clean_scrub_stamp":"2024-04-17 22:31:26.859642","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":"9","reported_epoch":"24","state":"active+clean","last_fresh":"2024-04-17 22:31:28.246480","last_change":"2024-04-17 22:31:28.238205","last_active":"2024-04-17 22:31:28.246480","last_peered":"2024-04-17 22:31:28.246480","last_clean":"2024-04-17 22:31:28.246480","last_became_active":"2024-04-17 22:31:28.236581","last_became_peered":"2024-04-17 22:31:28.236581","last_unstale":"2024-04-17 22:31:28.246480","last_undegraded":"2024-04-17 22:31:28.246480","last_fullsized":"2024-04-17 22:31:28.246480","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:27.861194","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:27.861194","last_clean_scrub_stamp":"2024-04-17 22:31:27.861194","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.30","version":"24'4","reported_seq":"14","reported_epoch":"24","state":"active+clean","last_fresh":"2024-04-17 22:31:28.249903","last_change":"2024-04-17 22:31:27.903283","last_active":"2024-04-17 22:31:28.249903","last_peered":"2024-04-17 22:31:28.249903","last_clean":"2024-04-17 22:31:28.249903","last_became_active":"2024-04-17 22:31:27.902939","last_became_peered":"2024-04-17 22:31:27.902939","last_unstale":"2024-04-17 22:31:28.249903","last_undegraded":"2024-04-17 22:31:28.249903","last_fullsized":"2024-04-17 22:31:28.249903","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:26.859642","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:26.859642","last_clean_scrub_stamp":"2024-04-17 22:31:26.859642","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":"9","reported_epoch":"24","state":"active+clean","last_fresh":"2024-04-17 22:31:28.249795","last_change":"2024-04-17 22:31:28.246619","last_active":"2024-04-17 22:31:28.249795","last_peered":"2024-04-17 22:31:28.249795","last_clean":"2024-04-17 22:31:28.249795","last_became_active":"2024-04-17 22:31:28.245921","last_became_peered":"2024-04-17 22:31:28.245921","last_unstale":"2024-04-17 22:31:28.249795","last_undegraded":"2024-04-17 22:31:28.249795","last_fullsized":"2024-04-17 22:31:28.249795","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:27.861194","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:27.861194","last_clean_scrub_stamp":"2024-04-17 22:31:27.861194","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":"10","reported_epoch":"24","state":"active+clean","last_fresh":"2024-04-17 22:31:28.215918","last_change":"2024-04-17 22:31:27.905691","last_active":"2024-04-17 22:31:28.215918","last_peered":"2024-04-17 22:31:28.215918","last_clean":"2024-04-17 22:31:28.215918","last_became_active":"2024-04-17 22:31:27.905102","last_became_peered":"2024-04-17 22:31:27.905102","last_unstale":"2024-04-17 22:31:28.215918","last_undegraded":"2024-04-17 22:31:28.215918","last_fullsized":"2024-04-17 22:31:28.215918","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:26.859642","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:26.859642","last_clean_scrub_stamp":"2024-04-17 22:31:26.859642","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":"9","reported_epoch":"24","state":"active+clean","last_fresh":"2024-04-17 22:31:28.249062","last_change":"2024-04-17 22:31:28.243697","last_active":"2024-04-17 22:31:28.249062","last_peered":"2024-04-17 22:31:28.249062","last_clean":"2024-04-17 22:31:28.249062","last_became_active":"2024-04-17 22:31:28.243400","last_became_peered":"2024-04-17 22:31:28.243400","last_unstale":"2024-04-17 22:31:28.249062","last_undegraded":"2024-04-17 22:31:28.249062","last_fullsized":"2024-04-17 22:31:28.249062","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:27.861194","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:27.861194","last_clean_scrub_stamp":"2024-04-17 22:31:27.861194","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":"10","reported_epoch":"24","state":"active+clean","last_fresh":"2024-04-17 22:31:28.214621","last_change":"2024-04-17 22:31:27.894928","last_active":"2024-04-17 22:31:28.214621","last_peered":"2024-04-17 22:31:28.214621","last_clean":"2024-04-17 22:31:28.214621","last_became_active":"2024-04-17 22:31:27.894641","last_became_peered":"2024-04-17 22:31:27.894641","last_unstale":"2024-04-17 22:31:28.214621","last_undegraded":"2024-04-17 22:31:28.214621","last_fullsized":"2024-04-17 22:31:28.214621","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:26.859642","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:26.859642","last_clean_scrub_stamp":"2024-04-17 22:31:26.859642","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":"9","reported_epoch":"24","state":"active+clean","last_fresh":"2024-04-17 22:31:28.239345","last_change":"2024-04-17 22:31:28.226646","last_active":"2024-04-17 22:31:28.239345","last_peered":"2024-04-17 22:31:28.239345","last_clean":"2024-04-17 22:31:28.239345","last_became_active":"2024-04-17 22:31:28.226280","last_became_peered":"2024-04-17 22:31:28.226280","last_unstale":"2024-04-17 22:31:28.239345","last_undegraded":"2024-04-17 22:31:28.239345","last_fullsized":"2024-04-17 22:31:28.239345","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:27.861194","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:27.861194","last_clean_scrub_stamp":"2024-04-17 22:31:27.861194","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2d","version":"24'1","reported_seq":"11","reported_epoch":"24","state":"active+clean","last_fresh":"2024-04-17 22:31:28.248628","last_change":"2024-04-17 22:31:27.897246","last_active":"2024-04-17 22:31:28.248628","last_peered":"2024-04-17 22:31:28.248628","last_clean":"2024-04-17 22:31:28.248628","last_became_active":"2024-04-17 22:31:27.896919","last_became_peered":"2024-04-17 22:31:27.896919","last_unstale":"2024-04-17 22:31:28.248628","last_undegraded":"2024-04-17 22:31:28.248628","last_fullsized":"2024-04-17 22:31:28.248628","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:26.859642","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:26.859642","last_clean_scrub_stamp":"2024-04-17 22:31:26.859642","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":"9","reported_epoch":"24","state":"active+clean","last_fresh":"2024-04-17 22:31:28.246940","last_change":"2024-04-17 22:31:28.239259","last_active":"2024-04-17 22:31:28.246940","last_peered":"2024-04-17 22:31:28.246940","last_clean":"2024-04-17 22:31:28.246940","last_became_active":"2024-04-17 22:31:28.238955","last_became_peered":"2024-04-17 22:31:28.238955","last_unstale":"2024-04-17 22:31:28.246940","last_undegraded":"2024-04-17 22:31:28.246940","last_fullsized":"2024-04-17 22:31:28.246940","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:27.861194","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:27.861194","last_clean_scrub_stamp":"2024-04-17 22:31:27.861194","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":"10","reported_epoch":"24","state":"active+clean","last_fresh":"2024-04-17 22:31:28.217933","last_change":"2024-04-17 22:31:27.901754","last_active":"2024-04-17 22:31:28.217933","last_peered":"2024-04-17 22:31:28.217933","last_clean":"2024-04-17 22:31:28.217933","last_became_active":"2024-04-17 22:31:27.901472","last_became_peered":"2024-04-17 22:31:27.901472","last_unstale":"2024-04-17 22:31:28.217933","last_undegraded":"2024-04-17 22:31:28.217933","last_fullsized":"2024-04-17 22:31:28.217933","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:26.859642","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:26.859642","last_clean_scrub_stamp":"2024-04-17 22:31:26.859642","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":"9","reported_epoch":"24","state":"active+clean","last_fresh":"2024-04-17 22:31:28.240970","last_change":"2024-04-17 22:31:28.230870","last_active":"2024-04-17 22:31:28.240970","last_peered":"2024-04-17 22:31:28.240970","last_clean":"2024-04-17 22:31:28.240970","last_became_active":"2024-04-17 22:31:28.230433","last_became_peered":"2024-04-17 22:31:28.230433","last_unstale":"2024-04-17 22:31:28.240970","last_undegraded":"2024-04-17 22:31:28.240970","last_fullsized":"2024-04-17 22:31:28.240970","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:27.861194","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:27.861194","last_clean_scrub_stamp":"2024-04-17 22:31:27.861194","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2b","version":"24'1","reported_seq":"11","reported_epoch":"24","state":"active+clean","last_fresh":"2024-04-17 22:31:28.245714","last_change":"2024-04-17 22:31:27.900151","last_active":"2024-04-17 22:31:28.245714","last_peered":"2024-04-17 22:31:28.245714","last_clean":"2024-04-17 22:31:28.245714","last_became_active":"2024-04-17 22:31:27.899674","last_became_peered":"2024-04-17 22:31:27.899674","last_unstale":"2024-04-17 22:31:28.245714","last_undegraded":"2024-04-17 22:31:28.245714","last_fullsized":"2024-04-17 22:31:28.245714","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:26.859642","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:26.859642","last_clean_scrub_stamp":"2024-04-17 22:31:26.859642","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":"9","reported_epoch":"24","state":"active+clean","last_fresh":"2024-04-17 22:31:28.246739","last_change":"2024-04-17 22:31:28.236947","last_active":"2024-04-17 22:31:28.246739","last_peered":"2024-04-17 22:31:28.246739","last_clean":"2024-04-17 22:31:28.246739","last_became_active":"2024-04-17 22:31:28.236625","last_became_peered":"2024-04-17 22:31:28.236625","last_unstale":"2024-04-17 22:31:28.246739","last_undegraded":"2024-04-17 22:31:28.246739","last_fullsized":"2024-04-17 22:31:28.246739","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:27.861194","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:27.861194","last_clean_scrub_stamp":"2024-04-17 22:31:27.861194","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":"10","reported_epoch":"24","state":"active+clean","last_fresh":"2024-04-17 22:31:28.214271","last_change":"2024-04-17 22:31:27.890918","last_active":"2024-04-17 22:31:28.214271","last_peered":"2024-04-17 22:31:28.214271","last_clean":"2024-04-17 22:31:28.214271","last_became_active":"2024-04-17 22:31:27.890584","last_became_peered":"2024-04-17 22:31:27.890584","last_unstale":"2024-04-17 22:31:28.214271","last_undegraded":"2024-04-17 22:31:28.214271","last_fullsized":"2024-04-17 22:31:28.214271","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:26.859642","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:26.859642","last_clean_scrub_stamp":"2024-04-17 22:31:26.859642","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":"9","reported_epoch":"24","state":"active+clean","last_fresh":"2024-04-17 22:31:28.244459","last_change":"2024-04-17 22:31:28.242862","last_active":"2024-04-17 22:31:28.244459","last_peered":"2024-04-17 22:31:28.244459","last_clean":"2024-04-17 22:31:28.244459","last_became_active":"2024-04-17 22:31:28.242552","last_became_peered":"2024-04-17 22:31:28.242552","last_unstale":"2024-04-17 22:31:28.244459","last_undegraded":"2024-04-17 22:31:28.244459","last_fullsized":"2024-04-17 22:31:28.244459","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:27.861194","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:27.861194","last_clean_scrub_stamp":"2024-04-17 22:31:27.861194","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":"10","reported_epoch":"24","state":"active+clean","last_fresh":"2024-04-17 22:31:28.212513","last_change":"2024-04-17 22:31:27.903938","last_active":"2024-04-17 22:31:28.212513","last_peered":"2024-04-17 22:31:28.212513","last_clean":"2024-04-17 22:31:28.212513","last_became_active":"2024-04-17 22:31:27.903441","last_became_peered":"2024-04-17 22:31:27.903441","last_unstale":"2024-04-17 22:31:28.212513","last_undegraded":"2024-04-17 22:31:28.212513","last_fullsized":"2024-04-17 22:31:28.212513","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:26.859642","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:26.859642","last_clean_scrub_stamp":"2024-04-17 22:31:26.859642","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":"9","reported_epoch":"24","state":"active+clean","last_fresh":"2024-04-17 22:31:28.246498","last_change":"2024-04-17 22:31:28.238276","last_active":"2024-04-17 22:31:28.246498","last_peered":"2024-04-17 22:31:28.246498","last_clean":"2024-04-17 22:31:28.246498","last_became_active":"2024-04-17 22:31:28.237034","last_became_peered":"2024-04-17 22:31:28.237034","last_unstale":"2024-04-17 22:31:28.246498","last_undegraded":"2024-04-17 22:31:28.246498","last_fullsized":"2024-04-17 22:31:28.246498","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:27.861194","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:27.861194","last_clean_scrub_stamp":"2024-04-17 22:31:27.861194","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":"10","reported_epoch":"24","state":"active+clean","last_fresh":"2024-04-17 22:31:28.220990","last_change":"2024-04-17 22:31:27.900610","last_active":"2024-04-17 22:31:28.220990","last_peered":"2024-04-17 22:31:28.220990","last_clean":"2024-04-17 22:31:28.220990","last_became_active":"2024-04-17 22:31:27.900297","last_became_peered":"2024-04-17 22:31:27.900297","last_unstale":"2024-04-17 22:31:28.220990","last_undegraded":"2024-04-17 22:31:28.220990","last_fullsized":"2024-04-17 22:31:28.220990","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:26.859642","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:26.859642","last_clean_scrub_stamp":"2024-04-17 22:31:26.859642","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":"9","reported_epoch":"24","state":"active+clean","last_fresh":"2024-04-17 22:31:28.249835","last_change":"2024-04-17 22:31:28.246953","last_active":"2024-04-17 22:31:28.249835","last_peered":"2024-04-17 22:31:28.249835","last_clean":"2024-04-17 22:31:28.249835","last_became_active":"2024-04-17 22:31:28.246346","last_became_peered":"2024-04-17 22:31:28.246346","last_unstale":"2024-04-17 22:31:28.249835","last_undegraded":"2024-04-17 22:31:28.249835","last_fullsized":"2024-04-17 22:31:28.249835","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:27.861194","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:27.861194","last_clean_scrub_stamp":"2024-04-17 22:31:27.861194","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":"10","reported_epoch":"24","state":"active+clean","last_fresh":"2024-04-17 22:31:28.222037","last_change":"2024-04-17 22:31:27.904119","last_active":"2024-04-17 22:31:28.222037","last_peered":"2024-04-17 22:31:28.222037","last_clean":"2024-04-17 22:31:28.222037","last_became_active":"2024-04-17 22:31:27.903788","last_became_peered":"2024-04-17 22:31:27.903788","last_unstale":"2024-04-17 22:31:28.222037","last_undegraded":"2024-04-17 22:31:28.222037","last_fullsized":"2024-04-17 22:31:28.222037","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:26.859642","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:26.859642","last_clean_scrub_stamp":"2024-04-17 22:31:26.859642","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":"9","reported_epoch":"24","state":"active+clean","last_fresh":"2024-04-17 22:31:28.245339","last_change":"2024-04-17 22:31:28.240850","last_active":"2024-04-17 22:31:28.245339","last_peered":"2024-04-17 22:31:28.245339","last_clean":"2024-04-17 22:31:28.245339","last_became_active":"2024-04-17 22:31:28.240560","last_became_peered":"2024-04-17 22:31:28.240560","last_unstale":"2024-04-17 22:31:28.245339","last_undegraded":"2024-04-17 22:31:28.245339","last_fullsized":"2024-04-17 22:31:28.245339","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:27.861194","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:27.861194","last_clean_scrub_stamp":"2024-04-17 22:31:27.861194","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":"10","reported_epoch":"24","state":"active+clean","last_fresh":"2024-04-17 22:31:28.220189","last_change":"2024-04-17 22:31:27.900441","last_active":"2024-04-17 22:31:28.220189","last_peered":"2024-04-17 22:31:28.220189","last_clean":"2024-04-17 22:31:28.220189","last_became_active":"2024-04-17 22:31:27.899323","last_became_peered":"2024-04-17 22:31:27.899323","last_unstale":"2024-04-17 22:31:28.220189","last_undegraded":"2024-04-17 22:31:28.220189","last_fullsized":"2024-04-17 22:31:28.220189","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:26.859642","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:26.859642","last_clean_scrub_stamp":"2024-04-17 22:31:26.859642","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":"9","reported_epoch":"24","state":"active+clean","last_fresh":"2024-04-17 22:31:28.236445","last_change":"2024-04-17 22:31:28.218932","last_active":"2024-04-17 22:31:28.236445","last_peered":"2024-04-17 22:31:28.236445","last_clean":"2024-04-17 22:31:28.236445","last_became_active":"2024-04-17 22:31:28.218619","last_became_peered":"2024-04-17 22:31:28.218619","last_unstale":"2024-04-17 22:31:28.236445","last_undegraded":"2024-04-17 22:31:28.236445","last_fullsized":"2024-04-17 22:31:28.236445","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:27.861194","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:27.861194","last_clean_scrub_stamp":"2024-04-17 22:31:27.861194","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":"10","reported_epoch":"24","state":"active+clean","last_fresh":"2024-04-17 22:31:28.224273","last_change":"2024-04-17 22:31:27.904704","last_active":"2024-04-17 22:31:28.224273","last_peered":"2024-04-17 22:31:28.224273","last_clean":"2024-04-17 22:31:28.224273","last_became_active":"2024-04-17 22:31:27.904488","last_became_peered":"2024-04-17 22:31:27.904488","last_unstale":"2024-04-17 22:31:28.224273","last_undegraded":"2024-04-17 22:31:28.224273","last_fullsized":"2024-04-17 22:31:28.224273","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:26.859642","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:26.859642","last_clean_scrub_stamp":"2024-04-17 22:31:26.859642","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":"9","reported_epoch":"24","state":"active+clean","last_fresh":"2024-04-17 22:31:28.238838","last_change":"2024-04-17 22:31:28.232995","last_active":"2024-04-17 22:31:28.238838","last_peered":"2024-04-17 22:31:28.238838","last_clean":"2024-04-17 22:31:28.238838","last_became_active":"2024-04-17 22:31:28.232458","last_became_peered":"2024-04-17 22:31:28.232458","last_unstale":"2024-04-17 22:31:28.238838","last_undegraded":"2024-04-17 22:31:28.238838","last_fullsized":"2024-04-17 22:31:28.238838","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:27.861194","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:27.861194","last_clean_scrub_stamp":"2024-04-17 22:31:27.861194","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":"10","reported_epoch":"24","state":"active+clean","last_fresh":"2024-04-17 22:31:28.219431","last_change":"2024-04-17 22:31:27.901338","last_active":"2024-04-17 22:31:28.219431","last_peered":"2024-04-17 22:31:28.219431","last_clean":"2024-04-17 22:31:28.219431","last_became_active":"2024-04-17 22:31:27.900883","last_became_peered":"2024-04-17 22:31:27.900883","last_unstale":"2024-04-17 22:31:28.219431","last_undegraded":"2024-04-17 22:31:28.219431","last_fullsized":"2024-04-17 22:31:28.219431","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:26.859642","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:26.859642","last_clean_scrub_stamp":"2024-04-17 22:31:26.859642","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":"9","reported_epoch":"24","state":"active+clean","last_fresh":"2024-04-17 22:31:28.249663","last_change":"2024-04-17 22:31:28.245559","last_active":"2024-04-17 22:31:28.249663","last_peered":"2024-04-17 22:31:28.249663","last_clean":"2024-04-17 22:31:28.249663","last_became_active":"2024-04-17 22:31:28.245278","last_became_peered":"2024-04-17 22:31:28.245278","last_unstale":"2024-04-17 22:31:28.249663","last_undegraded":"2024-04-17 22:31:28.249663","last_fullsized":"2024-04-17 22:31:28.249663","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:27.861194","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:27.861194","last_clean_scrub_stamp":"2024-04-17 22:31:27.861194","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":"10","reported_epoch":"24","state":"active+clean","last_fresh":"2024-04-17 22:31:28.226982","last_change":"2024-04-17 22:31:27.910204","last_active":"2024-04-17 22:31:28.226982","last_peered":"2024-04-17 22:31:28.226982","last_clean":"2024-04-17 22:31:28.226982","last_became_active":"2024-04-17 22:31:27.909533","last_became_peered":"2024-04-17 22:31:27.909533","last_unstale":"2024-04-17 22:31:28.226982","last_undegraded":"2024-04-17 22:31:28.226982","last_fullsized":"2024-04-17 22:31:28.226982","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:26.859642","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:26.859642","last_clean_scrub_stamp":"2024-04-17 22:31:26.859642","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":"9","reported_epoch":"24","state":"active+clean","last_fresh":"2024-04-17 22:31:28.235575","last_change":"2024-04-17 22:31:28.231769","last_active":"2024-04-17 22:31:28.235575","last_peered":"2024-04-17 22:31:28.235575","last_clean":"2024-04-17 22:31:28.235575","last_became_active":"2024-04-17 22:31:28.231455","last_became_peered":"2024-04-17 22:31:28.231455","last_unstale":"2024-04-17 22:31:28.235575","last_undegraded":"2024-04-17 22:31:28.235575","last_fullsized":"2024-04-17 22:31:28.235575","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:27.861194","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:27.861194","last_clean_scrub_stamp":"2024-04-17 22:31:27.861194","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":"10","reported_epoch":"24","state":"active+clean","last_fresh":"2024-04-17 22:31:28.214398","last_change":"2024-04-17 22:31:27.895587","last_active":"2024-04-17 22:31:28.214398","last_peered":"2024-04-17 22:31:28.214398","last_clean":"2024-04-17 22:31:28.214398","last_became_active":"2024-04-17 22:31:27.895251","last_became_peered":"2024-04-17 22:31:27.895251","last_unstale":"2024-04-17 22:31:28.214398","last_undegraded":"2024-04-17 22:31:28.214398","last_fullsized":"2024-04-17 22:31:28.214398","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:26.859642","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:26.859642","last_clean_scrub_stamp":"2024-04-17 22:31:26.859642","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":"9","reported_epoch":"24","state":"active+clean","last_fresh":"2024-04-17 22:31:28.249646","last_change":"2024-04-17 22:31:28.241297","last_active":"2024-04-17 22:31:28.249646","last_peered":"2024-04-17 22:31:28.249646","last_clean":"2024-04-17 22:31:28.249646","last_became_active":"2024-04-17 22:31:28.241026","last_became_peered":"2024-04-17 22:31:28.241026","last_unstale":"2024-04-17 22:31:28.249646","last_undegraded":"2024-04-17 22:31:28.249646","last_fullsized":"2024-04-17 22:31:28.249646","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:27.861194","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:27.861194","last_clean_scrub_stamp":"2024-04-17 22:31:27.861194","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":"10","reported_epoch":"24","state":"active+clean","last_fresh":"2024-04-17 22:31:28.220868","last_change":"2024-04-17 22:31:27.908229","last_active":"2024-04-17 22:31:28.220868","last_peered":"2024-04-17 22:31:28.220868","last_clean":"2024-04-17 22:31:28.220868","last_became_active":"2024-04-17 22:31:27.907856","last_became_peered":"2024-04-17 22:31:27.907856","last_unstale":"2024-04-17 22:31:28.220868","last_undegraded":"2024-04-17 22:31:28.220868","last_fullsized":"2024-04-17 22:31:28.220868","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:26.859642","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:26.859642","last_clean_scrub_stamp":"2024-04-17 22:31:26.859642","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":"9","reported_epoch":"24","state":"active+clean","last_fresh":"2024-04-17 22:31:28.242355","last_change":"2024-04-17 22:31:28.232210","last_active":"2024-04-17 22:31:28.242355","last_peered":"2024-04-17 22:31:28.242355","last_clean":"2024-04-17 22:31:28.242355","last_became_active":"2024-04-17 22:31:28.230691","last_became_peered":"2024-04-17 22:31:28.230691","last_unstale":"2024-04-17 22:31:28.242355","last_undegraded":"2024-04-17 22:31:28.242355","last_fullsized":"2024-04-17 22:31:28.242355","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:27.861194","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:27.861194","last_clean_scrub_stamp":"2024-04-17 22:31:27.861194","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":"10","reported_epoch":"24","state":"active+clean","last_fresh":"2024-04-17 22:31:28.221084","last_change":"2024-04-17 22:31:27.900205","last_active":"2024-04-17 22:31:28.221084","last_peered":"2024-04-17 22:31:28.221084","last_clean":"2024-04-17 22:31:28.221084","last_became_active":"2024-04-17 22:31:27.899611","last_became_peered":"2024-04-17 22:31:27.899611","last_unstale":"2024-04-17 22:31:28.221084","last_undegraded":"2024-04-17 22:31:28.221084","last_fullsized":"2024-04-17 22:31:28.221084","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:26.859642","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:26.859642","last_clean_scrub_stamp":"2024-04-17 22:31:26.859642","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":"9","reported_epoch":"24","state":"active+clean","last_fresh":"2024-04-17 22:31:28.239105","last_change":"2024-04-17 22:31:28.232371","last_active":"2024-04-17 22:31:28.239105","last_peered":"2024-04-17 22:31:28.239105","last_clean":"2024-04-17 22:31:28.239105","last_became_active":"2024-04-17 22:31:28.231777","last_became_peered":"2024-04-17 22:31:28.231777","last_unstale":"2024-04-17 22:31:28.239105","last_undegraded":"2024-04-17 22:31:28.239105","last_fullsized":"2024-04-17 22:31:28.239105","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:27.861194","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:27.861194","last_clean_scrub_stamp":"2024-04-17 22:31:27.861194","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"24'1","reported_seq":"11","reported_epoch":"24","state":"active+clean","last_fresh":"2024-04-17 22:31:28.248535","last_change":"2024-04-17 22:31:27.910061","last_active":"2024-04-17 22:31:28.248535","last_peered":"2024-04-17 22:31:28.248535","last_clean":"2024-04-17 22:31:28.248535","last_became_active":"2024-04-17 22:31:27.909742","last_became_peered":"2024-04-17 22:31:27.909742","last_unstale":"2024-04-17 22:31:28.248535","last_undegraded":"2024-04-17 22:31:28.248535","last_fullsized":"2024-04-17 22:31:28.248535","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:26.859642","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:26.859642","last_clean_scrub_stamp":"2024-04-17 22:31:26.859642","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":"9","reported_epoch":"24","state":"active+clean","last_fresh":"2024-04-17 22:31:28.242434","last_change":"2024-04-17 22:31:28.236008","last_active":"2024-04-17 22:31:28.242434","last_peered":"2024-04-17 22:31:28.242434","last_clean":"2024-04-17 22:31:28.242434","last_became_active":"2024-04-17 22:31:28.235719","last_became_peered":"2024-04-17 22:31:28.235719","last_unstale":"2024-04-17 22:31:28.242434","last_undegraded":"2024-04-17 22:31:28.242434","last_fullsized":"2024-04-17 22:31:28.242434","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:27.861194","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:27.861194","last_clean_scrub_stamp":"2024-04-17 22:31:27.861194","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1e","version":"24'1","reported_seq":"11","reported_epoch":"24","state":"active+clean","last_fresh":"2024-04-17 22:31:28.230812","last_change":"2024-04-17 22:31:27.888087","last_active":"2024-04-17 22:31:28.230812","last_peered":"2024-04-17 22:31:28.230812","last_clean":"2024-04-17 22:31:28.230812","last_became_active":"2024-04-17 22:31:27.887729","last_became_peered":"2024-04-17 22:31:27.887729","last_unstale":"2024-04-17 22:31:28.230812","last_undegraded":"2024-04-17 22:31:28.230812","last_fullsized":"2024-04-17 22:31:28.230812","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:26.859642","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:26.859642","last_clean_scrub_stamp":"2024-04-17 22:31:26.859642","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":"9","reported_epoch":"24","state":"active+clean","last_fresh":"2024-04-17 22:31:28.242241","last_change":"2024-04-17 22:31:28.232043","last_active":"2024-04-17 22:31:28.242241","last_peered":"2024-04-17 22:31:28.242241","last_clean":"2024-04-17 22:31:28.242241","last_became_active":"2024-04-17 22:31:28.231668","last_became_peered":"2024-04-17 22:31:28.231668","last_unstale":"2024-04-17 22:31:28.242241","last_undegraded":"2024-04-17 22:31:28.242241","last_fullsized":"2024-04-17 22:31:28.242241","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:27.861194","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:27.861194","last_clean_scrub_stamp":"2024-04-17 22:31:27.861194","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1d","version":"24'2","reported_seq":"12","reported_epoch":"24","state":"active+clean","last_fresh":"2024-04-17 22:31:28.248913","last_change":"2024-04-17 22:31:27.908248","last_active":"2024-04-17 22:31:28.248913","last_peered":"2024-04-17 22:31:28.248913","last_clean":"2024-04-17 22:31:28.248913","last_became_active":"2024-04-17 22:31:27.907898","last_became_peered":"2024-04-17 22:31:27.907898","last_unstale":"2024-04-17 22:31:28.248913","last_undegraded":"2024-04-17 22:31:28.248913","last_fullsized":"2024-04-17 22:31:28.248913","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:26.859642","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:26.859642","last_clean_scrub_stamp":"2024-04-17 22:31:26.859642","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":"9","reported_epoch":"24","state":"active+clean","last_fresh":"2024-04-17 22:31:28.238856","last_change":"2024-04-17 22:31:28.233157","last_active":"2024-04-17 22:31:28.238856","last_peered":"2024-04-17 22:31:28.238856","last_clean":"2024-04-17 22:31:28.238856","last_became_active":"2024-04-17 22:31:28.232672","last_became_peered":"2024-04-17 22:31:28.232672","last_unstale":"2024-04-17 22:31:28.238856","last_undegraded":"2024-04-17 22:31:28.238856","last_fullsized":"2024-04-17 22:31:28.238856","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:27.861194","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:27.861194","last_clean_scrub_stamp":"2024-04-17 22:31:27.861194","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":"10","reported_epoch":"24","state":"active+clean","last_fresh":"2024-04-17 22:31:28.216490","last_change":"2024-04-17 22:31:27.893879","last_active":"2024-04-17 22:31:28.216490","last_peered":"2024-04-17 22:31:28.216490","last_clean":"2024-04-17 22:31:28.216490","last_became_active":"2024-04-17 22:31:27.893568","last_became_peered":"2024-04-17 22:31:27.893568","last_unstale":"2024-04-17 22:31:28.216490","last_undegraded":"2024-04-17 22:31:28.216490","last_fullsized":"2024-04-17 22:31:28.216490","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:26.859642","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:26.859642","last_clean_scrub_stamp":"2024-04-17 22:31:26.859642","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":"9","reported_epoch":"24","state":"active+clean","last_fresh":"2024-04-17 22:31:28.232735","last_change":"2024-04-17 22:31:28.228313","last_active":"2024-04-17 22:31:28.232735","last_peered":"2024-04-17 22:31:28.232735","last_clean":"2024-04-17 22:31:28.232735","last_became_active":"2024-04-17 22:31:28.228013","last_became_peered":"2024-04-17 22:31:28.228013","last_unstale":"2024-04-17 22:31:28.232735","last_undegraded":"2024-04-17 22:31:28.232735","last_fullsized":"2024-04-17 22:31:28.232735","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:27.861194","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:27.861194","last_clean_scrub_stamp":"2024-04-17 22:31:27.861194","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":"10","reported_epoch":"24","state":"active+clean","last_fresh":"2024-04-17 22:31:28.217045","last_change":"2024-04-17 22:31:27.896399","last_active":"2024-04-17 22:31:28.217045","last_peered":"2024-04-17 22:31:28.217045","last_clean":"2024-04-17 22:31:28.217045","last_became_active":"2024-04-17 22:31:27.895909","last_became_peered":"2024-04-17 22:31:27.895909","last_unstale":"2024-04-17 22:31:28.217045","last_undegraded":"2024-04-17 22:31:28.217045","last_fullsized":"2024-04-17 22:31:28.217045","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:26.859642","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:26.859642","last_clean_scrub_stamp":"2024-04-17 22:31:26.859642","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":"9","reported_epoch":"24","state":"active+clean","last_fresh":"2024-04-17 22:31:28.241548","last_change":"2024-04-17 22:31:28.221258","last_active":"2024-04-17 22:31:28.241548","last_peered":"2024-04-17 22:31:28.241548","last_clean":"2024-04-17 22:31:28.241548","last_became_active":"2024-04-17 22:31:28.220762","last_became_peered":"2024-04-17 22:31:28.220762","last_unstale":"2024-04-17 22:31:28.241548","last_undegraded":"2024-04-17 22:31:28.241548","last_fullsized":"2024-04-17 22:31:28.241548","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:27.861194","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:27.861194","last_clean_scrub_stamp":"2024-04-17 22:31:27.861194","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":"10","reported_epoch":"24","state":"active+clean","last_fresh":"2024-04-17 22:31:28.213960","last_change":"2024-04-17 22:31:27.897785","last_active":"2024-04-17 22:31:28.213960","last_peered":"2024-04-17 22:31:28.213960","last_clean":"2024-04-17 22:31:28.213960","last_became_active":"2024-04-17 22:31:27.897272","last_became_peered":"2024-04-17 22:31:27.897272","last_unstale":"2024-04-17 22:31:28.213960","last_undegraded":"2024-04-17 22:31:28.213960","last_fullsized":"2024-04-17 22:31:28.213960","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:26.859642","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:26.859642","last_clean_scrub_stamp":"2024-04-17 22:31:26.859642","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":"9","reported_epoch":"24","state":"active+clean","last_fresh":"2024-04-17 22:31:28.234797","last_change":"2024-04-17 22:31:28.229350","last_active":"2024-04-17 22:31:28.234797","last_peered":"2024-04-17 22:31:28.234797","last_clean":"2024-04-17 22:31:28.234797","last_became_active":"2024-04-17 22:31:28.229072","last_became_peered":"2024-04-17 22:31:28.229072","last_unstale":"2024-04-17 22:31:28.234797","last_undegraded":"2024-04-17 22:31:28.234797","last_fullsized":"2024-04-17 22:31:28.234797","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:27.861194","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:27.861194","last_clean_scrub_stamp":"2024-04-17 22:31:27.861194","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":"10","reported_epoch":"24","state":"active+clean","last_fresh":"2024-04-17 22:31:28.220969","last_change":"2024-04-17 22:31:27.896926","last_active":"2024-04-17 22:31:28.220969","last_peered":"2024-04-17 22:31:28.220969","last_clean":"2024-04-17 22:31:28.220969","last_became_active":"2024-04-17 22:31:27.896626","last_became_peered":"2024-04-17 22:31:27.896626","last_unstale":"2024-04-17 22:31:28.220969","last_undegraded":"2024-04-17 22:31:28.220969","last_fullsized":"2024-04-17 22:31:28.220969","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:26.859642","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:26.859642","last_clean_scrub_stamp":"2024-04-17 22:31:26.859642","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":"9","reported_epoch":"24","state":"active+clean","last_fresh":"2024-04-17 22:31:28.247712","last_change":"2024-04-17 22:31:28.245538","last_active":"2024-04-17 22:31:28.247712","last_peered":"2024-04-17 22:31:28.247712","last_clean":"2024-04-17 22:31:28.247712","last_became_active":"2024-04-17 22:31:28.245248","last_became_peered":"2024-04-17 22:31:28.245248","last_unstale":"2024-04-17 22:31:28.247712","last_undegraded":"2024-04-17 22:31:28.247712","last_fullsized":"2024-04-17 22:31:28.247712","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:27.861194","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:27.861194","last_clean_scrub_stamp":"2024-04-17 22:31:27.861194","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":"10","reported_epoch":"24","state":"active+clean","last_fresh":"2024-04-17 22:31:28.214711","last_change":"2024-04-17 22:31:27.892320","last_active":"2024-04-17 22:31:28.214711","last_peered":"2024-04-17 22:31:28.214711","last_clean":"2024-04-17 22:31:28.214711","last_became_active":"2024-04-17 22:31:27.891989","last_became_peered":"2024-04-17 22:31:27.891989","last_unstale":"2024-04-17 22:31:28.214711","last_undegraded":"2024-04-17 22:31:28.214711","last_fullsized":"2024-04-17 22:31:28.214711","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:26.859642","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:26.859642","last_clean_scrub_stamp":"2024-04-17 22:31:26.859642","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":"9","reported_epoch":"24","state":"active+clean","last_fresh":"2024-04-17 22:31:28.249818","last_change":"2024-04-17 22:31:28.246786","last_active":"2024-04-17 22:31:28.249818","last_peered":"2024-04-17 22:31:28.249818","last_clean":"2024-04-17 22:31:28.249818","last_became_active":"2024-04-17 22:31:28.246142","last_became_peered":"2024-04-17 22:31:28.246142","last_unstale":"2024-04-17 22:31:28.249818","last_undegraded":"2024-04-17 22:31:28.249818","last_fullsized":"2024-04-17 22:31:28.249818","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:27.861194","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:27.861194","last_clean_scrub_stamp":"2024-04-17 22:31:27.861194","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":"10","reported_epoch":"24","state":"active+clean","last_fresh":"2024-04-17 22:31:28.234677","last_change":"2024-04-17 22:31:27.910110","last_active":"2024-04-17 22:31:28.234677","last_peered":"2024-04-17 22:31:28.234677","last_clean":"2024-04-17 22:31:28.234677","last_became_active":"2024-04-17 22:31:27.909808","last_became_peered":"2024-04-17 22:31:27.909808","last_unstale":"2024-04-17 22:31:28.234677","last_undegraded":"2024-04-17 22:31:28.234677","last_fullsized":"2024-04-17 22:31:28.234677","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:26.859642","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:26.859642","last_clean_scrub_stamp":"2024-04-17 22:31:26.859642","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":"9","reported_epoch":"24","state":"active+clean","last_fresh":"2024-04-17 22:31:28.250200","last_change":"2024-04-17 22:31:28.247905","last_active":"2024-04-17 22:31:28.250200","last_peered":"2024-04-17 22:31:28.250200","last_clean":"2024-04-17 22:31:28.250200","last_became_active":"2024-04-17 22:31:28.247624","last_became_peered":"2024-04-17 22:31:28.247624","last_unstale":"2024-04-17 22:31:28.250200","last_undegraded":"2024-04-17 22:31:28.250200","last_fullsized":"2024-04-17 22:31:28.250200","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:27.861194","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:27.861194","last_clean_scrub_stamp":"2024-04-17 22:31:27.861194","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":"10","reported_epoch":"24","state":"active+clean","last_fresh":"2024-04-17 22:31:28.216819","last_change":"2024-04-17 22:31:27.902903","last_active":"2024-04-17 22:31:28.216819","last_peered":"2024-04-17 22:31:28.216819","last_clean":"2024-04-17 22:31:28.216819","last_became_active":"2024-04-17 22:31:27.902599","last_became_peered":"2024-04-17 22:31:27.902599","last_unstale":"2024-04-17 22:31:28.216819","last_undegraded":"2024-04-17 22:31:28.216819","last_fullsized":"2024-04-17 22:31:28.216819","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:26.859642","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:26.859642","last_clean_scrub_stamp":"2024-04-17 22:31:26.859642","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":"9","reported_epoch":"24","state":"active+clean","last_fresh":"2024-04-17 22:31:28.248799","last_change":"2024-04-17 22:31:28.242943","last_active":"2024-04-17 22:31:28.248799","last_peered":"2024-04-17 22:31:28.248799","last_clean":"2024-04-17 22:31:28.248799","last_became_active":"2024-04-17 22:31:28.242664","last_became_peered":"2024-04-17 22:31:28.242664","last_unstale":"2024-04-17 22:31:28.248799","last_undegraded":"2024-04-17 22:31:28.248799","last_fullsized":"2024-04-17 22:31:28.248799","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:27.861194","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:27.861194","last_clean_scrub_stamp":"2024-04-17 22:31:27.861194","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"24'3","reported_seq":"13","reported_epoch":"24","state":"active+clean","last_fresh":"2024-04-17 22:31:28.247630","last_change":"2024-04-17 22:31:27.904898","last_active":"2024-04-17 22:31:28.247630","last_peered":"2024-04-17 22:31:28.247630","last_clean":"2024-04-17 22:31:28.247630","last_became_active":"2024-04-17 22:31:27.904577","last_became_peered":"2024-04-17 22:31:27.904577","last_unstale":"2024-04-17 22:31:28.247630","last_undegraded":"2024-04-17 22:31:28.247630","last_fullsized":"2024-04-17 22:31:28.247630","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:26.859642","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:26.859642","last_clean_scrub_stamp":"2024-04-17 22:31:26.859642","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":"9","reported_epoch":"24","state":"active+clean","last_fresh":"2024-04-17 22:31:28.249337","last_change":"2024-04-17 22:31:28.243377","last_active":"2024-04-17 22:31:28.249337","last_peered":"2024-04-17 22:31:28.249337","last_clean":"2024-04-17 22:31:28.249337","last_became_active":"2024-04-17 22:31:28.243155","last_became_peered":"2024-04-17 22:31:28.243155","last_unstale":"2024-04-17 22:31:28.249337","last_undegraded":"2024-04-17 22:31:28.249337","last_fullsized":"2024-04-17 22:31:28.249337","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:27.861194","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:27.861194","last_clean_scrub_stamp":"2024-04-17 22:31:27.861194","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"24'2","reported_seq":"12","reported_epoch":"24","state":"active+clean","last_fresh":"2024-04-17 22:31:28.248740","last_change":"2024-04-17 22:31:27.909719","last_active":"2024-04-17 22:31:28.248740","last_peered":"2024-04-17 22:31:28.248740","last_clean":"2024-04-17 22:31:28.248740","last_became_active":"2024-04-17 22:31:27.909351","last_became_peered":"2024-04-17 22:31:27.909351","last_unstale":"2024-04-17 22:31:28.248740","last_undegraded":"2024-04-17 22:31:28.248740","last_fullsized":"2024-04-17 22:31:28.248740","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:26.859642","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:26.859642","last_clean_scrub_stamp":"2024-04-17 22:31:26.859642","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":634,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":"9","reported_epoch":"24","state":"active+clean","last_fresh":"2024-04-17 22:31:28.245710","last_change":"2024-04-17 22:31:28.243590","last_active":"2024-04-17 22:31:28.245710","last_peered":"2024-04-17 22:31:28.245710","last_clean":"2024-04-17 22:31:28.245710","last_became_active":"2024-04-17 22:31:28.243315","last_became_peered":"2024-04-17 22:31:28.243315","last_unstale":"2024-04-17 22:31:28.245710","last_undegraded":"2024-04-17 22:31:28.245710","last_fullsized":"2024-04-17 22:31:28.245710","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:27.861194","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:27.861194","last_clean_scrub_stamp":"2024-04-17 22:31:27.861194","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"24'1","reported_seq":"11","reported_epoch":"24","state":"active+clean","last_fresh":"2024-04-17 22:31:28.249209","last_change":"2024-04-17 22:31:27.896442","last_active":"2024-04-17 22:31:28.249209","last_peered":"2024-04-17 22:31:28.249209","last_clean":"2024-04-17 22:31:28.249209","last_became_active":"2024-04-17 22:31:27.896068","last_became_peered":"2024-04-17 22:31:27.896068","last_unstale":"2024-04-17 22:31:28.249209","last_undegraded":"2024-04-17 22:31:28.249209","last_fullsized":"2024-04-17 22:31:28.249209","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:26.859642","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:26.859642","last_clean_scrub_stamp":"2024-04-17 22:31:26.859642","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":"9","reported_epoch":"24","state":"active+clean","last_fresh":"2024-04-17 22:31:28.250236","last_change":"2024-04-17 22:31:28.248301","last_active":"2024-04-17 22:31:28.250236","last_peered":"2024-04-17 22:31:28.250236","last_clean":"2024-04-17 22:31:28.250236","last_became_active":"2024-04-17 22:31:28.248003","last_became_peered":"2024-04-17 22:31:28.248003","last_unstale":"2024-04-17 22:31:28.250236","last_undegraded":"2024-04-17 22:31:28.250236","last_fullsized":"2024-04-17 22:31:28.250236","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:27.861194","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:27.861194","last_clean_scrub_stamp":"2024-04-17 22:31:27.861194","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":"10","reported_epoch":"24","state":"active+clean","last_fresh":"2024-04-17 22:31:28.217756","last_change":"2024-04-17 22:31:27.906786","last_active":"2024-04-17 22:31:28.217756","last_peered":"2024-04-17 22:31:28.217756","last_clean":"2024-04-17 22:31:28.217756","last_became_active":"2024-04-17 22:31:27.906426","last_became_peered":"2024-04-17 22:31:27.906426","last_unstale":"2024-04-17 22:31:28.217756","last_undegraded":"2024-04-17 22:31:28.217756","last_fullsized":"2024-04-17 22:31:28.217756","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:26.859642","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:26.859642","last_clean_scrub_stamp":"2024-04-17 22:31:26.859642","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":"9","reported_epoch":"24","state":"active+clean","last_fresh":"2024-04-17 22:31:28.242326","last_change":"2024-04-17 22:31:28.233952","last_active":"2024-04-17 22:31:28.242326","last_peered":"2024-04-17 22:31:28.242326","last_clean":"2024-04-17 22:31:28.242326","last_became_active":"2024-04-17 22:31:28.233501","last_became_peered":"2024-04-17 22:31:28.233501","last_unstale":"2024-04-17 22:31:28.242326","last_undegraded":"2024-04-17 22:31:28.242326","last_fullsized":"2024-04-17 22:31:28.242326","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:27.861194","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:27.861194","last_clean_scrub_stamp":"2024-04-17 22:31:27.861194","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":"10","reported_epoch":"24","state":"active+clean","last_fresh":"2024-04-17 22:31:28.221732","last_change":"2024-04-17 22:31:27.903920","last_active":"2024-04-17 22:31:28.221732","last_peered":"2024-04-17 22:31:28.221732","last_clean":"2024-04-17 22:31:28.221732","last_became_active":"2024-04-17 22:31:27.903611","last_became_peered":"2024-04-17 22:31:27.903611","last_unstale":"2024-04-17 22:31:28.221732","last_undegraded":"2024-04-17 22:31:28.221732","last_fullsized":"2024-04-17 22:31:28.221732","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:26.859642","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:26.859642","last_clean_scrub_stamp":"2024-04-17 22:31:26.859642","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":"9","reported_epoch":"24","state":"active+clean","last_fresh":"2024-04-17 22:31:28.248272","last_change":"2024-04-17 22:31:28.240219","last_active":"2024-04-17 22:31:28.248272","last_peered":"2024-04-17 22:31:28.248272","last_clean":"2024-04-17 22:31:28.248272","last_became_active":"2024-04-17 22:31:28.239619","last_became_peered":"2024-04-17 22:31:28.239619","last_unstale":"2024-04-17 22:31:28.248272","last_undegraded":"2024-04-17 22:31:28.248272","last_fullsized":"2024-04-17 22:31:28.248272","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:27.861194","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:27.861194","last_clean_scrub_stamp":"2024-04-17 22:31:27.861194","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"24'4","reported_seq":"14","reported_epoch":"24","state":"active+clean","last_fresh":"2024-04-17 22:31:28.242440","last_change":"2024-04-17 22:31:27.908645","last_active":"2024-04-17 22:31:28.242440","last_peered":"2024-04-17 22:31:28.242440","last_clean":"2024-04-17 22:31:28.242440","last_became_active":"2024-04-17 22:31:27.907968","last_became_peered":"2024-04-17 22:31:27.907968","last_unstale":"2024-04-17 22:31:28.242440","last_undegraded":"2024-04-17 22:31:28.242440","last_fullsized":"2024-04-17 22:31:28.242440","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:26.859642","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:26.859642","last_clean_scrub_stamp":"2024-04-17 22:31:26.859642","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":"9","reported_epoch":"24","state":"active+clean","last_fresh":"2024-04-17 22:31:28.248990","last_change":"2024-04-17 22:31:28.243804","last_active":"2024-04-17 22:31:28.248990","last_peered":"2024-04-17 22:31:28.248990","last_clean":"2024-04-17 22:31:28.248990","last_became_active":"2024-04-17 22:31:28.243487","last_became_peered":"2024-04-17 22:31:28.243487","last_unstale":"2024-04-17 22:31:28.248990","last_undegraded":"2024-04-17 22:31:28.248990","last_fullsized":"2024-04-17 22:31:28.248990","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:27.861194","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:27.861194","last_clean_scrub_stamp":"2024-04-17 22:31:27.861194","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"24'1","reported_seq":"11","reported_epoch":"24","state":"active+clean","last_fresh":"2024-04-17 22:31:28.245104","last_change":"2024-04-17 22:31:27.900670","last_active":"2024-04-17 22:31:28.245104","last_peered":"2024-04-17 22:31:28.245104","last_clean":"2024-04-17 22:31:28.245104","last_became_active":"2024-04-17 22:31:27.900334","last_became_peered":"2024-04-17 22:31:27.900334","last_unstale":"2024-04-17 22:31:28.245104","last_undegraded":"2024-04-17 22:31:28.245104","last_fullsized":"2024-04-17 22:31:28.245104","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:26.859642","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:26.859642","last_clean_scrub_stamp":"2024-04-17 22:31:26.859642","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":"9","reported_epoch":"24","state":"active+clean","last_fresh":"2024-04-17 22:31:28.237857","last_change":"2024-04-17 22:31:28.232961","last_active":"2024-04-17 22:31:28.237857","last_peered":"2024-04-17 22:31:28.237857","last_clean":"2024-04-17 22:31:28.237857","last_became_active":"2024-04-17 22:31:28.224352","last_became_peered":"2024-04-17 22:31:28.224352","last_unstale":"2024-04-17 22:31:28.237857","last_undegraded":"2024-04-17 22:31:28.237857","last_fullsized":"2024-04-17 22:31:28.237857","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:27.861194","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:27.861194","last_clean_scrub_stamp":"2024-04-17 22:31:27.861194","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"24'2","reported_seq":"12","reported_epoch":"24","state":"active+clean","last_fresh":"2024-04-17 22:31:28.244600","last_change":"2024-04-17 22:31:27.905543","last_active":"2024-04-17 22:31:28.244600","last_peered":"2024-04-17 22:31:28.244600","last_clean":"2024-04-17 22:31:28.244600","last_became_active":"2024-04-17 22:31:27.905245","last_became_peered":"2024-04-17 22:31:27.905245","last_unstale":"2024-04-17 22:31:28.244600","last_undegraded":"2024-04-17 22:31:28.244600","last_fullsized":"2024-04-17 22:31:28.244600","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:26.859642","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:26.859642","last_clean_scrub_stamp":"2024-04-17 22:31:26.859642","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":"9","reported_epoch":"24","state":"active+clean","last_fresh":"2024-04-17 22:31:28.232635","last_change":"2024-04-17 22:31:28.227853","last_active":"2024-04-17 22:31:28.232635","last_peered":"2024-04-17 22:31:28.232635","last_clean":"2024-04-17 22:31:28.232635","last_became_active":"2024-04-17 22:31:28.227583","last_became_peered":"2024-04-17 22:31:28.227583","last_unstale":"2024-04-17 22:31:28.232635","last_undegraded":"2024-04-17 22:31:28.232635","last_fullsized":"2024-04-17 22:31:28.232635","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:27.861194","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:27.861194","last_clean_scrub_stamp":"2024-04-17 22:31:27.861194","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":"10","reported_epoch":"24","state":"active+clean","last_fresh":"2024-04-17 22:31:28.224572","last_change":"2024-04-17 22:31:27.908291","last_active":"2024-04-17 22:31:28.224572","last_peered":"2024-04-17 22:31:28.224572","last_clean":"2024-04-17 22:31:28.224572","last_became_active":"2024-04-17 22:31:27.907931","last_became_peered":"2024-04-17 22:31:27.907931","last_unstale":"2024-04-17 22:31:28.224572","last_undegraded":"2024-04-17 22:31:28.224572","last_fullsized":"2024-04-17 22:31:28.224572","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:26.859642","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:26.859642","last_clean_scrub_stamp":"2024-04-17 22:31:26.859642","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":"9","reported_epoch":"24","state":"active+clean","last_fresh":"2024-04-17 22:31:28.225948","last_change":"2024-04-17 22:31:28.216995","last_active":"2024-04-17 22:31:28.225948","last_peered":"2024-04-17 22:31:28.225948","last_clean":"2024-04-17 22:31:28.225948","last_became_active":"2024-04-17 22:31:28.216701","last_became_peered":"2024-04-17 22:31:28.216701","last_unstale":"2024-04-17 22:31:28.225948","last_undegraded":"2024-04-17 22:31:28.225948","last_fullsized":"2024-04-17 22:31:28.225948","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:27.861194","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:27.861194","last_clean_scrub_stamp":"2024-04-17 22:31:27.861194","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":"10","reported_epoch":"24","state":"active+clean","last_fresh":"2024-04-17 22:31:28.217215","last_change":"2024-04-17 22:31:27.905663","last_active":"2024-04-17 22:31:28.217215","last_peered":"2024-04-17 22:31:28.217215","last_clean":"2024-04-17 22:31:28.217215","last_became_active":"2024-04-17 22:31:27.905323","last_became_peered":"2024-04-17 22:31:27.905323","last_unstale":"2024-04-17 22:31:28.217215","last_undegraded":"2024-04-17 22:31:28.217215","last_fullsized":"2024-04-17 22:31:28.217215","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:26.859642","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:26.859642","last_clean_scrub_stamp":"2024-04-17 22:31:26.859642","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":"9","reported_epoch":"24","state":"active+clean","last_fresh":"2024-04-17 22:31:28.238210","last_change":"2024-04-17 22:31:28.226068","last_active":"2024-04-17 22:31:28.238210","last_peered":"2024-04-17 22:31:28.238210","last_clean":"2024-04-17 22:31:28.238210","last_became_active":"2024-04-17 22:31:28.221073","last_became_peered":"2024-04-17 22:31:28.221073","last_unstale":"2024-04-17 22:31:28.238210","last_undegraded":"2024-04-17 22:31:28.238210","last_fullsized":"2024-04-17 22:31:28.238210","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:27.861194","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:27.861194","last_clean_scrub_stamp":"2024-04-17 22:31:27.861194","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"24'1","reported_seq":"11","reported_epoch":"24","state":"active+clean","last_fresh":"2024-04-17 22:31:28.231099","last_change":"2024-04-17 22:31:27.890696","last_active":"2024-04-17 22:31:28.231099","last_peered":"2024-04-17 22:31:28.231099","last_clean":"2024-04-17 22:31:28.231099","last_became_active":"2024-04-17 22:31:27.890476","last_became_peered":"2024-04-17 22:31:27.890476","last_unstale":"2024-04-17 22:31:28.231099","last_undegraded":"2024-04-17 22:31:28.231099","last_fullsized":"2024-04-17 22:31:28.231099","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:26.859642","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:26.859642","last_clean_scrub_stamp":"2024-04-17 22:31:26.859642","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":"9","reported_epoch":"24","state":"active+clean","last_fresh":"2024-04-17 22:31:28.241570","last_change":"2024-04-17 22:31:28.226040","last_active":"2024-04-17 22:31:28.241570","last_peered":"2024-04-17 22:31:28.241570","last_clean":"2024-04-17 22:31:28.241570","last_became_active":"2024-04-17 22:31:28.221032","last_became_peered":"2024-04-17 22:31:28.221032","last_unstale":"2024-04-17 22:31:28.241570","last_undegraded":"2024-04-17 22:31:28.241570","last_fullsized":"2024-04-17 22:31:28.241570","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:27.861194","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:27.861194","last_clean_scrub_stamp":"2024-04-17 22:31:27.861194","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":"10","reported_epoch":"24","state":"active+clean","last_fresh":"2024-04-17 22:31:28.213856","last_change":"2024-04-17 22:31:27.897951","last_active":"2024-04-17 22:31:28.213856","last_peered":"2024-04-17 22:31:28.213856","last_clean":"2024-04-17 22:31:28.213856","last_became_active":"2024-04-17 22:31:27.897492","last_became_peered":"2024-04-17 22:31:27.897492","last_unstale":"2024-04-17 22:31:28.213856","last_undegraded":"2024-04-17 22:31:28.213856","last_fullsized":"2024-04-17 22:31:28.213856","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:26.859642","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:26.859642","last_clean_scrub_stamp":"2024-04-17 22:31:26.859642","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":"9","reported_epoch":"24","state":"active+clean","last_fresh":"2024-04-17 22:31:28.242486","last_change":"2024-04-17 22:31:28.227420","last_active":"2024-04-17 22:31:28.242486","last_peered":"2024-04-17 22:31:28.242486","last_clean":"2024-04-17 22:31:28.242486","last_became_active":"2024-04-17 22:31:28.227117","last_became_peered":"2024-04-17 22:31:28.227117","last_unstale":"2024-04-17 22:31:28.242486","last_undegraded":"2024-04-17 22:31:28.242486","last_fullsized":"2024-04-17 22:31:28.242486","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:27.861194","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:27.861194","last_clean_scrub_stamp":"2024-04-17 22:31:27.861194","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":"10","reported_epoch":"24","state":"active+clean","last_fresh":"2024-04-17 22:31:28.223575","last_change":"2024-04-17 22:31:27.897168","last_active":"2024-04-17 22:31:28.223575","last_peered":"2024-04-17 22:31:28.223575","last_clean":"2024-04-17 22:31:28.223575","last_became_active":"2024-04-17 22:31:27.896855","last_became_peered":"2024-04-17 22:31:27.896855","last_unstale":"2024-04-17 22:31:28.223575","last_undegraded":"2024-04-17 22:31:28.223575","last_fullsized":"2024-04-17 22:31:28.223575","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:26.859642","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:26.859642","last_clean_scrub_stamp":"2024-04-17 22:31:26.859642","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":"9","reported_epoch":"24","state":"active+clean","last_fresh":"2024-04-17 22:31:28.234536","last_change":"2024-04-17 22:31:28.231800","last_active":"2024-04-17 22:31:28.234536","last_peered":"2024-04-17 22:31:28.234536","last_clean":"2024-04-17 22:31:28.234536","last_became_active":"2024-04-17 22:31:28.231508","last_became_peered":"2024-04-17 22:31:28.231508","last_unstale":"2024-04-17 22:31:28.234536","last_undegraded":"2024-04-17 22:31:28.234536","last_fullsized":"2024-04-17 22:31:28.234536","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:27.861194","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:27.861194","last_clean_scrub_stamp":"2024-04-17 22:31:27.861194","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":"10","reported_epoch":"24","state":"active+clean","last_fresh":"2024-04-17 22:31:28.220799","last_change":"2024-04-17 22:31:27.909868","last_active":"2024-04-17 22:31:28.220799","last_peered":"2024-04-17 22:31:28.220799","last_clean":"2024-04-17 22:31:28.220799","last_became_active":"2024-04-17 22:31:27.909546","last_became_peered":"2024-04-17 22:31:27.909546","last_unstale":"2024-04-17 22:31:28.220799","last_undegraded":"2024-04-17 22:31:28.220799","last_fullsized":"2024-04-17 22:31:28.220799","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:26.859642","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:26.859642","last_clean_scrub_stamp":"2024-04-17 22:31:26.859642","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":"9","reported_epoch":"24","state":"active+clean","last_fresh":"2024-04-17 22:31:28.248869","last_change":"2024-04-17 22:31:28.240742","last_active":"2024-04-17 22:31:28.248869","last_peered":"2024-04-17 22:31:28.248869","last_clean":"2024-04-17 22:31:28.248869","last_became_active":"2024-04-17 22:31:28.240452","last_became_peered":"2024-04-17 22:31:28.240452","last_unstale":"2024-04-17 22:31:28.248869","last_undegraded":"2024-04-17 22:31:28.248869","last_fullsized":"2024-04-17 22:31:28.248869","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:27.861194","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:27.861194","last_clean_scrub_stamp":"2024-04-17 22:31:27.861194","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":"10","reported_epoch":"24","state":"active+clean","last_fresh":"2024-04-17 22:31:28.220894","last_change":"2024-04-17 22:31:27.911380","last_active":"2024-04-17 22:31:28.220894","last_peered":"2024-04-17 22:31:28.220894","last_clean":"2024-04-17 22:31:28.220894","last_became_active":"2024-04-17 22:31:27.911084","last_became_peered":"2024-04-17 22:31:27.911084","last_unstale":"2024-04-17 22:31:28.220894","last_undegraded":"2024-04-17 22:31:28.220894","last_fullsized":"2024-04-17 22:31:28.220894","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:26.859642","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:26.859642","last_clean_scrub_stamp":"2024-04-17 22:31:26.859642","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":"22","reported_epoch":"24","state":"active+clean","last_fresh":"2024-04-17 22:31:28.215613","last_change":"2024-04-17 22:28:59.253977","last_active":"2024-04-17 22:31:28.215613","last_peered":"2024-04-17 22:31:28.215613","last_clean":"2024-04-17 22:31:28.215613","last_became_active":"2024-04-17 22:28:59.253641","last_became_peered":"2024-04-17 22:28:59.253641","last_unstale":"2024-04-17 22:31:28.215613","last_undegraded":"2024-04-17 22:31:28.215613","last_fullsized":"2024-04-17 22:31:28.215613","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:28:58.239187","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:28:58.239187","last_clean_scrub_stamp":"2024-04-17 22:28:58.239187","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":"9","reported_epoch":"24","state":"active+clean","last_fresh":"2024-04-17 22:31:28.249672","last_change":"2024-04-17 22:31:28.242918","last_active":"2024-04-17 22:31:28.249672","last_peered":"2024-04-17 22:31:28.249672","last_clean":"2024-04-17 22:31:28.249672","last_became_active":"2024-04-17 22:31:28.242603","last_became_peered":"2024-04-17 22:31:28.242603","last_unstale":"2024-04-17 22:31:28.249672","last_undegraded":"2024-04-17 22:31:28.249672","last_fullsized":"2024-04-17 22:31:28.249672","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:27.861194","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:27.861194","last_clean_scrub_stamp":"2024-04-17 22:31:27.861194","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":"10","reported_epoch":"24","state":"active+clean","last_fresh":"2024-04-17 22:31:28.213483","last_change":"2024-04-17 22:31:27.887028","last_active":"2024-04-17 22:31:28.213483","last_peered":"2024-04-17 22:31:28.213483","last_clean":"2024-04-17 22:31:28.213483","last_became_active":"2024-04-17 22:31:27.886651","last_became_peered":"2024-04-17 22:31:27.886651","last_unstale":"2024-04-17 22:31:28.213483","last_undegraded":"2024-04-17 22:31:28.213483","last_fullsized":"2024-04-17 22:31:28.213483","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:26.859642","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:26.859642","last_clean_scrub_stamp":"2024-04-17 22:31:26.859642","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":"22","reported_epoch":"24","state":"active+clean","last_fresh":"2024-04-17 22:31:28.222619","last_change":"2024-04-17 22:28:59.255733","last_active":"2024-04-17 22:31:28.222619","last_peered":"2024-04-17 22:31:28.222619","last_clean":"2024-04-17 22:31:28.222619","last_became_active":"2024-04-17 22:28:59.255411","last_became_peered":"2024-04-17 22:28:59.255411","last_unstale":"2024-04-17 22:31:28.222619","last_undegraded":"2024-04-17 22:31:28.222619","last_fullsized":"2024-04-17 22:31:28.222619","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:28:58.239187","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:28:58.239187","last_clean_scrub_stamp":"2024-04-17 22:28:58.239187","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":"9","reported_epoch":"24","state":"active+clean","last_fresh":"2024-04-17 22:31:28.250220","last_change":"2024-04-17 22:31:28.247975","last_active":"2024-04-17 22:31:28.250220","last_peered":"2024-04-17 22:31:28.250220","last_clean":"2024-04-17 22:31:28.250220","last_became_active":"2024-04-17 22:31:28.247701","last_became_peered":"2024-04-17 22:31:28.247701","last_unstale":"2024-04-17 22:31:28.250220","last_undegraded":"2024-04-17 22:31:28.250220","last_fullsized":"2024-04-17 22:31:28.250220","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:27.861194","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:27.861194","last_clean_scrub_stamp":"2024-04-17 22:31:27.861194","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":"10","reported_epoch":"24","state":"active+clean","last_fresh":"2024-04-17 22:31:28.215413","last_change":"2024-04-17 22:31:27.888001","last_active":"2024-04-17 22:31:28.215413","last_peered":"2024-04-17 22:31:28.215413","last_clean":"2024-04-17 22:31:28.215413","last_became_active":"2024-04-17 22:31:27.887621","last_became_peered":"2024-04-17 22:31:27.887621","last_unstale":"2024-04-17 22:31:28.215413","last_undegraded":"2024-04-17 22:31:28.215413","last_fullsized":"2024-04-17 22:31:28.215413","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:26.859642","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:26.859642","last_clean_scrub_stamp":"2024-04-17 22:31:26.859642","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":"22","reported_epoch":"24","state":"active+clean","last_fresh":"2024-04-17 22:31:28.217763","last_change":"2024-04-17 22:28:59.252728","last_active":"2024-04-17 22:31:28.217763","last_peered":"2024-04-17 22:31:28.217763","last_clean":"2024-04-17 22:31:28.217763","last_became_active":"2024-04-17 22:28:59.252345","last_became_peered":"2024-04-17 22:28:59.252345","last_unstale":"2024-04-17 22:31:28.217763","last_undegraded":"2024-04-17 22:31:28.217763","last_fullsized":"2024-04-17 22:31:28.217763","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:28:58.239187","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:28:58.239187","last_clean_scrub_stamp":"2024-04-17 22:28:58.239187","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":"9","reported_epoch":"24","state":"active+clean","last_fresh":"2024-04-17 22:31:28.236530","last_change":"2024-04-17 22:31:28.217885","last_active":"2024-04-17 22:31:28.236530","last_peered":"2024-04-17 22:31:28.236530","last_clean":"2024-04-17 22:31:28.236530","last_became_active":"2024-04-17 22:31:28.217595","last_became_peered":"2024-04-17 22:31:28.217595","last_unstale":"2024-04-17 22:31:28.236530","last_undegraded":"2024-04-17 22:31:28.236530","last_fullsized":"2024-04-17 22:31:28.236530","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:27.861194","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:27.861194","last_clean_scrub_stamp":"2024-04-17 22:31:27.861194","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"24'2","reported_seq":"12","reported_epoch":"24","state":"active+clean","last_fresh":"2024-04-17 22:31:28.233646","last_change":"2024-04-17 22:31:27.898716","last_active":"2024-04-17 22:31:28.233646","last_peered":"2024-04-17 22:31:28.233646","last_clean":"2024-04-17 22:31:28.233646","last_became_active":"2024-04-17 22:31:27.898382","last_became_peered":"2024-04-17 22:31:27.898382","last_unstale":"2024-04-17 22:31:28.233646","last_undegraded":"2024-04-17 22:31:28.233646","last_fullsized":"2024-04-17 22:31:28.233646","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:26.859642","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:26.859642","last_clean_scrub_stamp":"2024-04-17 22:31:26.859642","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":"22","reported_epoch":"24","state":"active+clean","last_fresh":"2024-04-17 22:31:28.214489","last_change":"2024-04-17 22:28:59.253392","last_active":"2024-04-17 22:31:28.214489","last_peered":"2024-04-17 22:31:28.214489","last_clean":"2024-04-17 22:31:28.214489","last_became_active":"2024-04-17 22:28:59.253049","last_became_peered":"2024-04-17 22:28:59.253049","last_unstale":"2024-04-17 22:31:28.214489","last_undegraded":"2024-04-17 22:31:28.214489","last_fullsized":"2024-04-17 22:31:28.214489","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:28:58.239187","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:28:58.239187","last_clean_scrub_stamp":"2024-04-17 22:28:58.239187","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":"9","reported_epoch":"24","state":"active+clean","last_fresh":"2024-04-17 22:31:28.246320","last_change":"2024-04-17 22:31:28.236943","last_active":"2024-04-17 22:31:28.246320","last_peered":"2024-04-17 22:31:28.246320","last_clean":"2024-04-17 22:31:28.246320","last_became_active":"2024-04-17 22:31:28.236616","last_became_peered":"2024-04-17 22:31:28.236616","last_unstale":"2024-04-17 22:31:28.246320","last_undegraded":"2024-04-17 22:31:28.246320","last_fullsized":"2024-04-17 22:31:28.246320","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:27.861194","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:27.861194","last_clean_scrub_stamp":"2024-04-17 22:31:27.861194","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":"10","reported_epoch":"24","state":"active+clean","last_fresh":"2024-04-17 22:31:28.224669","last_change":"2024-04-17 22:31:27.909892","last_active":"2024-04-17 22:31:28.224669","last_peered":"2024-04-17 22:31:28.224669","last_clean":"2024-04-17 22:31:28.224669","last_became_active":"2024-04-17 22:31:27.909398","last_became_peered":"2024-04-17 22:31:27.909398","last_unstale":"2024-04-17 22:31:28.224669","last_undegraded":"2024-04-17 22:31:28.224669","last_fullsized":"2024-04-17 22:31:28.224669","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:26.859642","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:26.859642","last_clean_scrub_stamp":"2024-04-17 22:31:26.859642","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":"22","reported_epoch":"24","state":"active+clean","last_fresh":"2024-04-17 22:31:28.221163","last_change":"2024-04-17 22:28:59.254907","last_active":"2024-04-17 22:31:28.221163","last_peered":"2024-04-17 22:31:28.221163","last_clean":"2024-04-17 22:31:28.221163","last_became_active":"2024-04-17 22:28:59.254633","last_became_peered":"2024-04-17 22:28:59.254633","last_unstale":"2024-04-17 22:31:28.221163","last_undegraded":"2024-04-17 22:31:28.221163","last_fullsized":"2024-04-17 22:31:28.221163","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:28:58.239187","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:28:58.239187","last_clean_scrub_stamp":"2024-04-17 22:28:58.239187","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":"9","reported_epoch":"24","state":"active+clean","last_fresh":"2024-04-17 22:31:28.246382","last_change":"2024-04-17 22:31:28.239236","last_active":"2024-04-17 22:31:28.246382","last_peered":"2024-04-17 22:31:28.246382","last_clean":"2024-04-17 22:31:28.246382","last_became_active":"2024-04-17 22:31:28.237247","last_became_peered":"2024-04-17 22:31:28.237247","last_unstale":"2024-04-17 22:31:28.246382","last_undegraded":"2024-04-17 22:31:28.246382","last_fullsized":"2024-04-17 22:31:28.246382","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:27.861194","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:27.861194","last_clean_scrub_stamp":"2024-04-17 22:31:27.861194","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":"10","reported_epoch":"24","state":"active+clean","last_fresh":"2024-04-17 22:31:28.219529","last_change":"2024-04-17 22:31:27.901355","last_active":"2024-04-17 22:31:28.219529","last_peered":"2024-04-17 22:31:28.219529","last_clean":"2024-04-17 22:31:28.219529","last_became_active":"2024-04-17 22:31:27.899539","last_became_peered":"2024-04-17 22:31:27.899539","last_unstale":"2024-04-17 22:31:28.219529","last_undegraded":"2024-04-17 22:31:28.219529","last_fullsized":"2024-04-17 22:31:28.219529","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:26.859642","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:26.859642","last_clean_scrub_stamp":"2024-04-17 22:31:26.859642","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":"22","reported_epoch":"24","state":"active+clean","last_fresh":"2024-04-17 22:31:28.221817","last_change":"2024-04-17 22:28:59.254217","last_active":"2024-04-17 22:31:28.221817","last_peered":"2024-04-17 22:31:28.221817","last_clean":"2024-04-17 22:31:28.221817","last_became_active":"2024-04-17 22:28:59.253877","last_became_peered":"2024-04-17 22:28:59.253877","last_unstale":"2024-04-17 22:31:28.221817","last_undegraded":"2024-04-17 22:31:28.221817","last_fullsized":"2024-04-17 22:31:28.221817","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:28:58.239187","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:28:58.239187","last_clean_scrub_stamp":"2024-04-17 22:28:58.239187","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":"9","reported_epoch":"24","state":"active+clean","last_fresh":"2024-04-17 22:31:28.246061","last_change":"2024-04-17 22:31:28.236916","last_active":"2024-04-17 22:31:28.246061","last_peered":"2024-04-17 22:31:28.246061","last_clean":"2024-04-17 22:31:28.246061","last_became_active":"2024-04-17 22:31:28.236596","last_became_peered":"2024-04-17 22:31:28.236596","last_unstale":"2024-04-17 22:31:28.246061","last_undegraded":"2024-04-17 22:31:28.246061","last_fullsized":"2024-04-17 22:31:28.246061","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:27.861194","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:27.861194","last_clean_scrub_stamp":"2024-04-17 22:31:27.861194","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":"10","reported_epoch":"24","state":"active+clean","last_fresh":"2024-04-17 22:31:28.216350","last_change":"2024-04-17 22:31:27.895017","last_active":"2024-04-17 22:31:28.216350","last_peered":"2024-04-17 22:31:28.216350","last_clean":"2024-04-17 22:31:28.216350","last_became_active":"2024-04-17 22:31:27.894519","last_became_peered":"2024-04-17 22:31:27.894519","last_unstale":"2024-04-17 22:31:28.216350","last_undegraded":"2024-04-17 22:31:28.216350","last_fullsized":"2024-04-17 22:31:28.216350","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:26.859642","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:26.859642","last_clean_scrub_stamp":"2024-04-17 22:31:26.859642","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":"22","reported_epoch":"24","state":"active+clean","last_fresh":"2024-04-17 22:31:28.235418","last_change":"2024-04-17 22:28:59.254830","last_active":"2024-04-17 22:31:28.235418","last_peered":"2024-04-17 22:31:28.235418","last_clean":"2024-04-17 22:31:28.235418","last_became_active":"2024-04-17 22:28:59.254505","last_became_peered":"2024-04-17 22:28:59.254505","last_unstale":"2024-04-17 22:31:28.235418","last_undegraded":"2024-04-17 22:31:28.235418","last_fullsized":"2024-04-17 22:31:28.235418","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:28:58.239187","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:28:58.239187","last_clean_scrub_stamp":"2024-04-17 22:28:58.239187","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":"9","reported_epoch":"24","state":"active+clean","last_fresh":"2024-04-17 22:31:28.250136","last_change":"2024-04-17 22:31:28.246676","last_active":"2024-04-17 22:31:28.250136","last_peered":"2024-04-17 22:31:28.250136","last_clean":"2024-04-17 22:31:28.250136","last_became_active":"2024-04-17 22:31:28.244744","last_became_peered":"2024-04-17 22:31:28.244744","last_unstale":"2024-04-17 22:31:28.250136","last_undegraded":"2024-04-17 22:31:28.250136","last_fullsized":"2024-04-17 22:31:28.250136","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:27.861194","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:27.861194","last_clean_scrub_stamp":"2024-04-17 22:31:27.861194","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":"10","reported_epoch":"24","state":"active+clean","last_fresh":"2024-04-17 22:31:28.217816","last_change":"2024-04-17 22:31:27.900858","last_active":"2024-04-17 22:31:28.217816","last_peered":"2024-04-17 22:31:28.217816","last_clean":"2024-04-17 22:31:28.217816","last_became_active":"2024-04-17 22:31:27.900258","last_became_peered":"2024-04-17 22:31:27.900258","last_unstale":"2024-04-17 22:31:28.217816","last_undegraded":"2024-04-17 22:31:28.217816","last_fullsized":"2024-04-17 22:31:28.217816","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:26.859642","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:26.859642","last_clean_scrub_stamp":"2024-04-17 22:31:26.859642","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":"22","reported_epoch":"24","state":"active+clean","last_fresh":"2024-04-17 22:31:28.222295","last_change":"2024-04-17 22:28:59.254055","last_active":"2024-04-17 22:31:28.222295","last_peered":"2024-04-17 22:31:28.222295","last_clean":"2024-04-17 22:31:28.222295","last_became_active":"2024-04-17 22:28:59.253750","last_became_peered":"2024-04-17 22:28:59.253750","last_unstale":"2024-04-17 22:31:28.222295","last_undegraded":"2024-04-17 22:31:28.222295","last_fullsized":"2024-04-17 22:31:28.222295","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:28:58.239187","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:28:58.239187","last_clean_scrub_stamp":"2024-04-17 22:28:58.239187","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":4},{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":2286,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":128,"acting":128,"num_store_stats":4},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705699,"num_pgs":67,"num_osds":1,"num_per_pool_osds":1,"kb":94371840,"kb_used":1048840,"kb_used_data":248,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323000,"statfs":{"total":96636764160,"available":95562752000,"internally_reserved":1073741824,"allocated":253952,"data_stored":72360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 17 22:30:59 2024","interfaces":[{"interface":"back","average":{"1min":0.308,"5min":0.30399999999999999,"15min":0.30399999999999999},"min":{"1min":0.20399999999999999,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"last":0.247},{"interface":"front","average":{"1min":0.30199999999999999,"5min":0.314,"15min":0.316},"min":{"1min":0.21099999999999999,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.44600000000000001,"5min":0.52400000000000002,"15min":0.52400000000000002},"last":0.23999999999999999}]},{"osd":1,"last update":"Wed Apr 17 22:30:59 2024","interfaces":[{"interface":"back","average":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"min":{"1min":0.223,"5min":0.19600000000000001,"15min":0.19600000000000001},"max":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"last":0.214},{"interface":"front","average":{"1min":0.33400000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"min":{"1min":0.249,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.438,"5min":0.55300000000000005,"15min":0.55300000000000005},"last":0.28100000000000003}]},{"osd":2,"last update":"Wed Apr 17 22:30:59 2024","interfaces":[{"interface":"back","average":{"1min":0.33800000000000002,"5min":0.34799999999999998,"15min":0.34999999999999998},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.46200000000000002,"5min":0.50600000000000001,"15min":0.50600000000000001},"last":0.26800000000000002},{"interface":"front","average":{"1min":0.32800000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"min":{"1min":0.23200000000000001,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.46899999999999997,"5min":0.47699999999999998,"15min":0.47699999999999998},"last":0.23400000000000001}]}]},{"osd":2,"up_from":9,"seq":38654705699,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"kb":94371840,"kb_used":1048816,"kb_used_data":232,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323024,"statfs":{"total":96636764160,"available":95562776576,"internally_reserved":1073741824,"allocated":237568,"data_stored":71614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 17 22:30:58 2024","interfaces":[{"interface":"back","average":{"1min":0.30099999999999999,"5min":0.28499999999999998,"15min":0.28299999999999997},"min":{"1min":0.22600000000000001,"5min":0.112,"15min":0.112},"max":{"1min":0.41499999999999998,"5min":0.46400000000000002,"15min":0.46400000000000002},"last":0.30299999999999999},{"interface":"front","average":{"1min":0.33600000000000002,"5min":0.314,"15min":0.31},"min":{"1min":0.27600000000000002,"5min":0.155,"15min":0.155},"max":{"1min":0.40400000000000003,"5min":0.49199999999999999,"15min":0.49199999999999999},"last":0.31}]},{"osd":1,"last update":"Wed Apr 17 22:30:58 2024","interfaces":[{"interface":"back","average":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"min":{"1min":0.214,"5min":0.17899999999999999,"15min":0.17899999999999999},"max":{"1min":0.377,"5min":0.52900000000000003,"15min":0.52900000000000003},"last":0.29099999999999998},{"interface":"front","average":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"min":{"1min":0.245,"5min":0.192,"15min":0.192},"max":{"1min":0.38800000000000001,"5min":0.47199999999999998,"15min":0.47199999999999998},"last":0.23200000000000001}]},{"osd":3,"last update":"Wed Apr 17 22:30:58 2024","interfaces":[{"interface":"back","average":{"1min":0.30399999999999999,"5min":0.29599999999999999,"15min":0.29499999999999998},"min":{"1min":0.24199999999999999,"5min":0.191,"15min":0.191},"max":{"1min":0.42599999999999999,"5min":0.45600000000000002,"15min":0.45600000000000002},"last":0.24199999999999999},{"interface":"front","average":{"1min":0.31900000000000001,"5min":0.311,"15min":0.31},"min":{"1min":0.26200000000000001,"5min":0.214,"15min":0.214},"max":{"1min":0.41299999999999998,"5min":0.48699999999999999,"15min":0.48699999999999999},"last":0.33900000000000002}]}]},{"osd":0,"up_from":9,"seq":38654705699,"num_pgs":62,"num_osds":1,"num_per_pool_osds":1,"kb":94371840,"kb_used":1048804,"kb_used_data":220,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323036,"statfs":{"total":96636764160,"available":95562788864,"internally_reserved":1073741824,"allocated":225280,"data_stored":70164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Apr 17 22:30:56 2024","interfaces":[{"interface":"back","average":{"1min":0.248,"5min":0.28299999999999997,"15min":0.28899999999999998},"min":{"1min":0.159,"5min":0.159,"15min":0.159},"max":{"1min":0.33200000000000002,"5min":0.40899999999999997,"15min":0.40899999999999997},"last":0.23799999999999999},{"interface":"front","average":{"1min":0.26000000000000001,"5min":0.29199999999999998,"15min":0.29699999999999999},"min":{"1min":0.189,"5min":0.16600000000000001,"15min":0.16600000000000001},"max":{"1min":0.35399999999999998,"5min":0.42999999999999999,"15min":0.42999999999999999},"last":0.14499999999999999}]},{"osd":2,"last update":"Wed Apr 17 22:30:56 2024","interfaces":[{"interface":"back","average":{"1min":0.28899999999999998,"5min":0.32100000000000001,"15min":0.32700000000000001},"min":{"1min":0.218,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.39500000000000002,"5min":0.48599999999999999,"15min":0.48599999999999999},"last":0.217},{"interface":"front","average":{"1min":0.27800000000000002,"5min":0.317,"15min":0.32300000000000001},"min":{"1min":0.19400000000000001,"5min":0.19400000000000001,"15min":0.19400000000000001},"max":{"1min":0.371,"5min":0.60699999999999998,"15min":0.60699999999999998},"last":0.224}]},{"osd":3,"last update":"Wed Apr 17 22:30:56 2024","interfaces":[{"interface":"back","average":{"1min":0.311,"5min":0.33400000000000002,"15min":0.33800000000000002},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.40600000000000003,"5min":0.505,"15min":0.505},"last":0.25600000000000001},{"interface":"front","average":{"1min":0.28599999999999998,"5min":0.32000000000000001,"15min":0.32600000000000001},"min":{"1min":0.20000000000000001,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.38300000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"last":0.16300000000000001}]}]},{"osd":1,"up_from":9,"seq":38654705698,"num_pgs":65,"num_osds":1,"num_per_pool_osds":1,"kb":94371840,"kb_used":1048812,"kb_used_data":228,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323028,"statfs":{"total":96636764160,"available":95562780672,"internally_reserved":1073741824,"allocated":233472,"data_stored":70730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,4,4,4],"upper_bound":32},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 17 22:31:00 2024","interfaces":[{"interface":"back","average":{"1min":0.26900000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"min":{"1min":0.128,"5min":0.128,"15min":0.128},"max":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"last":2.782},{"interface":"front","average":{"1min":0.28000000000000003,"5min":0.27600000000000002,"15min":0.27600000000000002},"min":{"1min":0.19400000000000001,"5min":0.191,"15min":0.191},"max":{"1min":0.441,"5min":0.441,"15min":0.441},"last":3.7250000000000001}]},{"osd":2,"last update":"Wed Apr 17 22:31:00 2024","interfaces":[{"interface":"back","average":{"1min":0.28599999999999998,"5min":0.27900000000000003,"15min":0.27800000000000002},"min":{"1min":0.153,"5min":0.153,"15min":0.153},"max":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"last":2.3100000000000001},{"interface":"front","average":{"1min":0.28599999999999998,"5min":0.29099999999999998,"15min":0.29199999999999998},"min":{"1min":0.18099999999999999,"5min":0.18099999999999999,"15min":0.18099999999999999},"max":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"last":0.16500000000000001}]},{"osd":3,"last update":"Wed Apr 17 22:31:00 2024","interfaces":[{"interface":"back","average":{"1min":0.30499999999999999,"5min":0.29699999999999999,"15min":0.29499999999999998},"min":{"1min":0.23300000000000001,"5min":0.218,"15min":0.218},"max":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"last":11.692},{"interface":"front","average":{"1min":0.309,"5min":0.315,"15min":0.316},"min":{"1min":0.245,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"last":5.3730000000000002}]}]}]}} 2024-04-17T22:31:31.399 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-17T22:31:31.399 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-17T22:31:31.567 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.578 7fcdfe3c7700 1 Processor -- start 2024-04-17T22:31:31.567 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.578 7fcdfe3c7700 1 -- start start 2024-04-17T22:31:31.567 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.579 7fcdfe3c7700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fcdf819fde0 0x7fcdf81a0230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:31.567 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.579 7fcdfe3c7700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fcdf81a0860 0x7fcdf81a3020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:31.567 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.579 7fcdfe3c7700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fcdf81a3650 0x7fcdf81a3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:31.567 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.579 7fcdfe3c7700 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcdf81104c0 con 0x7fcdf819fde0 2024-04-17T22:31:31.567 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.579 7fcdfe3c7700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdf8104180 con 0x7fcdf81a0860 2024-04-17T22:31:31.567 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.579 7fcdfe3c7700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcdf80f2c90 con 0x7fcdf81a3650 2024-04-17T22:31:31.567 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.579 7fcdf7086700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fcdf819fde0 0x7fcdf81a0230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:31.567 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.579 7fcdf7887700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fcdf81a3650 0x7fcdf81a3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:31.568 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.579 7fcdf6885700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fcdf81a0860 0x7fcdf81a3020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:31.568 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.579 7fcdf7086700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fcdf819fde0 0x7fcdf81a0230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:52166/0 (socket says 172.21.15.102:52166) 2024-04-17T22:31:31.568 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.579 7fcdf7086700 1 -- 172.21.15.102:0/2849095616 learned_addr learned my addr 172.21.15.102:0/2849095616 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:31:31.568 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.579 7fcdf6885700 1 -- 172.21.15.102:0/2849095616 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fcdf81a3650 msgr2=0x7fcdf81a3aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:31:31.568 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.579 7fcdf6885700 1 --2- 172.21.15.102:0/2849095616 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fcdf81a3650 0x7fcdf81a3aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:31.568 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.579 7fcdf6885700 1 -- 172.21.15.102:0/2849095616 >> v2:172.21.15.102:3300/0 conn(0x7fcdf819fde0 msgr2=0x7fcdf81a0230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:31:31.568 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.579 7fcdf6885700 1 --2- 172.21.15.102:0/2849095616 >> v2:172.21.15.102:3300/0 conn(0x7fcdf819fde0 0x7fcdf81a0230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:31.568 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.579 7fcdf6885700 1 -- 172.21.15.102:0/2849095616 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdf80f28e0 con 0x7fcdf81a0860 2024-04-17T22:31:31.569 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.579 7fcdf7086700 1 --2- 172.21.15.102:0/2849095616 >> v2:172.21.15.102:3300/0 conn(0x7fcdf819fde0 0x7fcdf81a0230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:31.569 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.579 7fcdf7887700 1 --2- 172.21.15.102:0/2849095616 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fcdf81a3650 0x7fcdf81a3aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:31.569 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.579 7fcdf6885700 1 --2- 172.21.15.102:0/2849095616 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fcdf81a0860 0x7fcdf81a3020 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 rx=0x7fcdf80f5a00 tx=0x7fcde800ba50).ready entity=mon.1 client_cookie=b2c827b71a30e479 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:31.569 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.579 7fcdf6084700 1 -- 172.21.15.102:0/2849095616 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fcdf80f28e0 con 0x7fcdf81a0860 2024-04-17T22:31:31.569 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.580 7fcdf6084700 1 -- 172.21.15.102:0/2849095616 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcde800d570 con 0x7fcdf81a0860 2024-04-17T22:31:31.570 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.580 7fcdf6084700 1 -- 172.21.15.102:0/2849095616 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fcde800d940 con 0x7fcdf81a0860 2024-04-17T22:31:31.570 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.580 7fcdfe3c7700 1 -- 172.21.15.102:0/2849095616 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fcdf81a0860 msgr2=0x7fcdf81a3020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:31.570 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.580 7fcdfe3c7700 1 --2- 172.21.15.102:0/2849095616 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fcdf81a0860 0x7fcdf81a3020 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 rx=0x7fcdf80f5a00 tx=0x7fcde800ba50).stop 2024-04-17T22:31:31.570 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.580 7fcdfe3c7700 1 -- 172.21.15.102:0/2849095616 shutdown_connections 2024-04-17T22:31:31.570 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.580 7fcdfe3c7700 1 --2- 172.21.15.102:0/2849095616 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fcdf81a3650 0x7fcdf81a3aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:31.570 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.580 7fcdfe3c7700 1 --2- 172.21.15.102:0/2849095616 >> v2:172.21.15.102:3300/0 conn(0x7fcdf819fde0 0x7fcdf81a0230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:31.571 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.580 7fcdfe3c7700 1 --2- 172.21.15.102:0/2849095616 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fcdf81a0860 0x7fcdf81a3020 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:31.571 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.580 7fcdfe3c7700 1 -- 172.21.15.102:0/2849095616 >> 172.21.15.102:0/2849095616 conn(0x7fcdf8113000 msgr2=0x7fcdf8113450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:31.571 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.580 7fcdfe3c7700 1 -- 172.21.15.102:0/2849095616 shutdown_connections 2024-04-17T22:31:31.571 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.580 7fcdfe3c7700 1 -- 172.21.15.102:0/2849095616 wait complete. 2024-04-17T22:31:31.588 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.603 7fcdfe3c7700 1 Processor -- start 2024-04-17T22:31:31.589 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.603 7fcdfe3c7700 1 -- start start 2024-04-17T22:31:31.591 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.603 7fcdfe3c7700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fcdf80de4b0 0x7fcdf81ad400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:31.591 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.603 7fcdfe3c7700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcdf81ada30 0x7fcdf81b3e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:31.592 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.603 7fcdfe3c7700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fcdf81ade80 0x7fcdf81b4480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:31.592 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.603 7fcdfe3c7700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdf80de900 con 0x7fcdf81ada30 2024-04-17T22:31:31.592 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.603 7fcdfe3c7700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdf81a5f90 con 0x7fcdf81ade80 2024-04-17T22:31:31.592 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.603 7fcdfe3c7700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcdf81a65c0 con 0x7fcdf80de4b0 2024-04-17T22:31:31.592 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.603 7fcdf7086700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fcdf80de4b0 0x7fcdf81ad400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:31.592 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.603 7fcdf6885700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcdf81ada30 0x7fcdf81b3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:31.592 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.603 7fcdf7086700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fcdf80de4b0 0x7fcdf81ad400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:33842/0 (socket says 172.21.15.102:33842) 2024-04-17T22:31:31.592 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.603 7fcdf7086700 1 -- 172.21.15.102:0/721733703 learned_addr learned my addr 172.21.15.102:0/721733703 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:31:31.592 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.603 7fcdf7887700 1 --2- 172.21.15.102:0/721733703 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fcdf81ade80 0x7fcdf81b4480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:31.592 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.604 7fcdf7086700 1 -- 172.21.15.102:0/721733703 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fcdf81ade80 msgr2=0x7fcdf81b4480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:31.593 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.604 7fcdf7086700 1 --2- 172.21.15.102:0/721733703 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fcdf81ade80 0x7fcdf81b4480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:31.593 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.604 7fcdf7086700 1 -- 172.21.15.102:0/721733703 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcdf81ada30 msgr2=0x7fcdf81b3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:31.593 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.604 7fcdf7086700 1 --2- 172.21.15.102:0/721733703 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcdf81ada30 0x7fcdf81b3e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:31.593 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.604 7fcdf7086700 1 -- 172.21.15.102:0/721733703 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdf80c4140 con 0x7fcdf80de4b0 2024-04-17T22:31:31.593 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.604 7fcdf7887700 1 --2- 172.21.15.102:0/721733703 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fcdf81ade80 0x7fcdf81b4480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:31.593 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.604 7fcdf6885700 1 --2- 172.21.15.102:0/721733703 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcdf81ada30 0x7fcdf81b3e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:31.593 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.604 7fcdf7086700 1 --2- 172.21.15.102:0/721733703 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fcdf80de4b0 0x7fcdf81ad400 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 rx=0x7fcdf80c55b0 tx=0x7fcdec00ecb0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:31.593 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.604 7fcdf4881700 1 -- 172.21.15.102:0/721733703 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fcdec00f040 con 0x7fcdf80de4b0 2024-04-17T22:31:31.593 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.604 7fcdf4881700 1 -- 172.21.15.102:0/721733703 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdec00c620 con 0x7fcdf80de4b0 2024-04-17T22:31:31.593 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.604 7fcdfe3c7700 1 -- 172.21.15.102:0/721733703 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdf81b4ab0 con 0x7fcdf80de4b0 2024-04-17T22:31:31.593 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.604 7fcdf4881700 1 -- 172.21.15.102:0/721733703 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fcdec00c9f0 con 0x7fcdf80de4b0 2024-04-17T22:31:31.593 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.604 7fcdfe3c7700 1 -- 172.21.15.102:0/721733703 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdf81bb5c0 con 0x7fcdf80de4b0 2024-04-17T22:31:31.593 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.604 7fcdf4881700 1 -- 172.21.15.102:0/721733703 <== mon.2 v2:172.21.15.102:3302/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7fcdec011070 con 0x7fcdf80de4b0 2024-04-17T22:31:31.593 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.604 7fcdf4881700 1 -- 172.21.15.102:0/721733703 <== mon.2 v2:172.21.15.102:3302/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 3194+0+0 (secure 0 0 0) 0x7fcdec010040 con 0x7fcdf80de4b0 2024-04-17T22:31:31.594 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.605 7fcdfe3c7700 1 -- 172.21.15.102:0/721733703 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcdf80f09f0 con 0x7fcdf80de4b0 2024-04-17T22:31:31.594 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.608 7fcdf4881700 1 -- 172.21.15.102:0/721733703 <== mon.2 v2:172.21.15.102:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+110455 (secure 0 0 0) 0x7fcdec00edf0 con 0x7fcdf80de4b0 2024-04-17T22:31:31.730 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.744 7fcdfe3c7700 1 -- 172.21.15.102:0/721733703 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fcdf80ad030 con 0x7fcdf80de4b0 2024-04-17T22:31:31.733 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.748 7fcdf4881700 1 -- 172.21.15.102:0/721733703 <== mon.2 v2:172.21.15.102:3302/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v9) v1 ==== 190+0+0 (secure 0 0 0) 0x7fcdec000c30 con 0x7fcdf80de4b0 2024-04-17T22:31:31.735 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.749 7fcdfe3c7700 1 -- 172.21.15.102:0/721733703 >> v1:172.21.15.102:6816/42360 conn(0x7fcdd80145a0 legacy=0x7fcdd8016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:31.735 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.749 7fcdfe3c7700 1 -- 172.21.15.102:0/721733703 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fcdf80de4b0 msgr2=0x7fcdf81ad400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:31.735 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.749 7fcdfe3c7700 1 --2- 172.21.15.102:0/721733703 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fcdf80de4b0 0x7fcdf81ad400 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 rx=0x7fcdf80c55b0 tx=0x7fcdec00ecb0).stop 2024-04-17T22:31:31.736 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.749 7fcdfe3c7700 1 -- 172.21.15.102:0/721733703 shutdown_connections 2024-04-17T22:31:31.736 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.749 7fcdfe3c7700 1 --2- 172.21.15.102:0/721733703 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fcdf81ade80 0x7fcdf81b4480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:31.736 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.749 7fcdfe3c7700 1 --2- 172.21.15.102:0/721733703 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fcdf80de4b0 0x7fcdf81ad400 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:31.736 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.749 7fcdfe3c7700 1 --2- 172.21.15.102:0/721733703 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcdf81ada30 0x7fcdf81b3e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:31.736 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.749 7fcdfe3c7700 1 -- 172.21.15.102:0/721733703 >> 172.21.15.102:0/721733703 conn(0x7fcdf80c5bb0 msgr2=0x7fcdf80d6890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:31.736 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.749 7fcdfe3c7700 1 -- 172.21.15.102:0/721733703 shutdown_connections 2024-04-17T22:31:31.736 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.749 7fcdfe3c7700 1 -- 172.21.15.102:0/721733703 wait complete. 2024-04-17T22:31:31.737 INFO:teuthology.orchestra.run.smithi102.stderr:updated caps for client.0 2024-04-17T22:31:31.746 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T22:31:31.872 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.884 7fa9caf04700 1 Processor -- start 2024-04-17T22:31:31.872 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.884 7fa9caf04700 1 -- start start 2024-04-17T22:31:31.872 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.884 7fa9caf04700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fa9c410e260 0x7fa9c410e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:31.872 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.884 7fa9caf04700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fa9c410ece0 0x7fa9c40f1750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:31.872 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.884 7fa9caf04700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa9c40f1d80 0x7fa9c41aa4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:31.873 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.884 7fa9caf04700 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9c40f9f30 con 0x7fa9c410ece0 2024-04-17T22:31:31.873 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.884 7fa9caf04700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9c40f9050 con 0x7fa9c40f1d80 2024-04-17T22:31:31.873 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.884 7fa9caf04700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa9c40fa1d0 con 0x7fa9c410e260 2024-04-17T22:31:31.873 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.884 7fa9c37fe700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fa9c410e260 0x7fa9c410e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:31.873 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.884 7fa9c2ffd700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fa9c410ece0 0x7fa9c40f1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:31.873 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.884 7fa9c3fff700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa9c40f1d80 0x7fa9c41aa4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:31.873 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.885 7fa9c37fe700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fa9c410e260 0x7fa9c410e6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:33846/0 (socket says 172.21.15.102:33846) 2024-04-17T22:31:31.873 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.885 7fa9c2ffd700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fa9c410ece0 0x7fa9c40f1750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:52174/0 (socket says 172.21.15.102:52174) 2024-04-17T22:31:31.873 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.885 7fa9c37fe700 1 -- 172.21.15.102:0/4079911139 learned_addr learned my addr 172.21.15.102:0/4079911139 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:31:31.873 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.885 7fa9c37fe700 1 -- 172.21.15.102:0/4079911139 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa9c40f1d80 msgr2=0x7fa9c41aa4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:31:31.873 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.885 7fa9c37fe700 1 --2- 172.21.15.102:0/4079911139 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa9c40f1d80 0x7fa9c41aa4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:31.873 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.885 7fa9c37fe700 1 -- 172.21.15.102:0/4079911139 >> v2:172.21.15.102:3300/0 conn(0x7fa9c410ece0 msgr2=0x7fa9c40f1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:31:31.873 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.885 7fa9c37fe700 1 --2- 172.21.15.102:0/4079911139 >> v2:172.21.15.102:3300/0 conn(0x7fa9c410ece0 0x7fa9c40f1750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:31.873 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.885 7fa9c37fe700 1 -- 172.21.15.102:0/4079911139 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9c40f21d0 con 0x7fa9c410e260 2024-04-17T22:31:31.873 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.885 7fa9c3fff700 1 --2- 172.21.15.102:0/4079911139 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa9c40f1d80 0x7fa9c41aa4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:31.874 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.885 7fa9c2ffd700 1 --2- 172.21.15.102:0/4079911139 >> v2:172.21.15.102:3300/0 conn(0x7fa9c410ece0 0x7fa9c40f1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:31.874 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.885 7fa9c37fe700 1 --2- 172.21.15.102:0/4079911139 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fa9c410e260 0x7fa9c410e6b0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 rx=0x7fa9c40f4490 tx=0x7fa9bc009a30).ready entity=mon.2 client_cookie=8fbd066946ee35fb server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:31.874 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.885 7fa9c27fc700 1 -- 172.21.15.102:0/4079911139 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa9c40f21d0 con 0x7fa9c410e260 2024-04-17T22:31:31.874 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.885 7fa9c27fc700 1 -- 172.21.15.102:0/4079911139 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9bc00f510 con 0x7fa9c410e260 2024-04-17T22:31:31.874 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.885 7fa9c27fc700 1 -- 172.21.15.102:0/4079911139 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa9bc00f6f0 con 0x7fa9c410e260 2024-04-17T22:31:31.874 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.885 7fa9caf04700 1 -- 172.21.15.102:0/4079911139 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fa9c410e260 msgr2=0x7fa9c410e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:31.874 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.885 7fa9caf04700 1 --2- 172.21.15.102:0/4079911139 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fa9c410e260 0x7fa9c410e6b0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 rx=0x7fa9c40f4490 tx=0x7fa9bc009a30).stop 2024-04-17T22:31:31.874 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.886 7fa9caf04700 1 -- 172.21.15.102:0/4079911139 shutdown_connections 2024-04-17T22:31:31.874 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.886 7fa9caf04700 1 --2- 172.21.15.102:0/4079911139 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa9c40f1d80 0x7fa9c41aa4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:31.874 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.886 7fa9caf04700 1 --2- 172.21.15.102:0/4079911139 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fa9c410e260 0x7fa9c410e6b0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:31.874 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.886 7fa9caf04700 1 --2- 172.21.15.102:0/4079911139 >> v2:172.21.15.102:3300/0 conn(0x7fa9c410ece0 0x7fa9c40f1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:31.874 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.886 7fa9caf04700 1 -- 172.21.15.102:0/4079911139 >> 172.21.15.102:0/4079911139 conn(0x7fa9c41138c0 msgr2=0x7fa9c40f4f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:31.874 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.886 7fa9caf04700 1 -- 172.21.15.102:0/4079911139 shutdown_connections 2024-04-17T22:31:31.875 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.886 7fa9caf04700 1 -- 172.21.15.102:0/4079911139 wait complete. 2024-04-17T22:31:31.894 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.909 7fa9caf04700 1 Processor -- start 2024-04-17T22:31:31.895 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.909 7fa9caf04700 1 -- start start 2024-04-17T22:31:31.897 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.909 7fa9caf04700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fa9c40ca770 0x7fa9c40cb950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:31.897 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.909 7fa9caf04700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa9c40cbf80 0x7fa9c40d5000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:31.897 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.909 7fa9caf04700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa9c40cc3d0 0x7fa9c40d5630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:31.897 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.909 7fa9caf04700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9c408f2e0 con 0x7fa9c40cc3d0 2024-04-17T22:31:31.897 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.909 7fa9caf04700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9c408ed60 con 0x7fa9c40cbf80 2024-04-17T22:31:31.897 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.909 7fa9caf04700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa9c41a8960 con 0x7fa9c40ca770 2024-04-17T22:31:31.897 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.909 7fa9c2ffd700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa9c40cbf80 0x7fa9c40d5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:31.897 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.909 7fa9c3fff700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa9c40cc3d0 0x7fa9c40d5630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:31.897 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.909 7fa9c37fe700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fa9c40ca770 0x7fa9c40cb950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:31.897 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.909 7fa9c2ffd700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa9c40cbf80 0x7fa9c40d5000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:32894/0 (socket says 172.21.15.102:32894) 2024-04-17T22:31:31.897 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.909 7fa9c3fff700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa9c40cc3d0 0x7fa9c40d5630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:52182/0 (socket says 172.21.15.102:52182) 2024-04-17T22:31:31.898 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.909 7fa9c2ffd700 1 -- 172.21.15.102:0/3005492383 learned_addr learned my addr 172.21.15.102:0/3005492383 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:31:31.898 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.910 7fa9c2ffd700 1 -- 172.21.15.102:0/3005492383 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fa9c40ca770 msgr2=0x7fa9c40cb950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:31.898 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.910 7fa9c2ffd700 1 --2- 172.21.15.102:0/3005492383 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fa9c40ca770 0x7fa9c40cb950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:31.898 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.910 7fa9c2ffd700 1 -- 172.21.15.102:0/3005492383 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa9c40cc3d0 msgr2=0x7fa9c40d5630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:31.898 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.910 7fa9c2ffd700 1 --2- 172.21.15.102:0/3005492383 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa9c40cc3d0 0x7fa9c40d5630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:31.898 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.910 7fa9c2ffd700 1 -- 172.21.15.102:0/3005492383 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9c40caf90 con 0x7fa9c40cbf80 2024-04-17T22:31:31.898 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.910 7fa9c2ffd700 1 --2- 172.21.15.102:0/3005492383 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa9c40cbf80 0x7fa9c40d5000 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 rx=0x7fa9c40d7620 tx=0x7fa9ac00ee60).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:31.898 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.910 7fa9c0ff9700 1 -- 172.21.15.102:0/3005492383 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa9c40caf90 con 0x7fa9c40cbf80 2024-04-17T22:31:31.898 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.910 7fa9c0ff9700 1 -- 172.21.15.102:0/3005492383 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9ac00c950 con 0x7fa9c40cbf80 2024-04-17T22:31:31.898 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.910 7fa9c0ff9700 1 -- 172.21.15.102:0/3005492383 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fa9ac00f040 con 0x7fa9c40cbf80 2024-04-17T22:31:31.898 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.910 7fa9caf04700 1 -- 172.21.15.102:0/3005492383 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9c40d5c60 con 0x7fa9c40cbf80 2024-04-17T22:31:31.898 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.910 7fa9caf04700 1 -- 172.21.15.102:0/3005492383 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9c41bb820 con 0x7fa9c40cbf80 2024-04-17T22:31:31.898 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.910 7fa9c0ff9700 1 -- 172.21.15.102:0/3005492383 <== mon.1 v2:172.21.15.102:3301/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7fa9ac022060 con 0x7fa9c40cbf80 2024-04-17T22:31:31.899 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.910 7fa9c0ff9700 1 -- 172.21.15.102:0/3005492383 <== mon.1 v2:172.21.15.102:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 3194+0+0 (secure 0 0 0) 0x7fa9ac010040 con 0x7fa9c40cbf80 2024-04-17T22:31:31.899 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.911 7fa9caf04700 1 -- 172.21.15.102:0/3005492383 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9c40f0820 con 0x7fa9c40cbf80 2024-04-17T22:31:31.900 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:31.914 7fa9c0ff9700 1 -- 172.21.15.102:0/3005492383 <== mon.1 v2:172.21.15.102:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+110455 (secure 0 0 0) 0x7fa9ac00c740 con 0x7fa9c40cbf80 2024-04-17T22:31:32.033 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.047 7fa9caf04700 1 -- 172.21.15.102:0/3005492383 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa9c40f5550 con 0x7fa9c40cbf80 2024-04-17T22:31:32.034 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.048 7fa9c0ff9700 1 -- 172.21.15.102:0/3005492383 <== mon.1 v2:172.21.15.102:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) v1 ==== 94+0+1704 (secure 0 0 0) 0x7fa9ac000fb0 con 0x7fa9c40cbf80 2024-04-17T22:31:32.034 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-17T22:31:32.034 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":11,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[],"filesystems":[{"mdsmap":{"epoch":11,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17 22:31:28.204771","modified":"2024-04-17 22:31:30.213118","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"-1 (unspecified)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5319},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5319":{"gid":5319,"name":"a","rank":0,"incarnation":9,"state":"up:active","state_seq":3,"addr":"172.21.15.102:6817/479367134","addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6817","nonce":479367134}]},"export_targets":[],"features":4611087854035861503,"flags":0}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2024-04-17T22:31:32.035 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.049 7fa9caf04700 1 -- 172.21.15.102:0/3005492383 >> v1:172.21.15.102:6816/42360 conn(0x7fa9a80145a0 legacy=0x7fa9a8016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:32.035 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.049 7fa9caf04700 1 -- 172.21.15.102:0/3005492383 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa9c40cbf80 msgr2=0x7fa9c40d5000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:32.035 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.049 7fa9caf04700 1 --2- 172.21.15.102:0/3005492383 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa9c40cbf80 0x7fa9c40d5000 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 rx=0x7fa9c40d7620 tx=0x7fa9ac00ee60).stop 2024-04-17T22:31:32.035 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.049 7fa9caf04700 1 -- 172.21.15.102:0/3005492383 shutdown_connections 2024-04-17T22:31:32.036 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.049 7fa9caf04700 1 --2- 172.21.15.102:0/3005492383 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa9c40cc3d0 0x7fa9c40d5630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:32.036 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.049 7fa9caf04700 1 --2- 172.21.15.102:0/3005492383 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fa9c40ca770 0x7fa9c40cb950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:32.036 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.049 7fa9caf04700 1 --2- 172.21.15.102:0/3005492383 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa9c40cbf80 0x7fa9c40d5000 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:32.036 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.049 7fa9caf04700 1 -- 172.21.15.102:0/3005492383 >> 172.21.15.102:0/3005492383 conn(0x7fa9c40c5480 msgr2=0x7fa9c40d6030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:32.036 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.049 7fa9caf04700 1 -- 172.21.15.102:0/3005492383 shutdown_connections 2024-04-17T22:31:32.036 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.049 7fa9caf04700 1 -- 172.21.15.102:0/3005492383 wait complete. 2024-04-17T22:31:32.036 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 11 2024-04-17T22:31:32.147 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 11, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-17 22:31:28.204771', 'modified': '2024-04-17 22:31:30.213118', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '-1 (unspecified)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5319}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5319': {'gid': 5319, 'name': 'a', 'rank': 0, 'incarnation': 9, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.102:6817/479367134', 'addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.102:6817', 'nonce': 479367134}]}, 'export_targets': [], 'features': 4611087854035861503, 'flags': 0}}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2024-04-17T22:31:32.147 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-17T22:31:32.147 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-17T22:31:32.276 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.288 7fbb1ad38700 1 Processor -- start 2024-04-17T22:31:32.276 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.289 7fbb1ad38700 1 -- start start 2024-04-17T22:31:32.277 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.289 7fbb1ad38700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fbb1410e240 0x7fbb1410e690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:32.277 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.289 7fbb1ad38700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fbb1410ecc0 0x7fbb140f1730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:32.277 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.289 7fbb1ad38700 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb140f21b0 con 0x7fbb140f1d60 2024-04-17T22:31:32.277 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.289 7fbb1ad38700 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb140f9f10 con 0x7fbb1410ecc0 2024-04-17T22:31:32.277 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.289 7fbb1ad38700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbb140f9030 con 0x7fbb1410e240 2024-04-17T22:31:32.277 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.289 7fbb12ffd700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fbb1410ecc0 0x7fbb140f1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:32.277 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.289 7fbb137fe700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fbb1410e240 0x7fbb1410e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:32.277 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.289 7fbb12ffd700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fbb1410ecc0 0x7fbb140f1730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:52198/0 (socket says 172.21.15.102:52198) 2024-04-17T22:31:32.277 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.289 7fbb13fff700 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7fbb140f1d60 0x7fbb141aa4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:33830/0 (socket says 172.21.15.102:33830) 2024-04-17T22:31:32.277 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.289 7fbb12ffd700 1 -- 172.21.15.102:0/2793600714 learned_addr learned my addr 172.21.15.102:0/2793600714 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:31:32.277 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.289 7fbb12ffd700 1 -- 172.21.15.102:0/2793600714 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fbb1410e240 msgr2=0x7fbb1410e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:31:32.277 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.289 7fbb12ffd700 1 --2- 172.21.15.102:0/2793600714 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fbb1410e240 0x7fbb1410e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:32.277 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.289 7fbb12ffd700 1 -- 172.21.15.102:0/2793600714 >> v1:172.21.15.102:6789/0 conn(0x7fbb140f1d60 legacy=0x7fbb141aa4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:32.278 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.289 7fbb12ffd700 1 -- 172.21.15.102:0/2793600714 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb140f23d0 con 0x7fbb1410ecc0 2024-04-17T22:31:32.278 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.289 7fbb127fc700 1 -- 172.21.15.102:0/2793600714 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3717859131 0 0) 0x7fbb0c001490 con 0x7fbb140f1d60 2024-04-17T22:31:32.278 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.289 7fbb137fe700 1 --2- 172.21.15.102:0/2793600714 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fbb1410e240 0x7fbb1410e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:32.278 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.289 7fbb12ffd700 1 --2- 172.21.15.102:0/2793600714 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbb1410ecc0 0x7fbb140f1730 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 rx=0x7fbb14103580 tx=0x7fbb04009a80).ready entity=mon.0 client_cookie=f2c453d323a8ff25 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:32.278 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.290 7fbb127fc700 1 -- 172.21.15.102:0/2793600714 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fbb140f23d0 con 0x7fbb1410ecc0 2024-04-17T22:31:32.278 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.290 7fbb127fc700 1 -- 172.21.15.102:0/2793600714 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb0400f510 con 0x7fbb1410ecc0 2024-04-17T22:31:32.278 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.290 7fbb127fc700 1 -- 172.21.15.102:0/2793600714 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fbb0400f740 con 0x7fbb1410ecc0 2024-04-17T22:31:32.278 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.290 7fbb1ad38700 1 -- 172.21.15.102:0/2793600714 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbb1410ecc0 msgr2=0x7fbb140f1730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:32.278 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.290 7fbb1ad38700 1 --2- 172.21.15.102:0/2793600714 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbb1410ecc0 0x7fbb140f1730 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 rx=0x7fbb14103580 tx=0x7fbb04009a80).stop 2024-04-17T22:31:32.278 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.290 7fbb1ad38700 1 -- 172.21.15.102:0/2793600714 shutdown_connections 2024-04-17T22:31:32.278 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.290 7fbb1ad38700 1 --2- 172.21.15.102:0/2793600714 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fbb1410e240 0x7fbb1410e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:32.278 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.290 7fbb1ad38700 1 --2- 172.21.15.102:0/2793600714 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbb1410ecc0 0x7fbb140f1730 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:32.278 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.290 7fbb1ad38700 1 -- 172.21.15.102:0/2793600714 >> 172.21.15.102:0/2793600714 conn(0x7fbb141138a0 msgr2=0x7fbb140f4f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:32.278 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.290 7fbb1ad38700 1 -- 172.21.15.102:0/2793600714 shutdown_connections 2024-04-17T22:31:32.279 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.290 7fbb1ad38700 1 -- 172.21.15.102:0/2793600714 wait complete. 2024-04-17T22:31:32.301 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.313 7fbb1ad38700 1 Processor -- start 2024-04-17T22:31:32.301 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.313 7fbb1ad38700 1 -- start start 2024-04-17T22:31:32.301 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.313 7fbb1ad38700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbb140dd5f0 0x7fbb140d60a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:32.301 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.313 7fbb1ad38700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fbb140d66d0 0x7fbb141b3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:32.301 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.313 7fbb1ad38700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fbb140d6b20 0x7fbb141b4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:32.301 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.313 7fbb1ad38700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb140fa1b0 con 0x7fbb140dd5f0 2024-04-17T22:31:32.301 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.313 7fbb1ad38700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb141a9040 con 0x7fbb140d66d0 2024-04-17T22:31:32.301 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.313 7fbb1ad38700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbb141b4c30 con 0x7fbb140d6b20 2024-04-17T22:31:32.301 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.314 7fbb137fe700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbb140dd5f0 0x7fbb140d60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:32.302 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.314 7fbb13fff700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fbb140d6b20 0x7fbb141b4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:32.302 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.314 7fbb137fe700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbb140dd5f0 0x7fbb140d60a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:52212/0 (socket says 172.21.15.102:52212) 2024-04-17T22:31:32.302 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.314 7fbb13fff700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fbb140d6b20 0x7fbb141b4600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:33870/0 (socket says 172.21.15.102:33870) 2024-04-17T22:31:32.302 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.314 7fbb12ffd700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fbb140d66d0 0x7fbb141b3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:32.302 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.314 7fbb137fe700 1 -- 172.21.15.102:0/637818998 learned_addr learned my addr 172.21.15.102:0/637818998 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:31:32.302 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.314 7fbb137fe700 1 -- 172.21.15.102:0/637818998 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fbb140d6b20 msgr2=0x7fbb141b4600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:32.302 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.314 7fbb137fe700 1 --2- 172.21.15.102:0/637818998 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fbb140d6b20 0x7fbb141b4600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:32.302 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.314 7fbb137fe700 1 -- 172.21.15.102:0/637818998 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fbb140d66d0 msgr2=0x7fbb141b3fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:32.302 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.314 7fbb137fe700 1 --2- 172.21.15.102:0/637818998 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fbb140d66d0 0x7fbb141b3fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:32.302 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.314 7fbb137fe700 1 -- 172.21.15.102:0/637818998 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb141b4e60 con 0x7fbb140dd5f0 2024-04-17T22:31:32.302 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.314 7fbb13fff700 1 --2- 172.21.15.102:0/637818998 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fbb140d6b20 0x7fbb141b4600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:32.302 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.314 7fbb12ffd700 1 --2- 172.21.15.102:0/637818998 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fbb140d66d0 0x7fbb141b3fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:32.302 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.314 7fbb137fe700 1 --2- 172.21.15.102:0/637818998 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbb140dd5f0 0x7fbb140d60a0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 rx=0x7fbb140c4890 tx=0x7fbb0800bf70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:32.302 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.314 7fbb10ff9700 1 -- 172.21.15.102:0/637818998 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fbb141b4e60 con 0x7fbb140dd5f0 2024-04-17T22:31:32.303 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.314 7fbb10ff9700 1 -- 172.21.15.102:0/637818998 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb0800c9a0 con 0x7fbb140dd5f0 2024-04-17T22:31:32.303 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.314 7fbb1ad38700 1 -- 172.21.15.102:0/637818998 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb141bb6c0 con 0x7fbb140dd5f0 2024-04-17T22:31:32.303 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.314 7fbb10ff9700 1 -- 172.21.15.102:0/637818998 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fbb0800f040 con 0x7fbb140dd5f0 2024-04-17T22:31:32.303 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.314 7fbb1ad38700 1 -- 172.21.15.102:0/637818998 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb141bbaa0 con 0x7fbb140dd5f0 2024-04-17T22:31:32.303 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.315 7fbb10ff9700 1 -- 172.21.15.102:0/637818998 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7fbb08011070 con 0x7fbb140dd5f0 2024-04-17T22:31:32.303 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.315 7fbb10ff9700 1 -- 172.21.15.102:0/637818998 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 3194+0+0 (secure 0 0 0) 0x7fbb08010040 con 0x7fbb140dd5f0 2024-04-17T22:31:32.303 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.317 7fbb2d5f1b80 1 Processor -- start 2024-04-17T22:31:32.303 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.317 7fbb2d5f1b80 1 -- start start 2024-04-17T22:31:32.310 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.318 7fbb2d5f1b80 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x55b568aed220 0x55b568aed670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:32.310 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.318 7fbb2d5f1b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b568aedca0 0x55b568ae08e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:32.310 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.318 7fbb2d5f1b80 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x55b568ae0f10 0x55b568ae1360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:32.310 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.318 7fbb2d5f1b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b568a18c80 con 0x55b568aedca0 2024-04-17T22:31:32.310 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.318 7fbb2d5f1b80 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b568a36a40 con 0x55b568ae0f10 2024-04-17T22:31:32.310 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.318 7fbb2d5f1b80 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x55b568a03e00 con 0x55b568aed220 2024-04-17T22:31:32.310 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.318 7fbb137fe700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x55b568aed220 0x55b568aed670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:32.310 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.318 7fbb12ffd700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b568aedca0 0x55b568ae08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:32.310 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.318 7fbb137fe700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x55b568aed220 0x55b568aed670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:33876/0 (socket says 172.21.15.102:33876) 2024-04-17T22:31:32.311 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.318 7fbb137fe700 1 -- 172.21.15.102:0/1834213198 learned_addr learned my addr 172.21.15.102:0/1834213198 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:31:32.311 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.318 7fbb13fff700 1 --2- 172.21.15.102:0/1834213198 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x55b568ae0f10 0x55b568ae1360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:32.311 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.318 7fbb12ffd700 1 -- 172.21.15.102:0/1834213198 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x55b568aed220 msgr2=0x55b568aed670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:31:32.311 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.318 7fbb12ffd700 1 --2- 172.21.15.102:0/1834213198 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x55b568aed220 0x55b568aed670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:32.311 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.318 7fbb12ffd700 1 -- 172.21.15.102:0/1834213198 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x55b568ae0f10 msgr2=0x55b568ae1360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:31:32.311 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.318 7fbb12ffd700 1 --2- 172.21.15.102:0/1834213198 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x55b568ae0f10 0x55b568ae1360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:32.311 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.318 7fbb12ffd700 1 -- 172.21.15.102:0/1834213198 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b568ab63a0 con 0x55b568aedca0 2024-04-17T22:31:32.311 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.318 7fbb137fe700 1 --2- 172.21.15.102:0/1834213198 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x55b568aed220 0x55b568aed670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:32.311 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.319 7fbb12ffd700 1 --2- 172.21.15.102:0/1834213198 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b568aedca0 0x55b568ae08e0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 rx=0x55b568a294f0 tx=0x7fbb04006830).ready entity=mon.0 client_cookie=8b3153d5fa2b182e server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:32.311 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.319 7fbb1ad38700 1 -- 172.21.15.102:0/1834213198 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x55b568ab63a0 con 0x55b568aedca0 2024-04-17T22:31:32.311 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.319 7fbb1ad38700 1 -- 172.21.15.102:0/1834213198 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb0400fbe0 con 0x55b568aedca0 2024-04-17T22:31:32.311 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.319 7fbb1ad38700 1 -- 172.21.15.102:0/1834213198 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fbb0401e430 con 0x55b568aedca0 2024-04-17T22:31:32.311 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.319 7fbb2d5f1b80 1 -- 172.21.15.102:0/1834213198 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b568aedca0 msgr2=0x55b568ae08e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:32.312 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.319 7fbb2d5f1b80 1 --2- 172.21.15.102:0/1834213198 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b568aedca0 0x55b568ae08e0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 rx=0x55b568a294f0 tx=0x7fbb04006830).stop 2024-04-17T22:31:32.312 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.319 7fbb2d5f1b80 1 -- 172.21.15.102:0/1834213198 shutdown_connections 2024-04-17T22:31:32.312 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.319 7fbb2d5f1b80 1 --2- 172.21.15.102:0/1834213198 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x55b568ae0f10 0x55b568ae1360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:32.312 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.319 7fbb2d5f1b80 1 --2- 172.21.15.102:0/1834213198 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x55b568aed220 0x55b568aed670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:32.312 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.319 7fbb2d5f1b80 1 --2- 172.21.15.102:0/1834213198 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b568aedca0 0x55b568ae08e0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:32.312 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.319 7fbb2d5f1b80 1 -- 172.21.15.102:0/1834213198 >> 172.21.15.102:0/1834213198 conn(0x55b568ad9cc0 msgr2=0x55b568afb200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:32.312 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.319 7fbb2d5f1b80 1 -- 172.21.15.102:0/1834213198 shutdown_connections 2024-04-17T22:31:32.312 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.319 7fbb2d5f1b80 1 -- 172.21.15.102:0/1834213198 wait complete. 2024-04-17T22:31:32.312 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.320 7fbb2d5f1b80 1 -- start start 2024-04-17T22:31:32.312 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.320 7fbb2d5f1b80 1 Processor -- start 2024-04-17T22:31:32.312 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.320 7fbb2d5f1b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b568aee9b0 0x55b568aef040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:32.312 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.320 7fbb2d5f1b80 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x55b568aef670 0x55b568b088a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:32.312 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.320 7fbb2d5f1b80 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x55b568aefac0 0x55b568b08ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:32.312 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.320 7fbb2d5f1b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b568ae5d30 con 0x55b568aee9b0 2024-04-17T22:31:32.313 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.320 7fbb2d5f1b80 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b568ae59c0 con 0x55b568aefac0 2024-04-17T22:31:32.313 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.320 7fbb2d5f1b80 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x55b568b09500 con 0x55b568aef670 2024-04-17T22:31:32.313 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.320 7fbb2d5f1b80 10 client.0 ll_register_callbacks cb 0x55b568a0ed30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T22:31:32.313 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.320 7fbb137fe700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b568aee9b0 0x55b568aef040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:32.313 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.320 7fbb137fe700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b568aee9b0 0x55b568aef040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:52238/0 (socket says 172.21.15.102:52238) 2024-04-17T22:31:32.313 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.320 7fbb137fe700 1 -- 172.21.15.102:0/2394642787 learned_addr learned my addr 172.21.15.102:0/2394642787 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:31:32.313 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.320 7fbb13fff700 1 --2- 172.21.15.102:0/2394642787 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x55b568aefac0 0x55b568b08ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:32.313 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.320 7fbb12ffd700 1 --2- 172.21.15.102:0/2394642787 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x55b568aef670 0x55b568b088a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:32.313 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.321 7fbb137fe700 1 -- 172.21.15.102:0/2394642787 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x55b568aef670 msgr2=0x55b568b088a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:31:32.313 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.321 7fbb137fe700 1 --2- 172.21.15.102:0/2394642787 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x55b568aef670 0x55b568b088a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:32.313 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.321 7fbb137fe700 1 -- 172.21.15.102:0/2394642787 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x55b568aefac0 msgr2=0x55b568b08ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:31:32.313 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.321 7fbb137fe700 1 --2- 172.21.15.102:0/2394642787 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x55b568aefac0 0x55b568b08ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:32.313 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.321 7fbb137fe700 1 -- 172.21.15.102:0/2394642787 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b568926b60 con 0x55b568aee9b0 2024-04-17T22:31:32.313 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.321 7fbb137fe700 1 --2- 172.21.15.102:0/2394642787 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b568aee9b0 0x55b568aef040 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 rx=0x7fbb080206f0 tx=0x7fbb08021c80).ready entity=mon.0 client_cookie=34aae92d2eba68fa server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:32.314 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.321 7fbaee7fc700 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-04-17T22:31:32.314 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.321 7fbaee7fc700 1 -- 172.21.15.102:0/2394642787 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x55b568926b60 con 0x55b568aee9b0 2024-04-17T22:31:32.314 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.321 7fbaee7fc700 1 -- 172.21.15.102:0/2394642787 <== mon.0 v2:172.21.15.102:3300/0 2 ==== osd_map(25..25 src has 1..25) v4 ==== 3194+0+0 (secure 0 0 0) 0x7fbb080215f0 con 0x55b568aee9b0 2024-04-17T22:31:32.314 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.321 7fbb2d5f1b80 1 -- 172.21.15.102:0/2394642787 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b568a21cd0 con 0x55b568aee9b0 2024-04-17T22:31:32.314 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.321 7fbaee7fc700 1 -- 172.21.15.102:0/2394642787 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=11) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbb08011e30 con 0x55b568aee9b0 2024-04-17T22:31:32.314 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.321 7fbb2d5f1b80 10 client.5544 fetch_fsmap learned FSMap version 11 2024-04-17T22:31:32.314 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.321 7fbb2d5f1b80 1 -- 172.21.15.102:0/2394642787 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=11,monmap=2+}) v3 -- 0x55b568b10250 con 0x55b568aee9b0 2024-04-17T22:31:32.314 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.321 7fbaee7fc700 1 -- 172.21.15.102:0/2394642787 <== mon.0 v2:172.21.15.102:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b568a21cd0 con 0x55b568aee9b0 2024-04-17T22:31:32.314 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.321 7fbaee7fc700 1 -- 172.21.15.102:0/2394642787 <== mon.0 v2:172.21.15.102:3300/0 5 ==== fsmap(e 11) v1 ==== 1025+0+0 (secure 0 0 0) 0x7fbb08026040 con 0x55b568aee9b0 2024-04-17T22:31:32.314 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.321 7fbb2d5f1b80 10 client.5544 fetch_fsmap finished waiting for FSMap version 11 2024-04-17T22:31:32.314 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.321 7fbb2d5f1b80 10 client.5544 resolve_mds: resolved ID 'a' to GID 5319 2024-04-17T22:31:32.314 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.321 7fbb2d5f1b80 20 client.5544 populate_metadata read hostname 'smithi102' 2024-04-17T22:31:32.314 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.321 7fbb2d5f1b80 4 client.5544 mds_command: new command op to 5319 tid=0[{"prefix": "get_command_descriptions"}] 2024-04-17T22:31:32.314 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.321 7fbb2d5f1b80 1 -- 172.21.15.102:0/2394642787 --> v1:172.21.15.102:6817/479367134 -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b56891aba0 con 0x55b568b107b0 2024-04-17T22:31:32.315 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.321 7fbaee7fc700 10 client.5544 ms_handle_connect on v1:172.21.15.102:6817/479367134 2024-04-17T22:31:32.315 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.322 7fbaee7fc700 1 -- 172.21.15.102:0/2394642787 <== mds.0 v1:172.21.15.102:6817/479367134 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3729 (unknown 0 0 338808096) 0x55b56891aba0 con 0x55b568b107b0 2024-04-17T22:31:32.315 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.322 7fbaee7fc700 10 client.5544 handle_command_reply: tid=0 2024-04-17T22:31:32.315 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.322 7fbb2d5f1b80 1 client.5544 shutdown 2024-04-17T22:31:32.315 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.322 7fbb2d5f1b80 1 -- 172.21.15.102:0/2394642787 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b568aee9b0 msgr2=0x55b568aef040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:32.315 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.322 7fbb2d5f1b80 1 --2- 172.21.15.102:0/2394642787 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b568aee9b0 0x55b568aef040 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 rx=0x7fbb080206f0 tx=0x7fbb08021c80).stop 2024-04-17T22:31:32.315 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.322 7fbb2d5f1b80 1 -- 172.21.15.102:0/2394642787 shutdown_connections 2024-04-17T22:31:32.315 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.322 7fbb2d5f1b80 1 --2- 172.21.15.102:0/2394642787 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x55b568aefac0 0x55b568b08ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:32.315 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.322 7fbb2d5f1b80 1 --2- 172.21.15.102:0/2394642787 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b568aee9b0 0x55b568aef040 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:32.315 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.322 7fbb2d5f1b80 1 --2- 172.21.15.102:0/2394642787 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x55b568aef670 0x55b568b088a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:32.315 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.322 7fbaee7fc700 0 client.5544 ms_handle_reset on v1:172.21.15.102:6817/479367134 2024-04-17T22:31:32.315 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.322 7fbb2d5f1b80 1 -- 172.21.15.102:0/2394642787 >> 172.21.15.102:0/2394642787 conn(0x55b568adef50 msgr2=0x55b568afb5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:32.315 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.322 7fbb2d5f1b80 1 -- 172.21.15.102:0/2394642787 shutdown_connections 2024-04-17T22:31:32.315 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.322 7fbb2d5f1b80 1 -- 172.21.15.102:0/2394642787 wait complete. 2024-04-17T22:31:32.316 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.322 7fbb2d5f1b80 20 client.5544 trim_cache size 0 max 16384 2024-04-17T22:31:32.318 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.332 7fbaf8ff9700 1 -- 172.21.15.102:0/637818998 >> v1:172.21.15.102:6816/42360 conn(0x7fbaf40145a0 legacy=0x7fbaf4016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:32.318 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.332 7fbaf8ff9700 1 -- 172.21.15.102:0/637818998 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbb140dd5f0 msgr2=0x7fbb140d60a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:32.318 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.332 7fbaf8ff9700 1 --2- 172.21.15.102:0/637818998 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbb140dd5f0 0x7fbb140d60a0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 rx=0x7fbb140c4890 tx=0x7fbb0800bf70).stop 2024-04-17T22:31:32.318 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.332 7fbaf8ff9700 1 -- 172.21.15.102:0/637818998 shutdown_connections 2024-04-17T22:31:32.319 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.332 7fbaf8ff9700 1 --2- 172.21.15.102:0/637818998 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fbb140d6b20 0x7fbb141b4600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:32.319 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.332 7fbaf8ff9700 1 --2- 172.21.15.102:0/637818998 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbb140dd5f0 0x7fbb140d60a0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:32.319 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.332 7fbaf8ff9700 1 --2- 172.21.15.102:0/637818998 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fbb140d66d0 0x7fbb141b3fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:32.319 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.332 7fbaf8ff9700 1 -- 172.21.15.102:0/637818998 >> 172.21.15.102:0/637818998 conn(0x7fbb140c6a60 msgr2=0x7fbb140c3070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:32.319 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.332 7fbaf8ff9700 1 -- 172.21.15.102:0/637818998 shutdown_connections 2024-04-17T22:31:32.319 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.332 7fbaf8ff9700 1 -- 172.21.15.102:0/637818998 wait complete. 2024-04-17T22:31:32.319 INFO:teuthology.orchestra.run.smithi102.stderr:no valid command found; 10 closest matches: 2024-04-17T22:31:32.319 INFO:teuthology.orchestra.run.smithi102.stderr:injectargs [...] 2024-04-17T22:31:32.319 INFO:teuthology.orchestra.run.smithi102.stderr:config set 2024-04-17T22:31:32.319 INFO:teuthology.orchestra.run.smithi102.stderr:config unset 2024-04-17T22:31:32.320 INFO:teuthology.orchestra.run.smithi102.stderr:exit 2024-04-17T22:31:32.320 INFO:teuthology.orchestra.run.smithi102.stderr:respawn 2024-04-17T22:31:32.320 INFO:teuthology.orchestra.run.smithi102.stderr:session kill 2024-04-17T22:31:32.320 INFO:teuthology.orchestra.run.smithi102.stderr:cpu_profiler status|flush 2024-04-17T22:31:32.320 INFO:teuthology.orchestra.run.smithi102.stderr:session ls { [...]} 2024-04-17T22:31:32.320 INFO:teuthology.orchestra.run.smithi102.stderr:client ls { [...]} 2024-04-17T22:31:32.320 INFO:teuthology.orchestra.run.smithi102.stderr:session evict { [...]} 2024-04-17T22:31:32.320 INFO:teuthology.orchestra.run.smithi102.stderr:Error EINVAL: invalid command 2024-04-17T22:31:32.328 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-17T22:31:32.329 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-17T22:31:32.453 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.467 7fb25075f700 1 Processor -- start 2024-04-17T22:31:32.453 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.467 7fb25075f700 1 -- start start 2024-04-17T22:31:32.456 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.468 7fb25075f700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb24810e260 0x7fb24810e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:32.456 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.468 7fb25075f700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb24810ece0 0x7fb2480f1750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:32.456 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.468 7fb25075f700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fb2480f1d80 0x7fb2481aa4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:32.457 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.468 7fb25075f700 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2480f9f30 con 0x7fb2480f1d80 2024-04-17T22:31:32.457 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.468 7fb25075f700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2480f9050 con 0x7fb24810e260 2024-04-17T22:31:32.457 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.468 7fb25075f700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb2480fa1d0 con 0x7fb24810ece0 2024-04-17T22:31:32.457 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.468 7fb24cd59700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb24810ece0 0x7fb2480f1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:32.457 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.468 7fb24d55a700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb24810e260 0x7fb24810e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:32.457 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.468 7fb24dd5b700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fb2480f1d80 0x7fb2481aa4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:32.457 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.468 7fb24cd59700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb24810ece0 0x7fb2480f1750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:33898/0 (socket says 172.21.15.102:33898) 2024-04-17T22:31:32.457 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.468 7fb24d55a700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb24810e260 0x7fb24810e6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:32930/0 (socket says 172.21.15.102:32930) 2024-04-17T22:31:32.457 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.468 7fb24cd59700 1 -- 172.21.15.102:0/900858449 learned_addr learned my addr 172.21.15.102:0/900858449 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:31:32.457 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.468 7fb24cd59700 1 -- 172.21.15.102:0/900858449 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb24810e260 msgr2=0x7fb24810e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:31:32.457 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.468 7fb24cd59700 1 --2- 172.21.15.102:0/900858449 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb24810e260 0x7fb24810e6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:32.457 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.468 7fb24cd59700 1 -- 172.21.15.102:0/900858449 >> v2:172.21.15.102:3300/0 conn(0x7fb2480f1d80 msgr2=0x7fb2481aa4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:31:32.458 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.468 7fb24cd59700 1 --2- 172.21.15.102:0/900858449 >> v2:172.21.15.102:3300/0 conn(0x7fb2480f1d80 0x7fb2481aa4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:32.458 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.468 7fb24cd59700 1 -- 172.21.15.102:0/900858449 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2480f21d0 con 0x7fb24810ece0 2024-04-17T22:31:32.458 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.468 7fb24dd5b700 1 --2- 172.21.15.102:0/900858449 >> v2:172.21.15.102:3300/0 conn(0x7fb2480f1d80 0x7fb2481aa4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:32.458 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.468 7fb24cd59700 1 --2- 172.21.15.102:0/900858449 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb24810ece0 0x7fb2480f1750 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 rx=0x7fb2481035a0 tx=0x7fb244009a00).ready entity=mon.2 client_cookie=5a4136faf5fdb8bb server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:32.458 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.468 7fb237fff700 1 -- 172.21.15.102:0/900858449 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb2480f21d0 con 0x7fb24810ece0 2024-04-17T22:31:32.458 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.468 7fb237fff700 1 -- 172.21.15.102:0/900858449 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb24400f510 con 0x7fb24810ece0 2024-04-17T22:31:32.458 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.468 7fb237fff700 1 -- 172.21.15.102:0/900858449 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb24400f6f0 con 0x7fb24810ece0 2024-04-17T22:31:32.458 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.468 7fb25075f700 1 -- 172.21.15.102:0/900858449 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb24810ece0 msgr2=0x7fb2480f1750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:32.458 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.468 7fb25075f700 1 --2- 172.21.15.102:0/900858449 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb24810ece0 0x7fb2480f1750 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 rx=0x7fb2481035a0 tx=0x7fb244009a00).stop 2024-04-17T22:31:32.459 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.469 7fb25075f700 1 -- 172.21.15.102:0/900858449 shutdown_connections 2024-04-17T22:31:32.459 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.469 7fb25075f700 1 --2- 172.21.15.102:0/900858449 >> v2:172.21.15.102:3300/0 conn(0x7fb2480f1d80 0x7fb2481aa4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:32.459 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.469 7fb25075f700 1 --2- 172.21.15.102:0/900858449 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb24810e260 0x7fb24810e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:32.459 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.469 7fb25075f700 1 --2- 172.21.15.102:0/900858449 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb24810ece0 0x7fb2480f1750 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:32.459 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.469 7fb25075f700 1 -- 172.21.15.102:0/900858449 >> 172.21.15.102:0/900858449 conn(0x7fb2481138c0 msgr2=0x7fb2480f4f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:32.459 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.469 7fb25075f700 1 -- 172.21.15.102:0/900858449 shutdown_connections 2024-04-17T22:31:32.459 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.469 7fb25075f700 1 -- 172.21.15.102:0/900858449 wait complete. 2024-04-17T22:31:32.480 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.492 7fb25075f700 1 Processor -- start 2024-04-17T22:31:32.480 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.492 7fb25075f700 1 -- start start 2024-04-17T22:31:32.480 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.492 7fb25075f700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb2480dfc20 0x7fb2481ad580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:32.480 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.492 7fb25075f700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb2481adbb0 0x7fb2481b3b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:32.480 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.492 7fb25075f700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb2481b41b0 0x7fb2481b4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:32.480 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.492 7fb25075f700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb24808f2e0 con 0x7fb2480dfc20 2024-04-17T22:31:32.481 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.492 7fb25075f700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2480e0070 con 0x7fb2481adbb0 2024-04-17T22:31:32.481 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.492 7fb25075f700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb2481b4c30 con 0x7fb2481b41b0 2024-04-17T22:31:32.481 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.492 7fb24cd59700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb2481adbb0 0x7fb2481b3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:32.481 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.492 7fb24d55a700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb2480dfc20 0x7fb2481ad580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:32.481 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.492 7fb24cd59700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb2481adbb0 0x7fb2481b3b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:32934/0 (socket says 172.21.15.102:32934) 2024-04-17T22:31:32.481 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.492 7fb24cd59700 1 -- 172.21.15.102:0/2308826200 learned_addr learned my addr 172.21.15.102:0/2308826200 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:31:32.481 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.492 7fb24dd5b700 1 --2- 172.21.15.102:0/2308826200 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb2481b41b0 0x7fb2481b4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:32.481 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.493 7fb24cd59700 1 -- 172.21.15.102:0/2308826200 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb2481b41b0 msgr2=0x7fb2481b4600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:32.481 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.493 7fb24cd59700 1 --2- 172.21.15.102:0/2308826200 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb2481b41b0 0x7fb2481b4600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:32.481 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.493 7fb24cd59700 1 -- 172.21.15.102:0/2308826200 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb2480dfc20 msgr2=0x7fb2481ad580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:32.481 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.493 7fb24cd59700 1 --2- 172.21.15.102:0/2308826200 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb2480dfc20 0x7fb2481ad580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:32.481 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.493 7fb24cd59700 1 -- 172.21.15.102:0/2308826200 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2481ae000 con 0x7fb2481adbb0 2024-04-17T22:31:32.481 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.493 7fb24d55a700 1 --2- 172.21.15.102:0/2308826200 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb2480dfc20 0x7fb2481ad580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:32.482 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.493 7fb24dd5b700 1 --2- 172.21.15.102:0/2308826200 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb2481b41b0 0x7fb2481b4600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:32.482 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.493 7fb24cd59700 1 --2- 172.21.15.102:0/2308826200 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb2481adbb0 0x7fb2481b3b80 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 rx=0x7fb2480d6530 tx=0x7fb2440068c0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:32.482 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.493 7fb2367fc700 1 -- 172.21.15.102:0/2308826200 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb2481ae000 con 0x7fb2481adbb0 2024-04-17T22:31:32.482 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.493 7fb2367fc700 1 -- 172.21.15.102:0/2308826200 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb24401c650 con 0x7fb2481adbb0 2024-04-17T22:31:32.482 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.493 7fb2367fc700 1 -- 172.21.15.102:0/2308826200 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb24401ca20 con 0x7fb2481adbb0 2024-04-17T22:31:32.482 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.493 7fb25075f700 1 -- 172.21.15.102:0/2308826200 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2480d1130 con 0x7fb2481adbb0 2024-04-17T22:31:32.482 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.493 7fb25075f700 1 -- 172.21.15.102:0/2308826200 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2481bb240 con 0x7fb2481adbb0 2024-04-17T22:31:32.482 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.493 7fb2367fc700 1 -- 172.21.15.102:0/2308826200 <== mon.1 v2:172.21.15.102:3301/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7fb244019030 con 0x7fb2481adbb0 2024-04-17T22:31:32.482 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.494 7fb2367fc700 1 -- 172.21.15.102:0/2308826200 <== mon.1 v2:172.21.15.102:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 3194+0+0 (secure 0 0 0) 0x7fb244018040 con 0x7fb2481adbb0 2024-04-17T22:31:32.482 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.494 7fb25075f700 1 -- 172.21.15.102:0/2308826200 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2480c4070 con 0x7fb2481adbb0 2024-04-17T22:31:32.483 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.497 7fb2367fc700 1 -- 172.21.15.102:0/2308826200 <== mon.1 v2:172.21.15.102:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+110455 (secure 0 0 0) 0x7fb24401c440 con 0x7fb2481adbb0 2024-04-17T22:31:32.618 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.632 7fb25075f700 1 -- 172.21.15.102:0/2308826200 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fb2480f0820 con 0x7fb2481adbb0 2024-04-17T22:31:32.618 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.632 7fb2367fc700 1 -- 172.21.15.102:0/2308826200 <== mon.1 v2:172.21.15.102:3301/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v11) v1 ==== 53+0+73 (secure 0 0 0) 0x7fb24400f300 con 0x7fb2481adbb0 2024-04-17T22:31:32.619 INFO:teuthology.orchestra.run.smithi102.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-17T22:31:32.620 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.633 7fb25075f700 1 -- 172.21.15.102:0/2308826200 >> v1:172.21.15.102:6816/42360 conn(0x7fb2280145a0 legacy=0x7fb228016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:32.620 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.633 7fb25075f700 1 -- 172.21.15.102:0/2308826200 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb2481adbb0 msgr2=0x7fb2481b3b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:32.620 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.633 7fb25075f700 1 --2- 172.21.15.102:0/2308826200 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb2481adbb0 0x7fb2481b3b80 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 rx=0x7fb2480d6530 tx=0x7fb2440068c0).stop 2024-04-17T22:31:32.620 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.634 7fb25075f700 1 -- 172.21.15.102:0/2308826200 shutdown_connections 2024-04-17T22:31:32.620 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.634 7fb25075f700 1 --2- 172.21.15.102:0/2308826200 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb2481b41b0 0x7fb2481b4600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:32.620 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.634 7fb25075f700 1 --2- 172.21.15.102:0/2308826200 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb2480dfc20 0x7fb2481ad580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:32.620 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.634 7fb25075f700 1 --2- 172.21.15.102:0/2308826200 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb2481adbb0 0x7fb2481b3b80 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:32.620 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.634 7fb25075f700 1 -- 172.21.15.102:0/2308826200 >> 172.21.15.102:0/2308826200 conn(0x7fb2481a7050 msgr2=0x7fb2480d6980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:32.621 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.634 7fb25075f700 1 -- 172.21.15.102:0/2308826200 shutdown_connections 2024-04-17T22:31:32.621 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:32.634 7fb25075f700 1 -- 172.21.15.102:0/2308826200 wait complete. 2024-04-17T22:31:32.630 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-17T22:31:32.631 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-17T22:31:32.631 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi190.front.sepia.ceph.com 2024-04-17T22:31:32.631 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-17T22:31:32.631 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-17T22:31:32.631 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-17T22:31:32.631 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-17T22:31:32.631 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-17T22:31:32.631 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-17T22:31:32.631 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T22:31:32.631 DEBUG:teuthology.orchestra.run.smithi190:> ip addr 2024-04-17T22:31:32.654 INFO:teuthology.orchestra.run.smithi190.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-17T22:31:32.654 INFO:teuthology.orchestra.run.smithi190.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-17T22:31:32.654 INFO:teuthology.orchestra.run.smithi190.stdout: inet 127.0.0.1/8 scope host lo 2024-04-17T22:31:32.655 INFO:teuthology.orchestra.run.smithi190.stdout: valid_lft forever preferred_lft forever 2024-04-17T22:31:32.655 INFO:teuthology.orchestra.run.smithi190.stdout: inet6 ::1/128 scope host 2024-04-17T22:31:32.655 INFO:teuthology.orchestra.run.smithi190.stdout: valid_lft forever preferred_lft forever 2024-04-17T22:31:32.655 INFO:teuthology.orchestra.run.smithi190.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-17T22:31:32.655 INFO:teuthology.orchestra.run.smithi190.stdout: link/ether 0c:c4:7a:d9:36:d0 brd ff:ff:ff:ff:ff:ff 2024-04-17T22:31:32.655 INFO:teuthology.orchestra.run.smithi190.stdout: altname enp7s0f0 2024-04-17T22:31:32.656 INFO:teuthology.orchestra.run.smithi190.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-17T22:31:32.656 INFO:teuthology.orchestra.run.smithi190.stdout: link/ether 0c:c4:7a:d9:36:d1 brd ff:ff:ff:ff:ff:ff 2024-04-17T22:31:32.656 INFO:teuthology.orchestra.run.smithi190.stdout: altname enp7s0f1 2024-04-17T22:31:32.656 INFO:teuthology.orchestra.run.smithi190.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-17T22:31:32.656 INFO:teuthology.orchestra.run.smithi190.stdout: link/ether 0c:c4:7a:88:77:62 brd ff:ff:ff:ff:ff:ff 2024-04-17T22:31:32.656 INFO:teuthology.orchestra.run.smithi190.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-17T22:31:32.656 INFO:teuthology.orchestra.run.smithi190.stdout: link/ether 0c:c4:7a:88:77:63 brd ff:ff:ff:ff:ff:ff 2024-04-17T22:31:32.657 INFO:teuthology.orchestra.run.smithi190.stdout: inet 172.21.15.190/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-17T22:31:32.657 INFO:teuthology.orchestra.run.smithi190.stdout: valid_lft 42532sec preferred_lft 42532sec 2024-04-17T22:31:32.657 INFO:teuthology.orchestra.run.smithi190.stdout: inet6 fe80::ec4:7aff:fe88:7763/64 scope link 2024-04-17T22:31:32.657 INFO:teuthology.orchestra.run.smithi190.stdout: valid_lft forever preferred_lft forever 2024-04-17T22:31:32.657 INFO:teuthology.orchestra.run.smithi190.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-17T22:31:32.657 INFO:teuthology.orchestra.run.smithi190.stdout: link/ether aa:db:76:d8:55:57 brd ff:ff:ff:ff:ff:ff 2024-04-17T22:31:32.658 INFO:teuthology.orchestra.run.smithi190.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-17T22:31:32.658 INFO:teuthology.orchestra.run.smithi190.stdout: valid_lft forever preferred_lft forever 2024-04-17T22:31:32.658 INFO:teuthology.orchestra.run.smithi190.stdout: inet6 fe80::18dc:79ff:feb0:22a0/64 scope link 2024-04-17T22:31:32.658 INFO:teuthology.orchestra.run.smithi190.stdout: valid_lft forever preferred_lft forever 2024-04-17T22:31:32.658 INFO:teuthology.orchestra.run.smithi190.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-17T22:31:32.658 INFO:teuthology.orchestra.run.smithi190.stdout: link/ether aa:db:76:d8:55:57 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-17T22:31:32.659 INFO:teuthology.orchestra.run.smithi190.stdout: inet6 fe80::a8db:76ff:fed8:5557/64 scope link 2024-04-17T22:31:32.659 INFO:teuthology.orchestra.run.smithi190.stdout: valid_lft forever preferred_lft forever 2024-04-17T22:31:32.659 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T22:31:32.659 DEBUG:teuthology.orchestra.run.smithi190:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-17T22:31:32.725 INFO:teuthology.orchestra.run.smithi190.stdout:1 2024-04-17T22:31:32.726 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T22:31:32.726 DEBUG:teuthology.orchestra.run.smithi190:> route 2024-04-17T22:31:32.789 INFO:teuthology.orchestra.run.smithi190.stdout:Kernel IP routing table 2024-04-17T22:31:32.789 INFO:teuthology.orchestra.run.smithi190.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-17T22:31:32.789 INFO:teuthology.orchestra.run.smithi190.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-17T22:31:32.789 INFO:teuthology.orchestra.run.smithi190.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-17T22:31:32.789 INFO:teuthology.orchestra.run.smithi190.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-17T22:31:32.789 DEBUG:teuthology.orchestra.run.smithi190:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-17T22:31:32.790 DEBUG:teuthology.orchestra.run.smithi190:> set -e 2024-04-17T22:31:32.790 DEBUG:teuthology.orchestra.run.smithi190:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-17T22:31:32.790 DEBUG:teuthology.orchestra.run.smithi190:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-17T22:31:32.790 DEBUG:teuthology.orchestra.run.smithi190:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-17T22:31:32.790 DEBUG:teuthology.orchestra.run.smithi190:> ') 2024-04-17T22:31:32.884 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T22:31:32 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-17T22:31:32.932 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T22:31:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T22:31:32.938 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T22:31:32.938 DEBUG:teuthology.orchestra.run.smithi190:> ip netns list 2024-04-17T22:31:32.956 INFO:teuthology.orchestra.run.smithi190.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-17T22:31:32.956 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-17T22:31:32.956 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T22:31:32.956 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip link set brx.0 up 2024-04-17T22:31:33.023 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-17T22:31:33.024 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-17T22:31:33.024 DEBUG:teuthology.orchestra.run.smithi190:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-17T22:31:33.081 INFO:teuthology.orchestra.run.smithi190.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-17T22:31:33.081 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-17T22:31:33.081 DEBUG:teuthology.orchestra.run.smithi190:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-17T22:31:33.139 DEBUG:teuthology.orchestra.run.smithi190:> sudo modprobe fuse 2024-04-17T22:31:33.207 DEBUG:teuthology.orchestra.run.smithi190:> cat /proc/self/mounts | awk '{print $2}' 2024-04-17T22:31:33.265 INFO:teuthology.orchestra.run.smithi190.stdout:/sys 2024-04-17T22:31:33.266 INFO:teuthology.orchestra.run.smithi190.stdout:/proc 2024-04-17T22:31:33.266 INFO:teuthology.orchestra.run.smithi190.stdout:/dev 2024-04-17T22:31:33.266 INFO:teuthology.orchestra.run.smithi190.stdout:/sys/kernel/security 2024-04-17T22:31:33.266 INFO:teuthology.orchestra.run.smithi190.stdout:/dev/shm 2024-04-17T22:31:33.266 INFO:teuthology.orchestra.run.smithi190.stdout:/dev/pts 2024-04-17T22:31:33.266 INFO:teuthology.orchestra.run.smithi190.stdout:/run 2024-04-17T22:31:33.266 INFO:teuthology.orchestra.run.smithi190.stdout:/sys/fs/cgroup 2024-04-17T22:31:33.266 INFO:teuthology.orchestra.run.smithi190.stdout:/sys/fs/cgroup/systemd 2024-04-17T22:31:33.266 INFO:teuthology.orchestra.run.smithi190.stdout:/sys/fs/pstore 2024-04-17T22:31:33.266 INFO:teuthology.orchestra.run.smithi190.stdout:/sys/fs/bpf 2024-04-17T22:31:33.266 INFO:teuthology.orchestra.run.smithi190.stdout:/sys/fs/cgroup/pids 2024-04-17T22:31:33.267 INFO:teuthology.orchestra.run.smithi190.stdout:/sys/fs/cgroup/cpu,cpuacct 2024-04-17T22:31:33.267 INFO:teuthology.orchestra.run.smithi190.stdout:/sys/fs/cgroup/net_cls,net_prio 2024-04-17T22:31:33.267 INFO:teuthology.orchestra.run.smithi190.stdout:/sys/fs/cgroup/rdma 2024-04-17T22:31:33.267 INFO:teuthology.orchestra.run.smithi190.stdout:/sys/fs/cgroup/hugetlb 2024-04-17T22:31:33.267 INFO:teuthology.orchestra.run.smithi190.stdout:/sys/fs/cgroup/blkio 2024-04-17T22:31:33.267 INFO:teuthology.orchestra.run.smithi190.stdout:/sys/fs/cgroup/freezer 2024-04-17T22:31:33.267 INFO:teuthology.orchestra.run.smithi190.stdout:/sys/fs/cgroup/cpuset 2024-04-17T22:31:33.267 INFO:teuthology.orchestra.run.smithi190.stdout:/sys/fs/cgroup/devices 2024-04-17T22:31:33.267 INFO:teuthology.orchestra.run.smithi190.stdout:/sys/fs/cgroup/memory 2024-04-17T22:31:33.267 INFO:teuthology.orchestra.run.smithi190.stdout:/sys/fs/cgroup/perf_event 2024-04-17T22:31:33.267 INFO:teuthology.orchestra.run.smithi190.stdout:/sys/kernel/tracing 2024-04-17T22:31:33.267 INFO:teuthology.orchestra.run.smithi190.stdout:/sys/kernel/config 2024-04-17T22:31:33.267 INFO:teuthology.orchestra.run.smithi190.stdout:/ 2024-04-17T22:31:33.268 INFO:teuthology.orchestra.run.smithi190.stdout:/sys/fs/selinux 2024-04-17T22:31:33.268 INFO:teuthology.orchestra.run.smithi190.stdout:/sys/kernel/debug 2024-04-17T22:31:33.268 INFO:teuthology.orchestra.run.smithi190.stdout:/dev/hugepages 2024-04-17T22:31:33.268 INFO:teuthology.orchestra.run.smithi190.stdout:/dev/mqueue 2024-04-17T22:31:33.268 INFO:teuthology.orchestra.run.smithi190.stdout:/proc/sys/fs/binfmt_misc 2024-04-17T22:31:33.268 INFO:teuthology.orchestra.run.smithi190.stdout:/sys/fs/fuse/connections 2024-04-17T22:31:33.269 INFO:teuthology.orchestra.run.smithi190.stdout:/var/lib/nfs/rpc_pipefs 2024-04-17T22:31:33.269 INFO:teuthology.orchestra.run.smithi190.stdout:/run/user/1000 2024-04-17T22:31:33.269 INFO:teuthology.orchestra.run.smithi190.stdout:/var/lib/ceph 2024-04-17T22:31:33.269 INFO:teuthology.orchestra.run.smithi190.stdout:/run/netns 2024-04-17T22:31:33.269 INFO:teuthology.orchestra.run.smithi190.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-17T22:31:33.269 INFO:teuthology.orchestra.run.smithi190.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-17T22:31:33.269 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T22:31:33.269 DEBUG:teuthology.orchestra.run.smithi190:> ls /sys/fs/fuse/connections 2024-04-17T22:31:33.324 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-17T22:31:33.324 DEBUG:teuthology.orchestra.run.smithi190:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-17T22:31:33.366 DEBUG:teuthology.orchestra.run.smithi190:> sudo modprobe fuse 2024-04-17T22:31:33.392 DEBUG:teuthology.orchestra.run.smithi190:> cat /proc/self/mounts | awk '{print $2}' 2024-04-17T22:31:33.447 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi190.stderr:2024-04-17T22:31:33.461+0000 7f0430a7d3c0 -1 init, newargv = 0x55bff8c824a0 newargc=15 2024-04-17T22:31:33.447 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi190.stderr:ceph-fuse[42802]: starting ceph client 2024-04-17T22:31:33.452 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi190.stderr:ceph-fuse[42802]: starting fuse 2024-04-17T22:31:33.452 INFO:teuthology.orchestra.run.smithi190.stdout:/sys 2024-04-17T22:31:33.452 INFO:teuthology.orchestra.run.smithi190.stdout:/proc 2024-04-17T22:31:33.452 INFO:teuthology.orchestra.run.smithi190.stdout:/dev 2024-04-17T22:31:33.452 INFO:teuthology.orchestra.run.smithi190.stdout:/sys/kernel/security 2024-04-17T22:31:33.452 INFO:teuthology.orchestra.run.smithi190.stdout:/dev/shm 2024-04-17T22:31:33.452 INFO:teuthology.orchestra.run.smithi190.stdout:/dev/pts 2024-04-17T22:31:33.452 INFO:teuthology.orchestra.run.smithi190.stdout:/run 2024-04-17T22:31:33.452 INFO:teuthology.orchestra.run.smithi190.stdout:/sys/fs/cgroup 2024-04-17T22:31:33.452 INFO:teuthology.orchestra.run.smithi190.stdout:/sys/fs/cgroup/systemd 2024-04-17T22:31:33.453 INFO:teuthology.orchestra.run.smithi190.stdout:/sys/fs/pstore 2024-04-17T22:31:33.453 INFO:teuthology.orchestra.run.smithi190.stdout:/sys/fs/bpf 2024-04-17T22:31:33.453 INFO:teuthology.orchestra.run.smithi190.stdout:/sys/fs/cgroup/pids 2024-04-17T22:31:33.453 INFO:teuthology.orchestra.run.smithi190.stdout:/sys/fs/cgroup/cpu,cpuacct 2024-04-17T22:31:33.453 INFO:teuthology.orchestra.run.smithi190.stdout:/sys/fs/cgroup/net_cls,net_prio 2024-04-17T22:31:33.453 INFO:teuthology.orchestra.run.smithi190.stdout:/sys/fs/cgroup/rdma 2024-04-17T22:31:33.453 INFO:teuthology.orchestra.run.smithi190.stdout:/sys/fs/cgroup/hugetlb 2024-04-17T22:31:33.453 INFO:teuthology.orchestra.run.smithi190.stdout:/sys/fs/cgroup/blkio 2024-04-17T22:31:33.453 INFO:teuthology.orchestra.run.smithi190.stdout:/sys/fs/cgroup/freezer 2024-04-17T22:31:33.453 INFO:teuthology.orchestra.run.smithi190.stdout:/sys/fs/cgroup/cpuset 2024-04-17T22:31:33.453 INFO:teuthology.orchestra.run.smithi190.stdout:/sys/fs/cgroup/devices 2024-04-17T22:31:33.453 INFO:teuthology.orchestra.run.smithi190.stdout:/sys/fs/cgroup/memory 2024-04-17T22:31:33.453 INFO:teuthology.orchestra.run.smithi190.stdout:/sys/fs/cgroup/perf_event 2024-04-17T22:31:33.453 INFO:teuthology.orchestra.run.smithi190.stdout:/sys/kernel/tracing 2024-04-17T22:31:33.453 INFO:teuthology.orchestra.run.smithi190.stdout:/sys/kernel/config 2024-04-17T22:31:33.454 INFO:teuthology.orchestra.run.smithi190.stdout:/ 2024-04-17T22:31:33.454 INFO:teuthology.orchestra.run.smithi190.stdout:/sys/fs/selinux 2024-04-17T22:31:33.454 INFO:teuthology.orchestra.run.smithi190.stdout:/sys/kernel/debug 2024-04-17T22:31:33.454 INFO:teuthology.orchestra.run.smithi190.stdout:/dev/hugepages 2024-04-17T22:31:33.454 INFO:teuthology.orchestra.run.smithi190.stdout:/dev/mqueue 2024-04-17T22:31:33.454 INFO:teuthology.orchestra.run.smithi190.stdout:/proc/sys/fs/binfmt_misc 2024-04-17T22:31:33.454 INFO:teuthology.orchestra.run.smithi190.stdout:/sys/fs/fuse/connections 2024-04-17T22:31:33.454 INFO:teuthology.orchestra.run.smithi190.stdout:/var/lib/nfs/rpc_pipefs 2024-04-17T22:31:33.454 INFO:teuthology.orchestra.run.smithi190.stdout:/run/user/1000 2024-04-17T22:31:33.454 INFO:teuthology.orchestra.run.smithi190.stdout:/var/lib/ceph 2024-04-17T22:31:33.454 INFO:teuthology.orchestra.run.smithi190.stdout:/run/netns 2024-04-17T22:31:33.454 INFO:teuthology.orchestra.run.smithi190.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-17T22:31:33.455 INFO:teuthology.orchestra.run.smithi190.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-17T22:31:33.455 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T22:31:33.456 DEBUG:teuthology.orchestra.run.smithi190:> ls /sys/fs/fuse/connections 2024-04-17T22:31:33.515 INFO:teuthology.orchestra.run.smithi190.stdout:46 2024-04-17T22:31:33.515 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [46] 2024-04-17T22:31:33.515 DEBUG:teuthology.orchestra.run.smithi190:> sudo stdin-killer -- python3 -c ' 2024-04-17T22:31:33.515 DEBUG:teuthology.orchestra.run.smithi190:> import glob 2024-04-17T22:31:33.515 DEBUG:teuthology.orchestra.run.smithi190:> import re 2024-04-17T22:31:33.515 DEBUG:teuthology.orchestra.run.smithi190:> import os 2024-04-17T22:31:33.515 DEBUG:teuthology.orchestra.run.smithi190:> import subprocess 2024-04-17T22:31:33.515 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T22:31:33.516 DEBUG:teuthology.orchestra.run.smithi190:> def _find_admin_socket(client_name): 2024-04-17T22:31:33.516 DEBUG:teuthology.orchestra.run.smithi190:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-17T22:31:33.516 DEBUG:teuthology.orchestra.run.smithi190:> files = glob.glob(asok_path) 2024-04-17T22:31:33.516 DEBUG:teuthology.orchestra.run.smithi190:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-17T22:31:33.516 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T22:31:33.516 DEBUG:teuthology.orchestra.run.smithi190:> # Given a non-glob path, it better be there 2024-04-17T22:31:33.516 DEBUG:teuthology.orchestra.run.smithi190:> if "*" not in asok_path: 2024-04-17T22:31:33.516 DEBUG:teuthology.orchestra.run.smithi190:> assert(len(files) == 1) 2024-04-17T22:31:33.516 DEBUG:teuthology.orchestra.run.smithi190:> return files[0] 2024-04-17T22:31:33.516 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T22:31:33.516 DEBUG:teuthology.orchestra.run.smithi190:> for f in files: 2024-04-17T22:31:33.516 DEBUG:teuthology.orchestra.run.smithi190:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-17T22:31:33.516 DEBUG:teuthology.orchestra.run.smithi190:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-17T22:31:33.516 DEBUG:teuthology.orchestra.run.smithi190:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-17T22:31:33.516 DEBUG:teuthology.orchestra.run.smithi190:> contents = proc_f.read() 2024-04-17T22:31:33.517 DEBUG:teuthology.orchestra.run.smithi190:> if mountpoint in contents: 2024-04-17T22:31:33.517 DEBUG:teuthology.orchestra.run.smithi190:> return f 2024-04-17T22:31:33.517 DEBUG:teuthology.orchestra.run.smithi190:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-17T22:31:33.517 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T22:31:33.517 DEBUG:teuthology.orchestra.run.smithi190:> print(_find_admin_socket("client.0")) 2024-04-17T22:31:33.517 DEBUG:teuthology.orchestra.run.smithi190:> ' 2024-04-17T22:31:33.638 INFO:teuthology.orchestra.run.smithi190.stdout:/var/run/ceph/ceph-client.0.42802.asok 2024-04-17T22:31:33.641 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T22:31:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T22:31:33.647 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.42802.asok 2024-04-17T22:31:33.647 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T22:31:33.647 DEBUG:teuthology.orchestra.run.smithi190:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.42802.asok status 2024-04-17T22:31:33.797 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-04-17T22:31:33.797 INFO:teuthology.orchestra.run.smithi190.stdout: "metadata": { 2024-04-17T22:31:33.797 INFO:teuthology.orchestra.run.smithi190.stdout: "ceph_sha1": "618f440892089921c3e944a991122ddc44e60516", 2024-04-17T22:31:33.797 INFO:teuthology.orchestra.run.smithi190.stdout: "ceph_version": "ceph version 16.2.15 (618f440892089921c3e944a991122ddc44e60516) pacific (stable)", 2024-04-17T22:31:33.798 INFO:teuthology.orchestra.run.smithi190.stdout: "entity_id": "0", 2024-04-17T22:31:33.798 INFO:teuthology.orchestra.run.smithi190.stdout: "hostname": "smithi190", 2024-04-17T22:31:33.798 INFO:teuthology.orchestra.run.smithi190.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-17T22:31:33.798 INFO:teuthology.orchestra.run.smithi190.stdout: "pid": "42802", 2024-04-17T22:31:33.798 INFO:teuthology.orchestra.run.smithi190.stdout: "root": "/" 2024-04-17T22:31:33.798 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-04-17T22:31:33.798 INFO:teuthology.orchestra.run.smithi190.stdout: "dentry_count": 0, 2024-04-17T22:31:33.798 INFO:teuthology.orchestra.run.smithi190.stdout: "dentry_pinned_count": 0, 2024-04-17T22:31:33.798 INFO:teuthology.orchestra.run.smithi190.stdout: "id": 5155, 2024-04-17T22:31:33.798 INFO:teuthology.orchestra.run.smithi190.stdout: "inst": { 2024-04-17T22:31:33.798 INFO:teuthology.orchestra.run.smithi190.stdout: "name": { 2024-04-17T22:31:33.798 INFO:teuthology.orchestra.run.smithi190.stdout: "type": "client", 2024-04-17T22:31:33.799 INFO:teuthology.orchestra.run.smithi190.stdout: "num": 5155 2024-04-17T22:31:33.799 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-04-17T22:31:33.799 INFO:teuthology.orchestra.run.smithi190.stdout: "addr": { 2024-04-17T22:31:33.799 INFO:teuthology.orchestra.run.smithi190.stdout: "type": "v1", 2024-04-17T22:31:33.799 INFO:teuthology.orchestra.run.smithi190.stdout: "addr": "172.21.15.190:0", 2024-04-17T22:31:33.799 INFO:teuthology.orchestra.run.smithi190.stdout: "nonce": 598282216 2024-04-17T22:31:33.799 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-04-17T22:31:33.799 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-04-17T22:31:33.799 INFO:teuthology.orchestra.run.smithi190.stdout: "addr": { 2024-04-17T22:31:33.799 INFO:teuthology.orchestra.run.smithi190.stdout: "type": "v1", 2024-04-17T22:31:33.799 INFO:teuthology.orchestra.run.smithi190.stdout: "addr": "172.21.15.190:0", 2024-04-17T22:31:33.799 INFO:teuthology.orchestra.run.smithi190.stdout: "nonce": 598282216 2024-04-17T22:31:33.799 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-04-17T22:31:33.800 INFO:teuthology.orchestra.run.smithi190.stdout: "inst_str": "client.5155 172.21.15.190:0/598282216", 2024-04-17T22:31:33.800 INFO:teuthology.orchestra.run.smithi190.stdout: "addr_str": "172.21.15.190:0/598282216", 2024-04-17T22:31:33.800 INFO:teuthology.orchestra.run.smithi190.stdout: "inode_count": 1, 2024-04-17T22:31:33.800 INFO:teuthology.orchestra.run.smithi190.stdout: "mds_epoch": 11, 2024-04-17T22:31:33.800 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_epoch": 25, 2024-04-17T22:31:33.800 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_epoch_barrier": 0, 2024-04-17T22:31:33.800 INFO:teuthology.orchestra.run.smithi190.stdout: "blocklisted": false, 2024-04-17T22:31:33.800 INFO:teuthology.orchestra.run.smithi190.stdout: "fs_name": "cephfs" 2024-04-17T22:31:33.800 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-04-17T22:31:33.808 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T22:31:33.808 DEBUG:teuthology.orchestra.run.smithi190:> stat --file-system '--printf=%T 2024-04-17T22:31:33.808 DEBUG:teuthology.orchestra.run.smithi190:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-17T22:31:33.827 INFO:teuthology.orchestra.run.smithi190.stdout:fuseblk 2024-04-17T22:31:33.827 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-17T22:31:33.828 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T22:31:33.828 DEBUG:teuthology.orchestra.run.smithi190:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-17T22:31:33.895 INFO:tasks.cephfs.test_newops:Test for new getvxattr op... 2024-04-17T22:31:33.895 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T22:31:33.895 DEBUG:teuthology.orchestra.run.smithi190:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir newop_getvxattr_dir) 2024-04-17T22:31:33.959 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T22:31:33.960 DEBUG:teuthology.orchestra.run.smithi190:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.pin.random ./newop_getvxattr_dir) 2024-04-17T22:31:34.019 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-17T22:31:34.020 INFO:teuthology.orchestra.run.smithi190.stderr:./newop_getvxattr_dir: ceph.dir.pin.random: No such attribute 2024-04-17T22:31:34.020 INFO:tasks.cephfs.test_newops:Test for new getvxattr op succeeds 2024-04-17T22:31:34.020 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-17T22:31:34.020 DEBUG:teuthology.orchestra.run.smithi102:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-17T22:31:34.057 DEBUG:teuthology.orchestra.run.smithi190:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-17T22:31:34.100 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-17T22:31:34.100 INFO:tasks.cephfs.mount:Terminating background process 2024-04-17T22:31:34.100 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-04-17T22:31:34.100 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T22:31:34.158 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T22:31:34.301 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:34.312 7fd8dfb3c700 1 Processor -- start 2024-04-17T22:31:34.302 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:34.312 7fd8dfb3c700 1 -- start start 2024-04-17T22:31:34.302 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:34.312 7fd8dfb3c700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fd8d80f57d0 0x7fd8d810afa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:34.302 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:34.312 7fd8dfb3c700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd8d810ba20 0x7fd8d81a0280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:34.302 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:34.312 7fd8dfb3c700 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8d8103ad0 con 0x7fd8d810b5d0 2024-04-17T22:31:34.302 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:34.312 7fd8dfb3c700 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8d810be70 con 0x7fd8d80f57d0 2024-04-17T22:31:34.302 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:34.312 7fd8dfb3c700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8d80fa300 con 0x7fd8d810ba20 2024-04-17T22:31:34.302 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:34.313 7fd8dc937700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fd8d80f57d0 0x7fd8d810afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:34.303 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:34.313 7fd8dd138700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd8d810ba20 0x7fd8d81a0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:34.303 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:34.313 7fd8d7fff700 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7fd8d810b5d0 0x7fd8d819fa80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:45616/0 (socket says 172.21.15.102:45616) 2024-04-17T22:31:34.303 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:34.313 7fd8d7fff700 1 -- 172.21.15.102:0/1556403181 learned_addr learned my addr 172.21.15.102:0/1556403181 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-17T22:31:34.303 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:34.313 7fd8dc937700 1 -- 172.21.15.102:0/1556403181 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd8d810ba20 msgr2=0x7fd8d81a0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:31:34.303 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:34.313 7fd8dc937700 1 --2- 172.21.15.102:0/1556403181 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd8d810ba20 0x7fd8d81a0280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:34.303 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:34.313 7fd8dc937700 1 -- 172.21.15.102:0/1556403181 >> v1:172.21.15.102:6789/0 conn(0x7fd8d810b5d0 legacy=0x7fd8d819fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:34.303 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:34.313 7fd8dc937700 1 -- 172.21.15.102:0/1556403181 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8d81a08b0 con 0x7fd8d80f57d0 2024-04-17T22:31:34.303 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:34.313 7fd8d77fe700 1 -- 172.21.15.102:0/1556403181 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1504960453 0 0) 0x7fd8c4001110 con 0x7fd8d810b5d0 2024-04-17T22:31:34.303 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:34.313 7fd8dd138700 1 --2- 172.21.15.102:0/1556403181 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd8d810ba20 0x7fd8d81a0280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:34.303 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:34.313 7fd8dc937700 1 --2- 172.21.15.102:0/1556403181 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd8d80f57d0 0x7fd8d810afa0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 rx=0x7fd8d8103cf0 tx=0x7fd8cc00dc80).ready entity=mon.0 client_cookie=aa9ca69efa8e4328 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:34.304 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:34.313 7fd8d77fe700 1 -- 172.21.15.102:0/1556403181 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd8d81a08b0 con 0x7fd8d80f57d0 2024-04-17T22:31:34.304 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:34.313 7fd8d77fe700 1 -- 172.21.15.102:0/1556403181 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8cc00bba0 con 0x7fd8d80f57d0 2024-04-17T22:31:34.304 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:34.313 7fd8d77fe700 1 -- 172.21.15.102:0/1556403181 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd8cc015070 con 0x7fd8d80f57d0 2024-04-17T22:31:34.304 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:34.313 7fd8dfb3c700 1 -- 172.21.15.102:0/1556403181 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd8d80f57d0 msgr2=0x7fd8d810afa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:34.304 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:34.313 7fd8dfb3c700 1 --2- 172.21.15.102:0/1556403181 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd8d80f57d0 0x7fd8d810afa0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 rx=0x7fd8d8103cf0 tx=0x7fd8cc00dc80).stop 2024-04-17T22:31:34.304 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:34.314 7fd8dfb3c700 1 -- 172.21.15.102:0/1556403181 shutdown_connections 2024-04-17T22:31:34.304 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:34.314 7fd8dfb3c700 1 --2- 172.21.15.102:0/1556403181 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd8d810ba20 0x7fd8d81a0280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:34.304 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:34.314 7fd8dfb3c700 1 --2- 172.21.15.102:0/1556403181 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd8d80f57d0 0x7fd8d810afa0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:34.305 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:34.314 7fd8dfb3c700 1 -- 172.21.15.102:0/1556403181 >> 172.21.15.102:0/1556403181 conn(0x7fd8d8113830 msgr2=0x7fd8d80f4ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:34.305 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:34.314 7fd8dfb3c700 1 -- 172.21.15.102:0/1556403181 shutdown_connections 2024-04-17T22:31:34.305 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:34.314 7fd8dfb3c700 1 -- 172.21.15.102:0/1556403181 wait complete. 2024-04-17T22:31:34.323 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:34.337 7fd8dfb3c700 1 Processor -- start 2024-04-17T22:31:34.323 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:34.337 7fd8dfb3c700 1 -- start start 2024-04-17T22:31:34.326 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:34.338 7fd8dfb3c700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd8d80d07e0 0x7fd8d80d0c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:34.326 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:34.338 7fd8dfb3c700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd8d80d1260 0x7fd8d81a0f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:34.327 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:34.338 7fd8dfb3c700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd8d81a1540 0x7fd8d81bb8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:34.327 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:34.338 7fd8dfb3c700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8d80c3940 con 0x7fd8d80d07e0 2024-04-17T22:31:34.327 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:34.338 7fd8dfb3c700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8d80ddfb0 con 0x7fd8d81a1540 2024-04-17T22:31:34.327 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:34.338 7fd8dfb3c700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd8d80ddb90 con 0x7fd8d80d1260 2024-04-17T22:31:34.327 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:34.338 7fd8dc937700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd8d80d07e0 0x7fd8d80d0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:34.327 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:34.338 7fd8d7fff700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd8d80d1260 0x7fd8d81a0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:34.327 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:34.338 7fd8dc937700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd8d80d07e0 0x7fd8d80d0c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:57308/0 (socket says 172.21.15.102:57308) 2024-04-17T22:31:34.327 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:34.338 7fd8dc937700 1 -- 172.21.15.102:0/1001597353 learned_addr learned my addr 172.21.15.102:0/1001597353 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:31:34.327 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:34.338 7fd8dd138700 1 --2- 172.21.15.102:0/1001597353 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd8d81a1540 0x7fd8d81bb8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:34.328 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:34.338 7fd8dc937700 1 -- 172.21.15.102:0/1001597353 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd8d80d1260 msgr2=0x7fd8d81a0f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:34.328 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:34.338 7fd8dc937700 1 --2- 172.21.15.102:0/1001597353 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd8d80d1260 0x7fd8d81a0f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:34.328 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:34.338 7fd8dc937700 1 -- 172.21.15.102:0/1001597353 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd8d81a1540 msgr2=0x7fd8d81bb8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:34.328 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:34.338 7fd8dc937700 1 --2- 172.21.15.102:0/1001597353 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd8d81a1540 0x7fd8d81bb8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:34.328 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:34.338 7fd8dc937700 1 -- 172.21.15.102:0/1001597353 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8d80ddd70 con 0x7fd8d80d07e0 2024-04-17T22:31:34.328 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:34.338 7fd8d7fff700 1 --2- 172.21.15.102:0/1001597353 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd8d80d1260 0x7fd8d81a0f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:34.328 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:34.338 7fd8dc937700 1 --2- 172.21.15.102:0/1001597353 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd8d80d07e0 0x7fd8d80d0c30 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 rx=0x7fd8d8103cf0 tx=0x7fd8cc0068c0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:34.328 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:34.339 7fd8d5ffb700 1 -- 172.21.15.102:0/1001597353 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd8d80ddd70 con 0x7fd8d80d07e0 2024-04-17T22:31:34.328 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:34.339 7fd8d5ffb700 1 -- 172.21.15.102:0/1001597353 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8cc01a440 con 0x7fd8d80d07e0 2024-04-17T22:31:34.328 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:34.339 7fd8dfb3c700 1 -- 172.21.15.102:0/1001597353 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8d81a1990 con 0x7fd8d80d07e0 2024-04-17T22:31:34.329 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:34.339 7fd8d5ffb700 1 -- 172.21.15.102:0/1001597353 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd8cc01a810 con 0x7fd8d80d07e0 2024-04-17T22:31:34.329 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:34.339 7fd8dfb3c700 1 -- 172.21.15.102:0/1001597353 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8d81bbf20 con 0x7fd8d80d07e0 2024-04-17T22:31:34.329 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:34.339 7fd8d5ffb700 1 -- 172.21.15.102:0/1001597353 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7fd8cc01aa10 con 0x7fd8d80d07e0 2024-04-17T22:31:34.329 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:34.339 7fd8d5ffb700 1 -- 172.21.15.102:0/1001597353 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 3194+0+0 (secure 0 0 0) 0x7fd8cc008d30 con 0x7fd8d80d07e0 2024-04-17T22:31:34.329 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:34.340 7fd8dfb3c700 1 -- 172.21.15.102:0/1001597353 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8d80f0790 con 0x7fd8d80d07e0 2024-04-17T22:31:34.329 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:34.343 7fd8d5ffb700 1 -- 172.21.15.102:0/1001597353 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+110455 (secure 0 0 0) 0x7fd8cc00b7a0 con 0x7fd8d80d07e0 2024-04-17T22:31:34.433 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:34.447 7fd8dfb3c700 1 -- 172.21.15.102:0/1001597353 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd8d81ae9f0 con 0x7fd8d80d07e0 2024-04-17T22:31:34.434 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:34.448 7fd8d5ffb700 1 -- 172.21.15.102:0/1001597353 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+8764 (secure 0 0 0) 0x7fd8cc010b10 con 0x7fd8d80d07e0 2024-04-17T22:31:34.435 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-17T22:31:34.435 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":25,"fsid":"cb63e789-091d-49b1-b120-55f9db526871","created":"2024-04-17 22:28:48.209635","modified":"2024-04-17 22:31:29.205644","last_up_change":"2024-04-17 22:28:54.236215","last_in_change":"2024-04-17 22:28:54.236215","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":4,"require_min_compat_client":"nautilus","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-04-17 22:28:57.478545","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-17 22:31:26.214102","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-17 22:31:27.183113","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3351ef85-ade4-4b90-a8f4-213f4e00ba01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6812","nonce":42955}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6813","nonce":42955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6815","nonce":42955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6814","nonce":42955}]},"public_addr":"172.21.15.102:6812/42955","cluster_addr":"172.21.15.102:6813/42955","heartbeat_back_addr":"172.21.15.102:6815/42955","heartbeat_front_addr":"172.21.15.102:6814/42955","state":["exists","up"]},{"osd":1,"uuid":"04f67eac-f36b-48f5-982d-4cfc776596d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6808","nonce":42958}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6809","nonce":42958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6811","nonce":42958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6810","nonce":42958}]},"public_addr":"172.21.15.102:6808/42958","cluster_addr":"172.21.15.102:6809/42958","heartbeat_back_addr":"172.21.15.102:6811/42958","heartbeat_front_addr":"172.21.15.102:6810/42958","state":["exists","up"]},{"osd":2,"uuid":"d524204b-70b2-40a8-8f73-f9e7d44318af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6804","nonce":42954}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6805","nonce":42954}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6807","nonce":42954}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6806","nonce":42954}]},"public_addr":"172.21.15.102:6804/42954","cluster_addr":"172.21.15.102:6805/42954","heartbeat_back_addr":"172.21.15.102:6807/42954","heartbeat_front_addr":"172.21.15.102:6806/42954","state":["exists","up"]},{"osd":3,"uuid":"4fc9d20c-1ad0-484f-aafa-2c9a7574245b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6800","nonce":42953}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6801","nonce":42953}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6803","nonce":42953}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6802","nonce":42953}]},"public_addr":"172.21.15.102:6800/42953","cluster_addr":"172.21.15.102:6801/42953","heartbeat_back_addr":"172.21.15.102:6803/42953","heartbeat_front_addr":"172.21.15.102:6802/42953","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2024-04-17T22:31:34.436 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:34.449 7fd8dfb3c700 1 -- 172.21.15.102:0/1001597353 >> v1:172.21.15.102:6816/42360 conn(0x7fd8c00145a0 legacy=0x7fd8c0016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:34.436 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:34.449 7fd8dfb3c700 1 -- 172.21.15.102:0/1001597353 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd8d80d07e0 msgr2=0x7fd8d80d0c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:34.436 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:34.449 7fd8dfb3c700 1 --2- 172.21.15.102:0/1001597353 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd8d80d07e0 0x7fd8d80d0c30 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 rx=0x7fd8d8103cf0 tx=0x7fd8cc0068c0).stop 2024-04-17T22:31:34.436 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:34.449 7fd8dfb3c700 1 -- 172.21.15.102:0/1001597353 shutdown_connections 2024-04-17T22:31:34.436 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:34.449 7fd8dfb3c700 1 --2- 172.21.15.102:0/1001597353 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd8d81a1540 0x7fd8d81bb8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:34.437 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:34.449 7fd8dfb3c700 1 --2- 172.21.15.102:0/1001597353 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd8d80d07e0 0x7fd8d80d0c30 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:34.437 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:34.449 7fd8dfb3c700 1 --2- 172.21.15.102:0/1001597353 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd8d80d1260 0x7fd8d81a0f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:34.437 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:34.449 7fd8dfb3c700 1 -- 172.21.15.102:0/1001597353 >> 172.21.15.102:0/1001597353 conn(0x7fd8d80c4ae0 msgr2=0x7fd8d81a85e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:34.437 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:34.449 7fd8dfb3c700 1 -- 172.21.15.102:0/1001597353 shutdown_connections 2024-04-17T22:31:34.437 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:34.449 7fd8dfb3c700 1 -- 172.21.15.102:0/1001597353 wait complete. 2024-04-17T22:31:34.446 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2024-04-17 22:28:57.478545', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '12', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-17 22:31:26.214102', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-04-17 22:31:27.183113', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-17T22:31:34.446 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2024-04-17T22:31:34.576 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:34.588 7f4d4e631700 1 Processor -- start 2024-04-17T22:31:34.576 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:34.588 7f4d4e631700 1 -- start start 2024-04-17T22:31:34.576 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:34.588 7f4d4e631700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f4d4810e240 0x7f4d4810e690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:34.577 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:34.588 7f4d4e631700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4d480f1f30 0x7f4d481ab4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:34.577 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:34.588 7f4d4e631700 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d480f2380 con 0x7f4d4810ecc0 2024-04-17T22:31:34.577 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:34.588 7f4d4e631700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d480f9f10 con 0x7f4d480f1f30 2024-04-17T22:31:34.577 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:34.588 7f4d4e631700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4d480f9030 con 0x7f4d4810e240 2024-04-17T22:31:34.577 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:34.588 7f4d473b0700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f4d4810e240 0x7f4d4810e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:34.577 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:34.588 7f4d46baf700 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f4d4810ecc0 0x7f4d480f1730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:45618/0 (socket says 172.21.15.102:45618) 2024-04-17T22:31:34.577 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:34.588 7f4d46baf700 1 -- 172.21.15.102:0/1690362394 learned_addr learned my addr 172.21.15.102:0/1690362394 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-17T22:31:34.577 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:34.588 7f4d47bb1700 1 --2- 172.21.15.102:0/1690362394 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4d480f1f30 0x7f4d481ab4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:34.577 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:34.589 7f4d473b0700 1 -- 172.21.15.102:0/1690362394 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4d480f1f30 msgr2=0x7f4d481ab4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:31:34.577 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:34.589 7f4d473b0700 1 --2- 172.21.15.102:0/1690362394 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4d480f1f30 0x7f4d481ab4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:34.577 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:34.589 7f4d473b0700 1 -- 172.21.15.102:0/1690362394 >> v1:172.21.15.102:6789/0 conn(0x7f4d4810ecc0 legacy=0x7f4d480f1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:34.577 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:34.589 7f4d473b0700 1 -- 172.21.15.102:0/1690362394 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d480f25a0 con 0x7f4d4810e240 2024-04-17T22:31:34.577 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:34.589 7f4d463ae700 1 -- 172.21.15.102:0/1690362394 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 655580231 0 0) 0x7f4d30001110 con 0x7f4d4810ecc0 2024-04-17T22:31:34.577 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:34.589 7f4d47bb1700 1 --2- 172.21.15.102:0/1690362394 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4d480f1f30 0x7f4d481ab4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:34.578 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:34.589 7f4d473b0700 1 --2- 172.21.15.102:0/1690362394 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f4d4810e240 0x7f4d4810e690 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 rx=0x7f4d480f4470 tx=0x7f4d3800b9d0).ready entity=mon.2 client_cookie=331f5e0ed4339513 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:34.578 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:34.589 7f4d463ae700 1 -- 172.21.15.102:0/1690362394 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f4d480f25a0 con 0x7f4d4810e240 2024-04-17T22:31:34.578 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:34.589 7f4d463ae700 1 -- 172.21.15.102:0/1690362394 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d3800d570 con 0x7f4d4810e240 2024-04-17T22:31:34.578 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:34.589 7f4d463ae700 1 -- 172.21.15.102:0/1690362394 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f4d3800d940 con 0x7f4d4810e240 2024-04-17T22:31:34.578 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:34.589 7f4d4e631700 1 -- 172.21.15.102:0/1690362394 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f4d4810e240 msgr2=0x7f4d4810e690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:34.578 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:34.589 7f4d4e631700 1 --2- 172.21.15.102:0/1690362394 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f4d4810e240 0x7f4d4810e690 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 rx=0x7f4d480f4470 tx=0x7f4d3800b9d0).stop 2024-04-17T22:31:34.578 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:34.590 7f4d4e631700 1 -- 172.21.15.102:0/1690362394 shutdown_connections 2024-04-17T22:31:34.578 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:34.590 7f4d4e631700 1 --2- 172.21.15.102:0/1690362394 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4d480f1f30 0x7f4d481ab4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:34.578 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:34.590 7f4d4e631700 1 --2- 172.21.15.102:0/1690362394 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f4d4810e240 0x7f4d4810e690 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:34.578 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:34.590 7f4d4e631700 1 -- 172.21.15.102:0/1690362394 >> 172.21.15.102:0/1690362394 conn(0x7f4d481138a0 msgr2=0x7f4d480f4f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:34.578 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:34.590 7f4d4e631700 1 -- 172.21.15.102:0/1690362394 shutdown_connections 2024-04-17T22:31:34.578 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:34.590 7f4d4e631700 1 -- 172.21.15.102:0/1690362394 wait complete. 2024-04-17T22:31:34.600 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:34.612 7f4d4e631700 1 Processor -- start 2024-04-17T22:31:34.600 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:34.613 7f4d4e631700 1 -- start start 2024-04-17T22:31:34.600 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:34.613 7f4d4e631700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f4d481adf50 0x7f4d481ae3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:34.600 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:34.613 7f4d4e631700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4d481b4470 0x7f4d481b48c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:34.601 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:34.613 7f4d4e631700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4d481b4ef0 0x7f4d481bb950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:34.601 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:34.613 7f4d4e631700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d480fa1b0 con 0x7f4d481b4ef0 2024-04-17T22:31:34.601 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:34.613 7f4d4e631700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d480e0490 con 0x7f4d481b4470 2024-04-17T22:31:34.601 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:34.613 7f4d4e631700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4d481ae9d0 con 0x7f4d481adf50 2024-04-17T22:31:34.601 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:34.613 7f4d473b0700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f4d481adf50 0x7f4d481ae3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:34.601 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:34.613 7f4d46baf700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4d481b4470 0x7f4d481b48c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:34.601 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:34.613 7f4d473b0700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f4d481adf50 0x7f4d481ae3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:48046/0 (socket says 172.21.15.102:48046) 2024-04-17T22:31:34.601 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:34.613 7f4d473b0700 1 -- 172.21.15.102:0/3793021328 learned_addr learned my addr 172.21.15.102:0/3793021328 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:31:34.601 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:34.613 7f4d47bb1700 1 --2- 172.21.15.102:0/3793021328 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4d481b4ef0 0x7f4d481bb950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:34.601 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:34.613 7f4d473b0700 1 -- 172.21.15.102:0/3793021328 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4d481b4470 msgr2=0x7f4d481b48c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:34.601 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:34.613 7f4d473b0700 1 --2- 172.21.15.102:0/3793021328 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4d481b4470 0x7f4d481b48c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:34.601 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:34.613 7f4d473b0700 1 -- 172.21.15.102:0/3793021328 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4d481b4ef0 msgr2=0x7f4d481bb950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:34.601 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:34.613 7f4d473b0700 1 --2- 172.21.15.102:0/3793021328 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4d481b4ef0 0x7f4d481bb950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:34.601 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:34.613 7f4d473b0700 1 -- 172.21.15.102:0/3793021328 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d480e08b0 con 0x7f4d481adf50 2024-04-17T22:31:34.601 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:34.613 7f4d46baf700 1 --2- 172.21.15.102:0/3793021328 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4d481b4470 0x7f4d481b48c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:34.602 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:34.613 7f4d47bb1700 1 --2- 172.21.15.102:0/3793021328 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4d481b4ef0 0x7f4d481bb950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:34.602 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:34.614 7f4d473b0700 1 --2- 172.21.15.102:0/3793021328 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f4d481adf50 0x7f4d481ae3a0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 rx=0x7f4d480f4470 tx=0x7f4d380068c0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:34.602 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:34.614 7f4d44bab700 1 -- 172.21.15.102:0/3793021328 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f4d480e08b0 con 0x7f4d481adf50 2024-04-17T22:31:34.602 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:34.614 7f4d44bab700 1 -- 172.21.15.102:0/3793021328 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d380117e0 con 0x7f4d481adf50 2024-04-17T22:31:34.602 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:34.614 7f4d44bab700 1 -- 172.21.15.102:0/3793021328 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f4d38017040 con 0x7f4d481adf50 2024-04-17T22:31:34.602 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:34.614 7f4d4e631700 1 -- 172.21.15.102:0/3793021328 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d481b5340 con 0x7f4d481adf50 2024-04-17T22:31:34.602 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:34.614 7f4d4e631700 1 -- 172.21.15.102:0/3793021328 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d481bbf80 con 0x7f4d481adf50 2024-04-17T22:31:34.602 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:34.614 7f4d44bab700 1 -- 172.21.15.102:0/3793021328 <== mon.2 v2:172.21.15.102:3302/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7f4d38011bb0 con 0x7f4d481adf50 2024-04-17T22:31:34.602 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:34.614 7f4d44bab700 1 -- 172.21.15.102:0/3793021328 <== mon.2 v2:172.21.15.102:3302/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 3194+0+0 (secure 0 0 0) 0x7f4d38018070 con 0x7f4d481adf50 2024-04-17T22:31:34.603 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:34.615 7f4d4e631700 1 -- 172.21.15.102:0/3793021328 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d480f0800 con 0x7f4d481adf50 2024-04-17T22:31:34.604 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:34.618 7f4d44bab700 1 -- 172.21.15.102:0/3793021328 <== mon.2 v2:172.21.15.102:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+110455 (secure 0 0 0) 0x7f4d380115d0 con 0x7f4d481adf50 2024-04-17T22:31:34.710 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:34.724 7f4d4e631700 1 -- 172.21.15.102:0/3793021328 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f4d480a64e0 con 0x7f4d481adf50 2024-04-17T22:31:34.710 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:34.724 7f4d44bab700 1 -- 172.21.15.102:0/3793021328 <== mon.2 v2:172.21.15.102:3302/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v25) v1 ==== 92+0+10 (secure 0 0 0) 0x7f4d3800d360 con 0x7f4d481adf50 2024-04-17T22:31:34.711 INFO:teuthology.orchestra.run.smithi102.stdout:pg_num: 8 2024-04-17T22:31:34.712 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:34.725 7f4d4e631700 1 -- 172.21.15.102:0/3793021328 >> v1:172.21.15.102:6816/42360 conn(0x7f4d2c0145a0 legacy=0x7f4d2c016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:34.712 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:34.725 7f4d4e631700 1 -- 172.21.15.102:0/3793021328 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f4d481adf50 msgr2=0x7f4d481ae3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:34.712 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:34.725 7f4d4e631700 1 --2- 172.21.15.102:0/3793021328 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f4d481adf50 0x7f4d481ae3a0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 rx=0x7f4d480f4470 tx=0x7f4d380068c0).stop 2024-04-17T22:31:34.712 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:34.725 7f4d4e631700 1 -- 172.21.15.102:0/3793021328 shutdown_connections 2024-04-17T22:31:34.712 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:34.725 7f4d4e631700 1 --2- 172.21.15.102:0/3793021328 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4d481b4ef0 0x7f4d481bb950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:34.713 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:34.725 7f4d4e631700 1 --2- 172.21.15.102:0/3793021328 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f4d481adf50 0x7f4d481ae3a0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:34.713 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:34.725 7f4d4e631700 1 --2- 172.21.15.102:0/3793021328 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4d481b4470 0x7f4d481b48c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:34.713 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:34.725 7f4d4e631700 1 -- 172.21.15.102:0/3793021328 >> 172.21.15.102:0/3793021328 conn(0x7f4d480c6be0 msgr2=0x7f4d481a84e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:34.713 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:34.726 7f4d4e631700 1 -- 172.21.15.102:0/3793021328 shutdown_connections 2024-04-17T22:31:34.713 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:34.726 7f4d4e631700 1 -- 172.21.15.102:0/3793021328 wait complete. 2024-04-17T22:31:34.725 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-17T22:31:34.855 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:34.867 7fb9f8583700 1 Processor -- start 2024-04-17T22:31:34.855 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:34.867 7fb9f8583700 1 -- start start 2024-04-17T22:31:34.855 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:34.868 7fb9f8583700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fb9f010e240 0x7fb9f010e690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:34.855 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:34.868 7fb9f8583700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb9f010ecc0 0x7fb9f00f1730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:34.855 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:34.868 7fb9f8583700 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9f00f21b0 con 0x7fb9f00f1d60 2024-04-17T22:31:34.855 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:34.868 7fb9f8583700 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9f00f9f10 con 0x7fb9f010e240 2024-04-17T22:31:34.856 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:34.868 7fb9f8583700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9f00f9030 con 0x7fb9f010ecc0 2024-04-17T22:31:34.856 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:34.868 7fb9f4b7d700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb9f010ecc0 0x7fb9f00f1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:34.856 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:34.868 7fb9f537e700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fb9f010e240 0x7fb9f010e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:34.856 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:34.868 7fb9f5b7f700 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7fb9f00f1d60 0x7fb9f01aa4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:45630/0 (socket says 172.21.15.102:45630) 2024-04-17T22:31:34.856 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:34.868 7fb9f5b7f700 1 -- 172.21.15.102:0/114465827 learned_addr learned my addr 172.21.15.102:0/114465827 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-17T22:31:34.856 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:34.868 7fb9f537e700 1 -- 172.21.15.102:0/114465827 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb9f010ecc0 msgr2=0x7fb9f00f1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:31:34.856 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:34.868 7fb9f537e700 1 --2- 172.21.15.102:0/114465827 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb9f010ecc0 0x7fb9f00f1730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:34.856 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:34.868 7fb9f537e700 1 -- 172.21.15.102:0/114465827 >> v1:172.21.15.102:6789/0 conn(0x7fb9f00f1d60 legacy=0x7fb9f01aa4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:34.856 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:34.868 7fb9effff700 1 -- 172.21.15.102:0/114465827 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2719137473 0 0) 0x7fb9e8001110 con 0x7fb9f00f1d60 2024-04-17T22:31:34.856 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:34.868 7fb9f537e700 1 -- 172.21.15.102:0/114465827 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9f00f23d0 con 0x7fb9f010e240 2024-04-17T22:31:34.856 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:34.868 7fb9f4b7d700 1 --2- 172.21.15.102:0/114465827 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb9f010ecc0 0x7fb9f00f1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:34.856 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:34.868 7fb9f537e700 1 --2- 172.21.15.102:0/114465827 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb9f010e240 0x7fb9f010e690 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 rx=0x7fb9e0002f70 tx=0x7fb9e000b9f0).ready entity=mon.0 client_cookie=9e88e777e207bed9 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:34.856 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:34.868 7fb9effff700 1 -- 172.21.15.102:0/114465827 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb9f00f23d0 con 0x7fb9f010e240 2024-04-17T22:31:34.857 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:34.868 7fb9effff700 1 -- 172.21.15.102:0/114465827 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9e000d610 con 0x7fb9f010e240 2024-04-17T22:31:34.857 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:34.868 7fb9effff700 1 -- 172.21.15.102:0/114465827 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb9e000d9e0 con 0x7fb9f010e240 2024-04-17T22:31:34.857 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:34.868 7fb9f8583700 1 -- 172.21.15.102:0/114465827 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb9f010e240 msgr2=0x7fb9f010e690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:34.857 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:34.868 7fb9f8583700 1 --2- 172.21.15.102:0/114465827 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb9f010e240 0x7fb9f010e690 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 rx=0x7fb9e0002f70 tx=0x7fb9e000b9f0).stop 2024-04-17T22:31:34.857 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:34.869 7fb9f8583700 1 -- 172.21.15.102:0/114465827 shutdown_connections 2024-04-17T22:31:34.857 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:34.869 7fb9f8583700 1 --2- 172.21.15.102:0/114465827 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb9f010e240 0x7fb9f010e690 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:34.857 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:34.869 7fb9f8583700 1 --2- 172.21.15.102:0/114465827 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb9f010ecc0 0x7fb9f00f1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:34.857 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:34.869 7fb9f8583700 1 -- 172.21.15.102:0/114465827 >> 172.21.15.102:0/114465827 conn(0x7fb9f01138a0 msgr2=0x7fb9f00f4f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:34.857 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:34.869 7fb9f8583700 1 -- 172.21.15.102:0/114465827 shutdown_connections 2024-04-17T22:31:34.857 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:34.869 7fb9f8583700 1 -- 172.21.15.102:0/114465827 wait complete. 2024-04-17T22:31:34.879 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:34.892 7fb9f8583700 1 Processor -- start 2024-04-17T22:31:34.879 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:34.892 7fb9f8583700 1 -- start start 2024-04-17T22:31:34.879 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:34.892 7fb9f8583700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb9f00cc360 0x7fb9f00d51d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:34.880 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:34.892 7fb9f8583700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb9f00d5800 0x7fb9f01b4290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:34.880 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:34.892 7fb9f8583700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb9f00d5c50 0x7fb9f01b48c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:34.880 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:34.892 7fb9f8583700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9f00fa1b0 con 0x7fb9f00d5c50 2024-04-17T22:31:34.880 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:34.892 7fb9f8583700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9f00dac10 con 0x7fb9f00cc360 2024-04-17T22:31:34.880 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:34.892 7fb9f8583700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb9f00cbca0 con 0x7fb9f00d5800 2024-04-17T22:31:34.880 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:34.892 7fb9f4b7d700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb9f00d5800 0x7fb9f01b4290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:34.880 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:34.892 7fb9f537e700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb9f00cc360 0x7fb9f00d51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:34.880 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:34.892 7fb9f5b7f700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb9f00d5c50 0x7fb9f01b48c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:34.880 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:34.892 7fb9f4b7d700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb9f00d5800 0x7fb9f01b4290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:48062/0 (socket says 172.21.15.102:48062) 2024-04-17T22:31:34.880 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:34.892 7fb9f537e700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb9f00cc360 0x7fb9f00d51d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:40102/0 (socket says 172.21.15.102:40102) 2024-04-17T22:31:34.880 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:34.892 7fb9f4b7d700 1 -- 172.21.15.102:0/3343683594 learned_addr learned my addr 172.21.15.102:0/3343683594 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:31:34.880 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:34.892 7fb9f537e700 1 -- 172.21.15.102:0/3343683594 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb9f00d5800 msgr2=0x7fb9f01b4290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:34.880 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:34.892 7fb9f537e700 1 --2- 172.21.15.102:0/3343683594 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb9f00d5800 0x7fb9f01b4290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:34.880 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:34.892 7fb9f537e700 1 -- 172.21.15.102:0/3343683594 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb9f00d5c50 msgr2=0x7fb9f01b48c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:34.881 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:34.892 7fb9f537e700 1 --2- 172.21.15.102:0/3343683594 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb9f00d5c50 0x7fb9f01b48c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:34.881 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:34.892 7fb9f537e700 1 -- 172.21.15.102:0/3343683594 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9f01b4ef0 con 0x7fb9f00cc360 2024-04-17T22:31:34.881 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:34.892 7fb9f4b7d700 1 --2- 172.21.15.102:0/3343683594 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb9f00d5800 0x7fb9f01b4290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:34.881 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:34.892 7fb9f5b7f700 1 --2- 172.21.15.102:0/3343683594 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb9f00d5c50 0x7fb9f01b48c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:34.881 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:34.893 7fb9f537e700 1 --2- 172.21.15.102:0/3343683594 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb9f00cc360 0x7fb9f00d51d0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 rx=0x7fb9f00c24e0 tx=0x7fb9e000a040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:34.881 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:34.893 7fb9ee7fc700 1 -- 172.21.15.102:0/3343683594 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb9f01b4ef0 con 0x7fb9f00cc360 2024-04-17T22:31:34.881 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:34.893 7fb9ee7fc700 1 -- 172.21.15.102:0/3343683594 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9e00139f0 con 0x7fb9f00cc360 2024-04-17T22:31:34.881 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:34.893 7fb9ee7fc700 1 -- 172.21.15.102:0/3343683594 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb9e0013dc0 con 0x7fb9f00cc360 2024-04-17T22:31:34.881 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:34.893 7fb9f8583700 1 -- 172.21.15.102:0/3343683594 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9f01b51b0 con 0x7fb9f00cc360 2024-04-17T22:31:34.881 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:34.893 7fb9f8583700 1 -- 172.21.15.102:0/3343683594 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9f01bbbe0 con 0x7fb9f00cc360 2024-04-17T22:31:34.881 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:34.893 7fb9ee7fc700 1 -- 172.21.15.102:0/3343683594 <== mon.1 v2:172.21.15.102:3301/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7fb9e0027070 con 0x7fb9f00cc360 2024-04-17T22:31:34.881 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:34.893 7fb9ee7fc700 1 -- 172.21.15.102:0/3343683594 <== mon.1 v2:172.21.15.102:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 3194+0+0 (secure 0 0 0) 0x7fb9e0016810 con 0x7fb9f00cc360 2024-04-17T22:31:34.882 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:34.894 7fb9f8583700 1 -- 172.21.15.102:0/3343683594 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9f01ae990 con 0x7fb9f00cc360 2024-04-17T22:31:34.883 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:34.897 7fb9ee7fc700 1 -- 172.21.15.102:0/3343683594 <== mon.1 v2:172.21.15.102:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+110455 (secure 0 0 0) 0x7fb9e00137e0 con 0x7fb9f00cc360 2024-04-17T22:31:34.987 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.001 7fb9f8583700 1 -- 172.21.15.102:0/3343683594 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb9f00a64e0 con 0x7fb9f00cc360 2024-04-17T22:31:34.987 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.001 7fb9ee7fc700 1 -- 172.21.15.102:0/3343683594 <== mon.1 v2:172.21.15.102:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb9e000d400 con 0x7fb9f00cc360 2024-04-17T22:31:34.988 INFO:teuthology.orchestra.run.smithi102.stdout:pg_num: 64 2024-04-17T22:31:34.989 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.002 7fb9f8583700 1 -- 172.21.15.102:0/3343683594 >> v1:172.21.15.102:6816/42360 conn(0x7fb9d40145a0 legacy=0x7fb9d4016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:34.989 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.003 7fb9f8583700 1 -- 172.21.15.102:0/3343683594 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb9f00cc360 msgr2=0x7fb9f00d51d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:34.989 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.003 7fb9f8583700 1 --2- 172.21.15.102:0/3343683594 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb9f00cc360 0x7fb9f00d51d0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 rx=0x7fb9f00c24e0 tx=0x7fb9e000a040).stop 2024-04-17T22:31:34.989 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.003 7fb9f8583700 1 -- 172.21.15.102:0/3343683594 shutdown_connections 2024-04-17T22:31:34.989 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.003 7fb9f8583700 1 --2- 172.21.15.102:0/3343683594 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb9f00d5c50 0x7fb9f01b48c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:34.989 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.003 7fb9f8583700 1 --2- 172.21.15.102:0/3343683594 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb9f00cc360 0x7fb9f00d51d0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:34.989 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.003 7fb9f8583700 1 --2- 172.21.15.102:0/3343683594 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb9f00d5800 0x7fb9f01b4290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:34.989 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.003 7fb9f8583700 1 -- 172.21.15.102:0/3343683594 >> 172.21.15.102:0/3343683594 conn(0x7fb9f00c5690 msgr2=0x7fb9f00d62b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:34.989 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.003 7fb9f8583700 1 -- 172.21.15.102:0/3343683594 shutdown_connections 2024-04-17T22:31:34.990 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.003 7fb9f8583700 1 -- 172.21.15.102:0/3343683594 wait complete. 2024-04-17T22:31:34.999 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-17T22:31:35.127 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.139 7fb0048e6700 1 Processor -- start 2024-04-17T22:31:35.128 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.139 7fb0048e6700 1 -- start start 2024-04-17T22:31:35.129 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.140 7fb0048e6700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb00010e240 0x7fb00010e690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:35.129 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.140 7fb0048e6700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fb0000f1f30 0x7fb0001ab4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:35.129 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.140 7fb0048e6700 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0000f2380 con 0x7fb00010ecc0 2024-04-17T22:31:35.129 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.140 7fb0048e6700 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0000f9f10 con 0x7fb0000f1f30 2024-04-17T22:31:35.129 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.140 7fb0048e6700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb0000f9030 con 0x7fb00010e240 2024-04-17T22:31:35.129 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.140 7faffddfc700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fb0000f1f30 0x7fb0001ab4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:35.130 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.140 7faffd5fb700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb00010e240 0x7fb00010e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:35.130 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.140 7faffddfc700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fb0000f1f30 0x7fb0001ab4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:57360/0 (socket says 172.21.15.102:57360) 2024-04-17T22:31:35.130 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.140 7faffd5fb700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb00010e240 0x7fb00010e690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:48070/0 (socket says 172.21.15.102:48070) 2024-04-17T22:31:35.132 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.140 7faffddfc700 1 -- 172.21.15.102:0/2926195801 learned_addr learned my addr 172.21.15.102:0/2926195801 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:31:35.132 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.140 7faffcdfa700 1 --1- 172.21.15.102:0/2926195801 >> v1:172.21.15.102:6789/0 conn(0x7fb00010ecc0 0x7fb0000f1730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:45634/0 (socket says 172.21.15.102:45634) 2024-04-17T22:31:35.132 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.140 7faffddfc700 1 -- 172.21.15.102:0/2926195801 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb00010e240 msgr2=0x7fb00010e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:31:35.132 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.140 7faffddfc700 1 --2- 172.21.15.102:0/2926195801 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb00010e240 0x7fb00010e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:35.132 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.140 7faffddfc700 1 -- 172.21.15.102:0/2926195801 >> v1:172.21.15.102:6789/0 conn(0x7fb00010ecc0 legacy=0x7fb0000f1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:35.132 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.140 7faffddfc700 1 -- 172.21.15.102:0/2926195801 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0000f25a0 con 0x7fb0000f1f30 2024-04-17T22:31:35.132 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.140 7fafeffff700 1 -- 172.21.15.102:0/2926195801 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 46470405 0 0) 0x7faff0001110 con 0x7fb00010ecc0 2024-04-17T22:31:35.132 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.140 7faffd5fb700 1 --2- 172.21.15.102:0/2926195801 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb00010e240 0x7fb00010e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:35.133 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.140 7faffddfc700 1 --2- 172.21.15.102:0/2926195801 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb0000f1f30 0x7fb0001ab4e0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 rx=0x7fb000103580 tx=0x7faff8009e10).ready entity=mon.0 client_cookie=af208b5146b5bae3 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:35.133 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.140 7fafeffff700 1 -- 172.21.15.102:0/2926195801 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb0000f25a0 con 0x7fb0000f1f30 2024-04-17T22:31:35.133 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.140 7fafeffff700 1 -- 172.21.15.102:0/2926195801 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faff800b890 con 0x7fb0000f1f30 2024-04-17T22:31:35.133 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.140 7fafeffff700 1 -- 172.21.15.102:0/2926195801 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7faff800ba70 con 0x7fb0000f1f30 2024-04-17T22:31:35.133 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.140 7fb0048e6700 1 -- 172.21.15.102:0/2926195801 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb0000f1f30 msgr2=0x7fb0001ab4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:35.133 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.140 7fb0048e6700 1 --2- 172.21.15.102:0/2926195801 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb0000f1f30 0x7fb0001ab4e0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 rx=0x7fb000103580 tx=0x7faff8009e10).stop 2024-04-17T22:31:35.133 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.141 7fb0048e6700 1 -- 172.21.15.102:0/2926195801 shutdown_connections 2024-04-17T22:31:35.133 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.141 7fb0048e6700 1 --2- 172.21.15.102:0/2926195801 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb00010e240 0x7fb00010e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:35.133 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.141 7fb0048e6700 1 --2- 172.21.15.102:0/2926195801 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb0000f1f30 0x7fb0001ab4e0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:35.133 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.141 7fb0048e6700 1 -- 172.21.15.102:0/2926195801 >> 172.21.15.102:0/2926195801 conn(0x7fb0001138a0 msgr2=0x7fb0000f4f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:35.133 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.141 7fb0048e6700 1 -- 172.21.15.102:0/2926195801 shutdown_connections 2024-04-17T22:31:35.133 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.141 7fb0048e6700 1 -- 172.21.15.102:0/2926195801 wait complete. 2024-04-17T22:31:35.152 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.163 7fb0048e6700 1 Processor -- start 2024-04-17T22:31:35.152 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.164 7fb0048e6700 1 -- start start 2024-04-17T22:31:35.152 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.164 7fb0048e6700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb0001ad850 0x7fb0001adca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:35.152 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.164 7fb0048e6700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb0001ae2d0 0x7fb0001b4250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:35.152 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.164 7fb0048e6700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb0001b4880 0x7fb0001b4cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:35.152 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.164 7fb0048e6700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0000fa1b0 con 0x7fb0001ae2d0 2024-04-17T22:31:35.152 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.164 7fb0048e6700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0000c4890 con 0x7fb0001b4880 2024-04-17T22:31:35.152 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.164 7fb0048e6700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb0000d4b50 con 0x7fb0001ad850 2024-04-17T22:31:35.153 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.164 7faffcdfa700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb0001ae2d0 0x7fb0001b4250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:35.153 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.164 7faffddfc700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb0001b4880 0x7fb0001b4cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:35.153 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.164 7faffd5fb700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb0001ad850 0x7fb0001adca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:35.153 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.164 7faffddfc700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb0001b4880 0x7fb0001b4cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:40108/0 (socket says 172.21.15.102:40108) 2024-04-17T22:31:35.153 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.164 7faffd5fb700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb0001ad850 0x7fb0001adca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:48074/0 (socket says 172.21.15.102:48074) 2024-04-17T22:31:35.153 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.164 7faffddfc700 1 -- 172.21.15.102:0/4125801371 learned_addr learned my addr 172.21.15.102:0/4125801371 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:31:35.153 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.164 7faffcdfa700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb0001ae2d0 0x7fb0001b4250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:57362/0 (socket says 172.21.15.102:57362) 2024-04-17T22:31:35.153 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.164 7faffcdfa700 1 -- 172.21.15.102:0/4125801371 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb0001ad850 msgr2=0x7fb0001adca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:35.153 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.164 7faffcdfa700 1 --2- 172.21.15.102:0/4125801371 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb0001ad850 0x7fb0001adca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:35.153 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.164 7faffcdfa700 1 -- 172.21.15.102:0/4125801371 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb0001b4880 msgr2=0x7fb0001b4cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:35.153 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.164 7faffcdfa700 1 --2- 172.21.15.102:0/4125801371 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb0001b4880 0x7fb0001b4cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:35.153 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.164 7faffcdfa700 1 -- 172.21.15.102:0/4125801371 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0000d4590 con 0x7fb0001ae2d0 2024-04-17T22:31:35.153 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.164 7faffd5fb700 1 --2- 172.21.15.102:0/4125801371 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb0001ad850 0x7fb0001adca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:35.153 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.164 7faffddfc700 1 --2- 172.21.15.102:0/4125801371 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb0001b4880 0x7fb0001b4cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:35.154 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.164 7faffcdfa700 1 --2- 172.21.15.102:0/4125801371 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb0001ae2d0 0x7fb0001b4250 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 rx=0x7fb0000c2730 tx=0x7faff000c800).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:35.154 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.165 7fafee7fc700 1 -- 172.21.15.102:0/4125801371 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb0000d4590 con 0x7fb0001ae2d0 2024-04-17T22:31:35.154 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.165 7fafee7fc700 1 -- 172.21.15.102:0/4125801371 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faff00070d0 con 0x7fb0001ae2d0 2024-04-17T22:31:35.154 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.165 7fb0048e6700 1 -- 172.21.15.102:0/4125801371 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0001bb940 con 0x7fb0001ae2d0 2024-04-17T22:31:35.154 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.165 7fafee7fc700 1 -- 172.21.15.102:0/4125801371 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7faff0005560 con 0x7fb0001ae2d0 2024-04-17T22:31:35.154 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.165 7fb0048e6700 1 -- 172.21.15.102:0/4125801371 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0001bbd20 con 0x7fb0001ae2d0 2024-04-17T22:31:35.154 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.165 7fafee7fc700 1 -- 172.21.15.102:0/4125801371 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7faff0005760 con 0x7fb0001ae2d0 2024-04-17T22:31:35.154 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.165 7fafee7fc700 1 -- 172.21.15.102:0/4125801371 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 3194+0+0 (secure 0 0 0) 0x7faff000f6c0 con 0x7fb0001ae2d0 2024-04-17T22:31:35.154 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.166 7fb0048e6700 1 -- 172.21.15.102:0/4125801371 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0000f0800 con 0x7fb0001ae2d0 2024-04-17T22:31:35.154 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.169 7fafee7fc700 1 -- 172.21.15.102:0/4125801371 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+110455 (secure 0 0 0) 0x7faff001d360 con 0x7fb0001ae2d0 2024-04-17T22:31:35.261 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.275 7fb0048e6700 1 -- 172.21.15.102:0/4125801371 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb0001ae970 con 0x7fb0001ae2d0 2024-04-17T22:31:35.261 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.275 7fafee7fc700 1 -- 172.21.15.102:0/4125801371 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb0000f1440 con 0x7fb0001ae2d0 2024-04-17T22:31:35.262 INFO:teuthology.orchestra.run.smithi102.stdout:pg_num: 64 2024-04-17T22:31:35.263 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.276 7fb0048e6700 1 -- 172.21.15.102:0/4125801371 >> v1:172.21.15.102:6816/42360 conn(0x7fafdc0145a0 legacy=0x7fafdc016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:35.263 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.276 7fb0048e6700 1 -- 172.21.15.102:0/4125801371 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb0001ae2d0 msgr2=0x7fb0001b4250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:35.263 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.276 7fb0048e6700 1 --2- 172.21.15.102:0/4125801371 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb0001ae2d0 0x7fb0001b4250 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 rx=0x7fb0000c2730 tx=0x7faff000c800).stop 2024-04-17T22:31:35.263 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.277 7fb0048e6700 1 -- 172.21.15.102:0/4125801371 shutdown_connections 2024-04-17T22:31:35.263 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.277 7fb0048e6700 1 --2- 172.21.15.102:0/4125801371 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb0001b4880 0x7fb0001b4cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:35.263 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.277 7fb0048e6700 1 --2- 172.21.15.102:0/4125801371 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb0001ad850 0x7fb0001adca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:35.263 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.277 7fb0048e6700 1 --2- 172.21.15.102:0/4125801371 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb0001ae2d0 0x7fb0001b4250 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:35.264 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.277 7fb0048e6700 1 -- 172.21.15.102:0/4125801371 >> 172.21.15.102:0/4125801371 conn(0x7fb0000c6ae0 msgr2=0x7fb0001a84e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:35.264 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.277 7fb0048e6700 1 -- 172.21.15.102:0/4125801371 shutdown_connections 2024-04-17T22:31:35.264 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.277 7fb0048e6700 1 -- 172.21.15.102:0/4125801371 wait complete. 2024-04-17T22:31:35.274 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-17T22:31:35.401 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.413 7f6dbe454700 1 Processor -- start 2024-04-17T22:31:35.401 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.413 7f6dbe454700 1 -- start start 2024-04-17T22:31:35.401 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.414 7f6dbe454700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6db810c8f0 0x7f6db81a62e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:35.401 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.414 7f6dbe454700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f6db81a6910 0x7f6db81a8d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:35.401 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.414 7f6dbe454700 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6db81a1bf0 con 0x7f6db80f5750 2024-04-17T22:31:35.401 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.414 7f6dbe454700 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6db80f93c0 con 0x7f6db81a6910 2024-04-17T22:31:35.402 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.414 7f6dbe454700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6db810cd40 con 0x7f6db810c8f0 2024-04-17T22:31:35.402 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.414 7f6daffff700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6db810c8f0 0x7f6db81a62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:35.402 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.414 7f6db7887700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f6db81a6910 0x7f6db81a8d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:35.402 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.414 7f6daffff700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6db810c8f0 0x7f6db81a62e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:40122/0 (socket says 172.21.15.102:40122) 2024-04-17T22:31:35.402 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.414 7f6db7887700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f6db81a6910 0x7f6db81a8d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:57376/0 (socket says 172.21.15.102:57376) 2024-04-17T22:31:35.402 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.414 7f6db7086700 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f6db80f5750 0x7f6db810c0f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:45640/0 (socket says 172.21.15.102:45640) 2024-04-17T22:31:35.402 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.414 7f6daffff700 1 -- 172.21.15.102:0/242045423 learned_addr learned my addr 172.21.15.102:0/242045423 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:31:35.402 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.414 7f6daffff700 1 -- 172.21.15.102:0/242045423 >> v2:172.21.15.102:3300/0 conn(0x7f6db81a6910 msgr2=0x7f6db81a8d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:31:35.402 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.414 7f6daffff700 1 --2- 172.21.15.102:0/242045423 >> v2:172.21.15.102:3300/0 conn(0x7f6db81a6910 0x7f6db81a8d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:35.402 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.414 7f6daffff700 1 -- 172.21.15.102:0/242045423 >> v1:172.21.15.102:6789/0 conn(0x7f6db80f5750 legacy=0x7f6db810c0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:35.402 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.414 7f6daffff700 1 -- 172.21.15.102:0/242045423 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6db81aa3b0 con 0x7f6db810c8f0 2024-04-17T22:31:35.402 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.414 7f6db6885700 1 -- 172.21.15.102:0/242045423 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3326349267 0 0) 0x7f6db81a1bf0 con 0x7f6db80f5750 2024-04-17T22:31:35.403 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.414 7f6db7887700 1 --2- 172.21.15.102:0/242045423 >> v2:172.21.15.102:3300/0 conn(0x7f6db81a6910 0x7f6db81a8d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:35.403 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.414 7f6daffff700 1 --2- 172.21.15.102:0/242045423 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6db810c8f0 0x7f6db81a62e0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 rx=0x7f6da0002f70 tx=0x7f6da00099d0).ready entity=mon.1 client_cookie=92a820166d072bfb server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:35.403 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.414 7f6db6885700 1 -- 172.21.15.102:0/242045423 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f6db81aa3b0 con 0x7f6db810c8f0 2024-04-17T22:31:35.403 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.414 7f6db6885700 1 -- 172.21.15.102:0/242045423 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6da000f510 con 0x7f6db810c8f0 2024-04-17T22:31:35.403 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.414 7f6db6885700 1 -- 172.21.15.102:0/242045423 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f6da000f6f0 con 0x7f6db810c8f0 2024-04-17T22:31:35.403 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.414 7f6dbe454700 1 -- 172.21.15.102:0/242045423 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6db810c8f0 msgr2=0x7f6db81a62e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:35.403 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.414 7f6dbe454700 1 --2- 172.21.15.102:0/242045423 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6db810c8f0 0x7f6db81a62e0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 rx=0x7f6da0002f70 tx=0x7f6da00099d0).stop 2024-04-17T22:31:35.403 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.415 7f6dbe454700 1 -- 172.21.15.102:0/242045423 shutdown_connections 2024-04-17T22:31:35.403 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.415 7f6dbe454700 1 --2- 172.21.15.102:0/242045423 >> v2:172.21.15.102:3300/0 conn(0x7f6db81a6910 0x7f6db81a8d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:35.403 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.415 7f6dbe454700 1 --2- 172.21.15.102:0/242045423 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6db810c8f0 0x7f6db81a62e0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:35.403 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.415 7f6dbe454700 1 -- 172.21.15.102:0/242045423 >> 172.21.15.102:0/242045423 conn(0x7f6db81138a0 msgr2=0x7f6db80f4f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:35.403 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.415 7f6dbe454700 1 -- 172.21.15.102:0/242045423 shutdown_connections 2024-04-17T22:31:35.403 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.415 7f6dbe454700 1 -- 172.21.15.102:0/242045423 wait complete. 2024-04-17T22:31:35.424 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.438 7f6dbe454700 1 Processor -- start 2024-04-17T22:31:35.424 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.438 7f6dbe454700 1 -- start start 2024-04-17T22:31:35.426 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.438 7f6dbe454700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6db80dfc20 0x7f6db80ca1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:35.426 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.438 7f6dbe454700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6db80ca7d0 0x7f6db81b1810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:35.426 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.438 7f6dbe454700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f6db80cac20 0x7f6db81b1e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:35.426 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.438 7f6dbe454700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6db81a3a60 con 0x7f6db80dfc20 2024-04-17T22:31:35.426 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.439 7f6dbe454700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6db81b2470 con 0x7f6db80ca7d0 2024-04-17T22:31:35.426 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.439 7f6dbe454700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6db81b26a0 con 0x7f6db80cac20 2024-04-17T22:31:35.427 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.439 7f6db7086700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6db80dfc20 0x7f6db80ca1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:35.427 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.439 7f6daffff700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6db80ca7d0 0x7f6db81b1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:35.427 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.439 7f6daffff700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6db80ca7d0 0x7f6db81b1810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:40136/0 (socket says 172.21.15.102:40136) 2024-04-17T22:31:35.427 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.439 7f6daffff700 1 -- 172.21.15.102:0/1068734488 learned_addr learned my addr 172.21.15.102:0/1068734488 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:31:35.427 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.439 7f6db7887700 1 --2- 172.21.15.102:0/1068734488 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f6db80cac20 0x7f6db81b1e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:35.427 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.439 7f6db7086700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6db80dfc20 0x7f6db80ca1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:57388/0 (socket says 172.21.15.102:57388) 2024-04-17T22:31:35.427 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.439 7f6daffff700 1 -- 172.21.15.102:0/1068734488 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f6db80cac20 msgr2=0x7f6db81b1e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:35.427 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.439 7f6daffff700 1 --2- 172.21.15.102:0/1068734488 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f6db80cac20 0x7f6db81b1e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:35.427 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.439 7f6daffff700 1 -- 172.21.15.102:0/1068734488 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6db80dfc20 msgr2=0x7f6db80ca1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:35.427 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.439 7f6daffff700 1 --2- 172.21.15.102:0/1068734488 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6db80dfc20 0x7f6db80ca1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:35.427 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.439 7f6daffff700 1 -- 172.21.15.102:0/1068734488 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6db81b8f00 con 0x7f6db80ca7d0 2024-04-17T22:31:35.427 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.439 7f6daffff700 1 --2- 172.21.15.102:0/1068734488 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6db80ca7d0 0x7f6db81b1810 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 rx=0x7f6da0002f70 tx=0x7f6da00068c0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:35.427 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.439 7f6db5082700 1 -- 172.21.15.102:0/1068734488 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f6db81b8f00 con 0x7f6db80ca7d0 2024-04-17T22:31:35.427 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.439 7f6db5082700 1 -- 172.21.15.102:0/1068734488 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6da00118a0 con 0x7f6db80ca7d0 2024-04-17T22:31:35.428 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.439 7f6dbe454700 1 -- 172.21.15.102:0/1068734488 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6db81b9100 con 0x7f6db80ca7d0 2024-04-17T22:31:35.428 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.439 7f6db5082700 1 -- 172.21.15.102:0/1068734488 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f6da0011c70 con 0x7f6db80ca7d0 2024-04-17T22:31:35.428 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.439 7f6dbe454700 1 -- 172.21.15.102:0/1068734488 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6db81b94e0 con 0x7f6db80ca7d0 2024-04-17T22:31:35.428 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.440 7f6db5082700 1 -- 172.21.15.102:0/1068734488 <== mon.1 v2:172.21.15.102:3301/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7f6da0019030 con 0x7f6db80ca7d0 2024-04-17T22:31:35.428 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.440 7f6db5082700 1 -- 172.21.15.102:0/1068734488 <== mon.1 v2:172.21.15.102:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 3194+0+0 (secure 0 0 0) 0x7f6da0018040 con 0x7f6db80ca7d0 2024-04-17T22:31:35.428 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.440 7f6dbe454700 1 -- 172.21.15.102:0/1068734488 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6db80f0800 con 0x7f6db80ca7d0 2024-04-17T22:31:35.429 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.443 7f6db5082700 1 -- 172.21.15.102:0/1068734488 <== mon.1 v2:172.21.15.102:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+110455 (secure 0 0 0) 0x7f6da0011690 con 0x7f6db80ca7d0 2024-04-17T22:31:35.575 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.588 7f6dbe454700 1 -- 172.21.15.102:0/1068734488 >> v1:172.21.15.102:6816/42360 conn(0x7f6d9c0145a0 legacy=0x7f6d9c016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:35.575 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.589 7f6dbe454700 1 -- 172.21.15.102:0/1068734488 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6db80ca7d0 msgr2=0x7f6db81b1810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:35.575 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.589 7f6dbe454700 1 --2- 172.21.15.102:0/1068734488 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6db80ca7d0 0x7f6db81b1810 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 rx=0x7f6da0002f70 tx=0x7f6da00068c0).stop 2024-04-17T22:31:35.575 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.589 7f6dbe454700 1 -- 172.21.15.102:0/1068734488 shutdown_connections 2024-04-17T22:31:35.575 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.589 7f6dbe454700 1 --2- 172.21.15.102:0/1068734488 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f6db80cac20 0x7f6db81b1e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:35.575 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.589 7f6dbe454700 1 --2- 172.21.15.102:0/1068734488 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6db80dfc20 0x7f6db80ca1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:35.575 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.589 7f6dbe454700 1 --2- 172.21.15.102:0/1068734488 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6db80ca7d0 0x7f6db81b1810 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:35.576 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.589 7f6dbe454700 1 -- 172.21.15.102:0/1068734488 >> 172.21.15.102:0/1068734488 conn(0x7f6db80c5a40 msgr2=0x7f6db80d6850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:35.576 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.589 7f6dbe454700 1 -- 172.21.15.102:0/1068734488 shutdown_connections 2024-04-17T22:31:35.576 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.589 7f6dbe454700 1 -- 172.21.15.102:0/1068734488 wait complete. 2024-04-17T22:31:35.576 INFO:teuthology.orchestra.run.smithi102.stderr:no valid command found; 10 closest matches: 2024-04-17T22:31:35.576 INFO:teuthology.orchestra.run.smithi102.stderr:osd perf 2024-04-17T22:31:35.576 INFO:teuthology.orchestra.run.smithi102.stderr:osd df {plain|tree} {class|name} {} 2024-04-17T22:31:35.576 INFO:teuthology.orchestra.run.smithi102.stderr:osd blocked-by 2024-04-17T22:31:35.576 INFO:teuthology.orchestra.run.smithi102.stderr:osd pool stats {} 2024-04-17T22:31:35.576 INFO:teuthology.orchestra.run.smithi102.stderr:osd pool scrub [...] 2024-04-17T22:31:35.576 INFO:teuthology.orchestra.run.smithi102.stderr:osd pool deep-scrub [...] 2024-04-17T22:31:35.577 INFO:teuthology.orchestra.run.smithi102.stderr:osd pool repair [...] 2024-04-17T22:31:35.577 INFO:teuthology.orchestra.run.smithi102.stderr:osd pool force-recovery [...] 2024-04-17T22:31:35.577 INFO:teuthology.orchestra.run.smithi102.stderr:osd pool force-backfill [...] 2024-04-17T22:31:35.577 INFO:teuthology.orchestra.run.smithi102.stderr:osd pool cancel-force-recovery [...] 2024-04-17T22:31:35.577 INFO:teuthology.orchestra.run.smithi102.stderr:Error EINVAL: invalid command 2024-04-17T22:31:35.585 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-17T22:31:35.586 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blacklist ls 2024-04-17T22:31:35.713 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.725 7f8ec1dc1700 1 Processor -- start 2024-04-17T22:31:35.713 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.725 7f8ec1dc1700 1 -- start start 2024-04-17T22:31:35.713 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.725 7f8ec1dc1700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f8ebc0f5840 0x7f8ebc10b010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:35.713 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.725 7f8ec1dc1700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f8ebc10b640 0x7f8ebc19faf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:35.713 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.725 7f8ec1dc1700 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ebc103b40 con 0x7f8ebc10ba90 2024-04-17T22:31:35.713 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.725 7f8ec1dc1700 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ebc10bee0 con 0x7f8ebc0f5840 2024-04-17T22:31:35.713 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.725 7f8ec1dc1700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8ebc0fa370 con 0x7f8ebc10b640 2024-04-17T22:31:35.713 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.725 7f8ebaab2700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f8ebc0f5840 0x7f8ebc10b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:35.713 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.725 7f8eba2b1700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f8ebc10b640 0x7f8ebc19faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:35.714 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.725 7f8ebaab2700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f8ebc0f5840 0x7f8ebc10b010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:57396/0 (socket says 172.21.15.102:57396) 2024-04-17T22:31:35.714 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.725 7f8ebb2b3700 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f8ebc10ba90 0x7f8ebc1a0120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:45648/0 (socket says 172.21.15.102:45648) 2024-04-17T22:31:35.714 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.725 7f8eba2b1700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f8ebc10b640 0x7f8ebc19faf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:48088/0 (socket says 172.21.15.102:48088) 2024-04-17T22:31:35.714 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.725 7f8ebaab2700 1 -- 172.21.15.102:0/2536844641 learned_addr learned my addr 172.21.15.102:0/2536844641 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:31:35.714 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.726 7f8ebaab2700 1 -- 172.21.15.102:0/2536844641 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f8ebc10b640 msgr2=0x7f8ebc19faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:31:35.714 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.726 7f8ebaab2700 1 --2- 172.21.15.102:0/2536844641 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f8ebc10b640 0x7f8ebc19faf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:35.714 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.726 7f8ebaab2700 1 -- 172.21.15.102:0/2536844641 >> v1:172.21.15.102:6789/0 conn(0x7f8ebc10ba90 legacy=0x7f8ebc1a0120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:35.714 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.726 7f8ebaab2700 1 -- 172.21.15.102:0/2536844641 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ebc1a0920 con 0x7f8ebc0f5840 2024-04-17T22:31:35.714 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.726 7f8eb9ab0700 1 -- 172.21.15.102:0/2536844641 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2876201344 0 0) 0x7f8eb4001490 con 0x7f8ebc10ba90 2024-04-17T22:31:35.714 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.726 7f8eba2b1700 1 --2- 172.21.15.102:0/2536844641 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f8ebc10b640 0x7f8ebc19faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:35.714 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.726 7f8ebaab2700 1 --2- 172.21.15.102:0/2536844641 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8ebc0f5840 0x7f8ebc10b010 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 rx=0x7f8ebc103d60 tx=0x7f8eb0009a80).ready entity=mon.0 client_cookie=4f98a0219c10c66d server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:35.714 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.726 7f8eb9ab0700 1 -- 172.21.15.102:0/2536844641 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8ebc1a0920 con 0x7f8ebc0f5840 2024-04-17T22:31:35.714 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.726 7f8eb9ab0700 1 -- 172.21.15.102:0/2536844641 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8eb000f510 con 0x7f8ebc0f5840 2024-04-17T22:31:35.714 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.726 7f8eb9ab0700 1 -- 172.21.15.102:0/2536844641 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8eb000f6f0 con 0x7f8ebc0f5840 2024-04-17T22:31:35.715 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.726 7f8ec1dc1700 1 -- 172.21.15.102:0/2536844641 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8ebc0f5840 msgr2=0x7f8ebc10b010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:35.715 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.726 7f8ec1dc1700 1 --2- 172.21.15.102:0/2536844641 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8ebc0f5840 0x7f8ebc10b010 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 rx=0x7f8ebc103d60 tx=0x7f8eb0009a80).stop 2024-04-17T22:31:35.715 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.727 7f8ec1dc1700 1 -- 172.21.15.102:0/2536844641 shutdown_connections 2024-04-17T22:31:35.715 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.727 7f8ec1dc1700 1 --2- 172.21.15.102:0/2536844641 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8ebc0f5840 0x7f8ebc10b010 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:35.715 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.727 7f8ec1dc1700 1 --2- 172.21.15.102:0/2536844641 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f8ebc10b640 0x7f8ebc19faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:35.715 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.727 7f8ec1dc1700 1 -- 172.21.15.102:0/2536844641 >> 172.21.15.102:0/2536844641 conn(0x7f8ebc1138a0 msgr2=0x7f8ebc0f4f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:35.715 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.727 7f8ec1dc1700 1 -- 172.21.15.102:0/2536844641 shutdown_connections 2024-04-17T22:31:35.715 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.727 7f8ec1dc1700 1 -- 172.21.15.102:0/2536844641 wait complete. 2024-04-17T22:31:35.738 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.749 7f8ec1dc1700 1 Processor -- start 2024-04-17T22:31:35.738 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.749 7f8ec1dc1700 1 -- start start 2024-04-17T22:31:35.738 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.750 7f8ec1dc1700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f8ebc0d3090 0x7f8ebc0d34e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:35.738 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.750 7f8ec1dc1700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f8ebc0d3b10 0x7f8ebc1af8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:35.738 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.750 7f8ec1dc1700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8ebc1afef0 0x7f8ebc1b0340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:35.739 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.750 7f8ec1dc1700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ebc1b0970 con 0x7f8ebc1afef0 2024-04-17T22:31:35.739 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.750 7f8ec1dc1700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ebc0c3380 con 0x7f8ebc0d3b10 2024-04-17T22:31:35.739 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.750 7f8ec1dc1700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8ebc0d9510 con 0x7f8ebc0d3090 2024-04-17T22:31:35.739 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.750 7f8ebaab2700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f8ebc0d3090 0x7f8ebc0d34e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:35.739 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.750 7f8eba2b1700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f8ebc0d3b10 0x7f8ebc1af8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:35.739 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.750 7f8ebb2b3700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8ebc1afef0 0x7f8ebc1b0340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:35.739 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.750 7f8ebaab2700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f8ebc0d3090 0x7f8ebc0d34e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:48098/0 (socket says 172.21.15.102:48098) 2024-04-17T22:31:35.739 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.750 7f8eba2b1700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f8ebc0d3b10 0x7f8ebc1af8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:40142/0 (socket says 172.21.15.102:40142) 2024-04-17T22:31:35.739 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.750 7f8ebaab2700 1 -- 172.21.15.102:0/1440306283 learned_addr learned my addr 172.21.15.102:0/1440306283 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:31:35.739 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.750 7f8ebb2b3700 1 -- 172.21.15.102:0/1440306283 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f8ebc0d3090 msgr2=0x7f8ebc0d34e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:35.739 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.750 7f8ebb2b3700 1 --2- 172.21.15.102:0/1440306283 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f8ebc0d3090 0x7f8ebc0d34e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:35.739 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.750 7f8ebb2b3700 1 -- 172.21.15.102:0/1440306283 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f8ebc0d3b10 msgr2=0x7f8ebc1af8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:35.739 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.750 7f8ebb2b3700 1 --2- 172.21.15.102:0/1440306283 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f8ebc0d3b10 0x7f8ebc1af8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:35.740 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.750 7f8ebb2b3700 1 -- 172.21.15.102:0/1440306283 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ebc0d90f0 con 0x7f8ebc1afef0 2024-04-17T22:31:35.740 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.750 7f8eba2b1700 1 --2- 172.21.15.102:0/1440306283 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f8ebc0d3b10 0x7f8ebc1af8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:35.740 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.750 7f8ebb2b3700 1 --2- 172.21.15.102:0/1440306283 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8ebc1afef0 0x7f8ebc1b0340 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 rx=0x7f8ebc0c5c60 tx=0x7f8eb4001fd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:35.740 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.750 7f8eabfff700 1 -- 172.21.15.102:0/1440306283 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8ebc0d90f0 con 0x7f8ebc1afef0 2024-04-17T22:31:35.740 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.751 7f8eabfff700 1 -- 172.21.15.102:0/1440306283 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8eb4005ad0 con 0x7f8ebc1afef0 2024-04-17T22:31:35.740 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.751 7f8ec1dc1700 1 -- 172.21.15.102:0/1440306283 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ebc1a1410 con 0x7f8ebc1afef0 2024-04-17T22:31:35.740 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.751 7f8eabfff700 1 -- 172.21.15.102:0/1440306283 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8eb4014070 con 0x7f8ebc1afef0 2024-04-17T22:31:35.740 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.751 7f8ec1dc1700 1 -- 172.21.15.102:0/1440306283 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ebc1b7150 con 0x7f8ebc1afef0 2024-04-17T22:31:35.740 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.751 7f8eabfff700 1 -- 172.21.15.102:0/1440306283 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7f8eb40070d0 con 0x7f8ebc1afef0 2024-04-17T22:31:35.740 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.751 7f8eabfff700 1 -- 172.21.15.102:0/1440306283 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 3194+0+0 (secure 0 0 0) 0x7f8eb4019100 con 0x7f8ebc1afef0 2024-04-17T22:31:35.741 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.752 7f8ec1dc1700 1 -- 172.21.15.102:0/1440306283 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ebc0f0800 con 0x7f8ebc1afef0 2024-04-17T22:31:35.741 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.755 7f8eabfff700 1 -- 172.21.15.102:0/1440306283 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+110455 (secure 0 0 0) 0x7f8eb400e0d0 con 0x7f8ebc1afef0 2024-04-17T22:31:35.860 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.874 7f8ec1dc1700 1 -- 172.21.15.102:0/1440306283 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd blacklist ls"} v 0) v1 -- 0x7f8ebc0d5cc0 con 0x7f8ebc1afef0 2024-04-17T22:31:35.860 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.874 7f8eabfff700 1 -- 172.21.15.102:0/1440306283 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd blacklist ls"}]=0 listed 0 entries v25) v1 ==== 80+0+0 (secure 0 0 0) 0x7f8ebc114420 con 0x7f8ebc1afef0 2024-04-17T22:31:35.862 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.876 7f8ec1dc1700 1 -- 172.21.15.102:0/1440306283 >> v1:172.21.15.102:6816/42360 conn(0x7f8e980145a0 legacy=0x7f8e98016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:35.862 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.876 7f8ec1dc1700 1 -- 172.21.15.102:0/1440306283 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8ebc1afef0 msgr2=0x7f8ebc1b0340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:35.862 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.876 7f8ec1dc1700 1 --2- 172.21.15.102:0/1440306283 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8ebc1afef0 0x7f8ebc1b0340 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 rx=0x7f8ebc0c5c60 tx=0x7f8eb4001fd0).stop 2024-04-17T22:31:35.862 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.876 7f8ec1dc1700 1 -- 172.21.15.102:0/1440306283 shutdown_connections 2024-04-17T22:31:35.862 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.876 7f8ec1dc1700 1 --2- 172.21.15.102:0/1440306283 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8ebc1afef0 0x7f8ebc1b0340 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:35.863 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.876 7f8ec1dc1700 1 --2- 172.21.15.102:0/1440306283 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f8ebc0d3090 0x7f8ebc0d34e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:35.863 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.876 7f8ec1dc1700 1 --2- 172.21.15.102:0/1440306283 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f8ebc0d3b10 0x7f8ebc1af8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:35.863 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.876 7f8ec1dc1700 1 -- 172.21.15.102:0/1440306283 >> 172.21.15.102:0/1440306283 conn(0x7f8ebc0dff60 msgr2=0x7f8ebc0c1e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:35.863 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.876 7f8ec1dc1700 1 -- 172.21.15.102:0/1440306283 shutdown_connections 2024-04-17T22:31:35.863 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:35.876 7f8ec1dc1700 1 -- 172.21.15.102:0/1440306283 wait complete. 2024-04-17T22:31:35.863 INFO:teuthology.orchestra.run.smithi102.stderr:listed 0 entries 2024-04-17T22:31:35.872 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi190.front.sepia.ceph.com... 2024-04-17T22:31:35.873 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T22:31:35.873 DEBUG:teuthology.orchestra.run.smithi190:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-17T22:31:35.914 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi190.stderr:ceph-fuse[42802]: fuse finished with error 0 and tester_r 0 2024-04-17T22:31:35.915 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi190.front.sepia.ceph.com 2024-04-17T22:31:35.915 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T22:31:35.915 DEBUG:teuthology.orchestra.run.smithi190:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-17T22:31:36.044 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T22:31:36.172 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.184 7f7f3bd83700 1 Processor -- start 2024-04-17T22:31:36.172 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.184 7f7f3bd83700 1 -- start start 2024-04-17T22:31:36.172 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.184 7f7f3bd83700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f7f341a64d0 0x7f7f341a8930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:36.172 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.184 7f7f3bd83700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f7f341a8f60 0x7f7f341ab3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:36.172 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.184 7f7f3bd83700 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f3410eeb0 con 0x7f7f3410e260 2024-04-17T22:31:36.172 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.184 7f7f3bd83700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f340f9f30 con 0x7f7f341a64d0 2024-04-17T22:31:36.173 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.184 7f7f3bd83700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7f340f9050 con 0x7f7f341a8f60 2024-04-17T22:31:36.173 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.184 7f7f33fff700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f7f341a64d0 0x7f7f341a8930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:36.173 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.184 7f7f3937f700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f7f341a8f60 0x7f7f341ab3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:36.173 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.184 7f7f38b7e700 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f7f3410e260 0x7f7f3410e6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:45660/0 (socket says 172.21.15.102:45660) 2024-04-17T22:31:36.173 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.184 7f7f38b7e700 1 -- 172.21.15.102:0/3633989988 learned_addr learned my addr 172.21.15.102:0/3633989988 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-17T22:31:36.173 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.184 7f7f33fff700 1 -- 172.21.15.102:0/3633989988 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f7f341a8f60 msgr2=0x7f7f341ab3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:31:36.173 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.184 7f7f33fff700 1 --2- 172.21.15.102:0/3633989988 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f7f341a8f60 0x7f7f341ab3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:36.173 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.184 7f7f33fff700 1 -- 172.21.15.102:0/3633989988 >> v1:172.21.15.102:6789/0 conn(0x7f7f3410e260 legacy=0x7f7f3410e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:36.173 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.184 7f7f337fe700 1 -- 172.21.15.102:0/3633989988 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1379800646 0 0) 0x7f7f28001110 con 0x7f7f3410e260 2024-04-17T22:31:36.173 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.184 7f7f33fff700 1 -- 172.21.15.102:0/3633989988 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f341aca00 con 0x7f7f341a64d0 2024-04-17T22:31:36.173 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.184 7f7f3937f700 1 --2- 172.21.15.102:0/3633989988 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f7f341a8f60 0x7f7f341ab3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:36.174 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.184 7f7f33fff700 1 --2- 172.21.15.102:0/3633989988 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f7f341a64d0 0x7f7f341a8930 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 rx=0x7f7f340f4490 tx=0x7f7f2000ba50).ready entity=mon.1 client_cookie=5b947d2b74fad59c server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:36.174 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.185 7f7f337fe700 1 -- 172.21.15.102:0/3633989988 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f7f341aca00 con 0x7f7f341a64d0 2024-04-17T22:31:36.174 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.185 7f7f337fe700 1 -- 172.21.15.102:0/3633989988 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f2000d570 con 0x7f7f341a64d0 2024-04-17T22:31:36.174 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.185 7f7f337fe700 1 -- 172.21.15.102:0/3633989988 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f7f2000d940 con 0x7f7f341a64d0 2024-04-17T22:31:36.174 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.185 7f7f3bd83700 1 -- 172.21.15.102:0/3633989988 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f7f341a64d0 msgr2=0x7f7f341a8930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:36.174 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.185 7f7f3bd83700 1 --2- 172.21.15.102:0/3633989988 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f7f341a64d0 0x7f7f341a8930 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 rx=0x7f7f340f4490 tx=0x7f7f2000ba50).stop 2024-04-17T22:31:36.174 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.185 7f7f3bd83700 1 -- 172.21.15.102:0/3633989988 shutdown_connections 2024-04-17T22:31:36.174 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.185 7f7f3bd83700 1 --2- 172.21.15.102:0/3633989988 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f7f341a8f60 0x7f7f341ab3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:36.174 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.185 7f7f3bd83700 1 --2- 172.21.15.102:0/3633989988 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f7f341a64d0 0x7f7f341a8930 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:36.174 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.185 7f7f3bd83700 1 -- 172.21.15.102:0/3633989988 >> 172.21.15.102:0/3633989988 conn(0x7f7f341138c0 msgr2=0x7f7f340f4f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:36.175 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.185 7f7f3bd83700 1 -- 172.21.15.102:0/3633989988 shutdown_connections 2024-04-17T22:31:36.175 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.185 7f7f3bd83700 1 -- 172.21.15.102:0/3633989988 wait complete. 2024-04-17T22:31:36.197 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.208 7f7f3bd83700 1 Processor -- start 2024-04-17T22:31:36.197 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.208 7f7f3bd83700 1 -- start start 2024-04-17T22:31:36.197 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.208 7f7f3bd83700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f7f340cdd70 0x7f7f340d5fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:36.197 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.208 7f7f3bd83700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f7f340d6600 0x7f7f341b3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:36.197 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.208 7f7f3bd83700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7f340d6a50 0x7f7f341b4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:36.197 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.208 7f7f3bd83700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f340fa1d0 con 0x7f7f340d6a50 2024-04-17T22:31:36.197 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.208 7f7f3bd83700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f340ce1c0 con 0x7f7f340cdd70 2024-04-17T22:31:36.197 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.208 7f7f3bd83700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7f341a8f00 con 0x7f7f340d6600 2024-04-17T22:31:36.197 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.208 7f7f38b7e700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f7f340cdd70 0x7f7f340d5fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:36.197 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.208 7f7f38b7e700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f7f340cdd70 0x7f7f340d5fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:40154/0 (socket says 172.21.15.102:40154) 2024-04-17T22:31:36.197 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.208 7f7f3937f700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7f340d6a50 0x7f7f341b4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:36.198 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.208 7f7f38b7e700 1 -- 172.21.15.102:0/3632301855 learned_addr learned my addr 172.21.15.102:0/3632301855 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:31:36.198 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.208 7f7f33fff700 1 --2- 172.21.15.102:0/3632301855 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f7f340d6600 0x7f7f341b3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:36.198 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.209 7f7f38b7e700 1 -- 172.21.15.102:0/3632301855 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f7f340d6600 msgr2=0x7f7f341b3fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:36.198 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.209 7f7f38b7e700 1 --2- 172.21.15.102:0/3632301855 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f7f340d6600 0x7f7f341b3fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:36.198 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.209 7f7f38b7e700 1 -- 172.21.15.102:0/3632301855 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7f340d6a50 msgr2=0x7f7f341b4600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:36.198 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.209 7f7f38b7e700 1 --2- 172.21.15.102:0/3632301855 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7f340d6a50 0x7f7f341b4600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:36.198 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.209 7f7f38b7e700 1 -- 172.21.15.102:0/3632301855 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f340cd320 con 0x7f7f340cdd70 2024-04-17T22:31:36.198 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.209 7f7f33fff700 1 --2- 172.21.15.102:0/3632301855 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f7f340d6600 0x7f7f341b3fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:36.198 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.209 7f7f3937f700 1 --2- 172.21.15.102:0/3632301855 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7f340d6a50 0x7f7f341b4600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:36.199 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.209 7f7f38b7e700 1 --2- 172.21.15.102:0/3632301855 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f7f340cdd70 0x7f7f340d5fd0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 rx=0x7f7f340d75c0 tx=0x7f7f2800c860).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:36.199 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.209 7f7f31ffb700 1 -- 172.21.15.102:0/3632301855 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f7f340cd320 con 0x7f7f340cdd70 2024-04-17T22:31:36.199 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.209 7f7f31ffb700 1 -- 172.21.15.102:0/3632301855 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f280072e0 con 0x7f7f340cdd70 2024-04-17T22:31:36.199 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.209 7f7f3bd83700 1 -- 172.21.15.102:0/3632301855 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f341b4c30 con 0x7f7f340cdd70 2024-04-17T22:31:36.199 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.209 7f7f31ffb700 1 -- 172.21.15.102:0/3632301855 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f7f28005810 con 0x7f7f340cdd70 2024-04-17T22:31:36.199 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.209 7f7f3bd83700 1 -- 172.21.15.102:0/3632301855 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f341bb7d0 con 0x7f7f340cdd70 2024-04-17T22:31:36.199 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.209 7f7f31ffb700 1 -- 172.21.15.102:0/3632301855 <== mon.1 v2:172.21.15.102:3301/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7f7f28005a10 con 0x7f7f340cdd70 2024-04-17T22:31:36.199 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.209 7f7f31ffb700 1 -- 172.21.15.102:0/3632301855 <== mon.1 v2:172.21.15.102:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 3194+0+0 (secure 0 0 0) 0x7f7f28014b20 con 0x7f7f340cdd70 2024-04-17T22:31:36.199 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.210 7f7f3bd83700 1 -- 172.21.15.102:0/3632301855 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f340f0820 con 0x7f7f340cdd70 2024-04-17T22:31:36.200 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.213 7f7f31ffb700 1 -- 172.21.15.102:0/3632301855 <== mon.1 v2:172.21.15.102:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+110455 (secure 0 0 0) 0x7f7f280070d0 con 0x7f7f340cdd70 2024-04-17T22:31:36.334 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.347 7f7f3bd83700 1 -- 172.21.15.102:0/3632301855 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7f341ae710 con 0x7f7f340cdd70 2024-04-17T22:31:36.334 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.348 7f7f31ffb700 1 -- 172.21.15.102:0/3632301855 <== mon.1 v2:172.21.15.102:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) v1 ==== 94+0+1704 (secure 0 0 0) 0x7f7f28000c30 con 0x7f7f340cdd70 2024-04-17T22:31:36.335 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-17T22:31:36.335 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":11,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[],"filesystems":[{"mdsmap":{"epoch":11,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17 22:31:28.204771","modified":"2024-04-17 22:31:30.213118","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"-1 (unspecified)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5319},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5319":{"gid":5319,"name":"a","rank":0,"incarnation":9,"state":"up:active","state_seq":3,"addr":"172.21.15.102:6817/479367134","addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6817","nonce":479367134}]},"export_targets":[],"features":4611087854035861503,"flags":0}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2024-04-17T22:31:36.337 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.349 7f7f3bd83700 1 -- 172.21.15.102:0/3632301855 >> v1:172.21.15.102:6816/42360 conn(0x7f7f1c0145a0 legacy=0x7f7f1c016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:36.337 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.349 7f7f3bd83700 1 -- 172.21.15.102:0/3632301855 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f7f340cdd70 msgr2=0x7f7f340d5fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:36.338 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.349 7f7f3bd83700 1 --2- 172.21.15.102:0/3632301855 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f7f340cdd70 0x7f7f340d5fd0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 rx=0x7f7f340d75c0 tx=0x7f7f2800c860).stop 2024-04-17T22:31:36.338 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.349 7f7f3bd83700 1 -- 172.21.15.102:0/3632301855 shutdown_connections 2024-04-17T22:31:36.338 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.349 7f7f3bd83700 1 --2- 172.21.15.102:0/3632301855 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7f340d6a50 0x7f7f341b4600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:36.338 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.349 7f7f3bd83700 1 --2- 172.21.15.102:0/3632301855 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f7f340cdd70 0x7f7f340d5fd0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:36.339 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.349 7f7f3bd83700 1 --2- 172.21.15.102:0/3632301855 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f7f340d6600 0x7f7f341b3fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:36.339 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.349 7f7f3bd83700 1 -- 172.21.15.102:0/3632301855 >> 172.21.15.102:0/3632301855 conn(0x7f7f340c38b0 msgr2=0x7f7f340dae50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:36.339 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.349 7f7f3bd83700 1 -- 172.21.15.102:0/3632301855 shutdown_connections 2024-04-17T22:31:36.339 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.349 7f7f3bd83700 1 -- 172.21.15.102:0/3632301855 wait complete. 2024-04-17T22:31:36.339 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 11 2024-04-17T22:31:36.346 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T22:31:36.474 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.485 7fe5efc1b700 1 Processor -- start 2024-04-17T22:31:36.474 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.485 7fe5efc1b700 1 -- start start 2024-04-17T22:31:36.474 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.486 7fe5efc1b700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe5e810e1d0 0x7fe5e810e620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:36.474 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.486 7fe5efc1b700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fe5e810ec50 0x7fe5e80f16c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:36.474 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.486 7fe5efc1b700 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5e80f2140 con 0x7fe5e80f1cf0 2024-04-17T22:31:36.474 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.486 7fe5efc1b700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5e80f9ea0 con 0x7fe5e810e1d0 2024-04-17T22:31:36.474 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.486 7fe5efc1b700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe5e80f8fc0 con 0x7fe5e810ec50 2024-04-17T22:31:36.474 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.486 7fe5e7fff700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fe5e810ec50 0x7fe5e80f16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:36.474 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.486 7fe5ed217700 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7fe5e80f1cf0 0x7fe5e81aa460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:45676/0 (socket says 172.21.15.102:45676) 2024-04-17T22:31:36.474 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.486 7fe5ed217700 1 -- 172.21.15.102:0/3981437136 learned_addr learned my addr 172.21.15.102:0/3981437136 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-17T22:31:36.474 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.486 7fe5eca16700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe5e810e1d0 0x7fe5e810e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:36.474 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.486 7fe5e7fff700 1 -- 172.21.15.102:0/3981437136 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe5e810e1d0 msgr2=0x7fe5e810e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:31:36.475 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.486 7fe5e7fff700 1 --2- 172.21.15.102:0/3981437136 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe5e810e1d0 0x7fe5e810e620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:36.475 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.486 7fe5e77fe700 1 -- 172.21.15.102:0/3981437136 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3809903757 0 0) 0x7fe5e0001490 con 0x7fe5e80f1cf0 2024-04-17T22:31:36.475 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.486 7fe5e7fff700 1 -- 172.21.15.102:0/3981437136 >> v1:172.21.15.102:6789/0 conn(0x7fe5e80f1cf0 legacy=0x7fe5e81aa460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:36.475 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.486 7fe5e7fff700 1 -- 172.21.15.102:0/3981437136 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5e80f2360 con 0x7fe5e810ec50 2024-04-17T22:31:36.475 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.486 7fe5eca16700 1 --2- 172.21.15.102:0/3981437136 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe5e810e1d0 0x7fe5e810e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:36.475 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.486 7fe5e7fff700 1 --2- 172.21.15.102:0/3981437136 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fe5e810ec50 0x7fe5e80f16c0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 rx=0x7fe5d4002f70 tx=0x7fe5d400b9a0).ready entity=mon.2 client_cookie=d5f6ea91336bae86 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:36.475 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.487 7fe5e77fe700 1 -- 172.21.15.102:0/3981437136 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe5e80f2360 con 0x7fe5e810ec50 2024-04-17T22:31:36.475 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.487 7fe5e77fe700 1 -- 172.21.15.102:0/3981437136 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5d400d570 con 0x7fe5e810ec50 2024-04-17T22:31:36.475 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.487 7fe5e77fe700 1 -- 172.21.15.102:0/3981437136 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe5d400d940 con 0x7fe5e810ec50 2024-04-17T22:31:36.475 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.487 7fe5efc1b700 1 -- 172.21.15.102:0/3981437136 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fe5e810ec50 msgr2=0x7fe5e80f16c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:36.475 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.487 7fe5efc1b700 1 --2- 172.21.15.102:0/3981437136 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fe5e810ec50 0x7fe5e80f16c0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 rx=0x7fe5d4002f70 tx=0x7fe5d400b9a0).stop 2024-04-17T22:31:36.475 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.487 7fe5efc1b700 1 -- 172.21.15.102:0/3981437136 shutdown_connections 2024-04-17T22:31:36.476 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.487 7fe5efc1b700 1 --2- 172.21.15.102:0/3981437136 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe5e810e1d0 0x7fe5e810e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:36.476 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.487 7fe5efc1b700 1 --2- 172.21.15.102:0/3981437136 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fe5e810ec50 0x7fe5e80f16c0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:36.476 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.487 7fe5efc1b700 1 -- 172.21.15.102:0/3981437136 >> 172.21.15.102:0/3981437136 conn(0x7fe5e8113830 msgr2=0x7fe5e80f4ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:36.476 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.487 7fe5efc1b700 1 -- 172.21.15.102:0/3981437136 shutdown_connections 2024-04-17T22:31:36.476 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.487 7fe5efc1b700 1 -- 172.21.15.102:0/3981437136 wait complete. 2024-04-17T22:31:36.496 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.510 7fe5efc1b700 1 Processor -- start 2024-04-17T22:31:36.496 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.510 7fe5efc1b700 1 -- start start 2024-04-17T22:31:36.499 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.511 7fe5efc1b700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fe5e80d5de0 0x7fe5e80d6230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:36.499 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.511 7fe5efc1b700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe5e80d6860 0x7fe5e81b3e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:36.499 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.511 7fe5efc1b700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe5e81b44b0 0x7fe5e81b4900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:36.499 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.511 7fe5efc1b700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5e80fa140 con 0x7fe5e81b44b0 2024-04-17T22:31:36.499 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.511 7fe5efc1b700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5e80c43d0 con 0x7fe5e80d6860 2024-04-17T22:31:36.499 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.511 7fe5efc1b700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe5e80dd880 con 0x7fe5e80d5de0 2024-04-17T22:31:36.499 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.511 7fe5e7fff700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe5e80d6860 0x7fe5e81b3e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:36.499 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.511 7fe5ed217700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe5e81b44b0 0x7fe5e81b4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:36.500 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.511 7fe5e7fff700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe5e80d6860 0x7fe5e81b3e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:40172/0 (socket says 172.21.15.102:40172) 2024-04-17T22:31:36.500 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.511 7fe5eca16700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fe5e80d5de0 0x7fe5e80d6230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:36.500 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.511 7fe5e7fff700 1 -- 172.21.15.102:0/3607202077 learned_addr learned my addr 172.21.15.102:0/3607202077 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:31:36.500 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.511 7fe5ed217700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe5e81b44b0 0x7fe5e81b4900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:57428/0 (socket says 172.21.15.102:57428) 2024-04-17T22:31:36.500 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.511 7fe5eca16700 1 -- 172.21.15.102:0/3607202077 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe5e80d6860 msgr2=0x7fe5e81b3e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:36.500 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.511 7fe5eca16700 1 --2- 172.21.15.102:0/3607202077 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe5e80d6860 0x7fe5e81b3e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:36.500 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.511 7fe5eca16700 1 -- 172.21.15.102:0/3607202077 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe5e81b44b0 msgr2=0x7fe5e81b4900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:36.500 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.511 7fe5eca16700 1 --2- 172.21.15.102:0/3607202077 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe5e81b44b0 0x7fe5e81b4900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:36.500 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.511 7fe5eca16700 1 -- 172.21.15.102:0/3607202077 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5e80ddb80 con 0x7fe5e80d5de0 2024-04-17T22:31:36.500 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.511 7fe5e7fff700 1 --2- 172.21.15.102:0/3607202077 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe5e80d6860 0x7fe5e81b3e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:36.500 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.511 7fe5ed217700 1 --2- 172.21.15.102:0/3607202077 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe5e81b44b0 0x7fe5e81b4900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:36.500 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.511 7fe5eca16700 1 --2- 172.21.15.102:0/3607202077 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fe5e80d5de0 0x7fe5e80d6230 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 rx=0x7fe5e80c4f20 tx=0x7fe5dc00ec80).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:36.500 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.511 7fe5e5ffb700 1 -- 172.21.15.102:0/3607202077 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe5e80ddb80 con 0x7fe5e80d5de0 2024-04-17T22:31:36.500 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.511 7fe5e5ffb700 1 -- 172.21.15.102:0/3607202077 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5dc00c810 con 0x7fe5e80d5de0 2024-04-17T22:31:36.501 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.511 7fe5e5ffb700 1 -- 172.21.15.102:0/3607202077 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fe5dc00f040 con 0x7fe5e80d5de0 2024-04-17T22:31:36.501 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.511 7fe5efc1b700 1 -- 172.21.15.102:0/3607202077 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5e80d7d50 con 0x7fe5e80d5de0 2024-04-17T22:31:36.501 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.512 7fe5efc1b700 1 -- 172.21.15.102:0/3607202077 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5e81bb750 con 0x7fe5e80d5de0 2024-04-17T22:31:36.501 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.512 7fe5e5ffb700 1 -- 172.21.15.102:0/3607202077 <== mon.2 v2:172.21.15.102:3302/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7fe5dc011070 con 0x7fe5e80d5de0 2024-04-17T22:31:36.501 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.512 7fe5e5ffb700 1 -- 172.21.15.102:0/3607202077 <== mon.2 v2:172.21.15.102:3302/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 3194+0+0 (secure 0 0 0) 0x7fe5dc010040 con 0x7fe5e80d5de0 2024-04-17T22:31:36.501 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.512 7fe5efc1b700 1 -- 172.21.15.102:0/3607202077 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5e80f0790 con 0x7fe5e80d5de0 2024-04-17T22:31:36.501 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.516 7fe5e5ffb700 1 -- 172.21.15.102:0/3607202077 <== mon.2 v2:172.21.15.102:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+110455 (secure 0 0 0) 0x7fe5dc00edc0 con 0x7fe5e80d5de0 2024-04-17T22:31:36.607 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.621 7fe5efc1b700 1 -- 172.21.15.102:0/3607202077 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe5e80f54c0 con 0x7fe5e80d5de0 2024-04-17T22:31:36.607 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.622 7fe5e5ffb700 1 -- 172.21.15.102:0/3607202077 <== mon.2 v2:172.21.15.102:3302/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+8764 (secure 0 0 0) 0x7fe5dc000c30 con 0x7fe5e80d5de0 2024-04-17T22:31:36.608 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-17T22:31:36.608 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":25,"fsid":"cb63e789-091d-49b1-b120-55f9db526871","created":"2024-04-17 22:28:48.209635","modified":"2024-04-17 22:31:29.205644","last_up_change":"2024-04-17 22:28:54.236215","last_in_change":"2024-04-17 22:28:54.236215","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":4,"require_min_compat_client":"nautilus","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-04-17 22:28:57.478545","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-17 22:31:26.214102","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-17 22:31:27.183113","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3351ef85-ade4-4b90-a8f4-213f4e00ba01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6812","nonce":42955}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6813","nonce":42955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6815","nonce":42955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6814","nonce":42955}]},"public_addr":"172.21.15.102:6812/42955","cluster_addr":"172.21.15.102:6813/42955","heartbeat_back_addr":"172.21.15.102:6815/42955","heartbeat_front_addr":"172.21.15.102:6814/42955","state":["exists","up"]},{"osd":1,"uuid":"04f67eac-f36b-48f5-982d-4cfc776596d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6808","nonce":42958}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6809","nonce":42958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6811","nonce":42958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6810","nonce":42958}]},"public_addr":"172.21.15.102:6808/42958","cluster_addr":"172.21.15.102:6809/42958","heartbeat_back_addr":"172.21.15.102:6811/42958","heartbeat_front_addr":"172.21.15.102:6810/42958","state":["exists","up"]},{"osd":2,"uuid":"d524204b-70b2-40a8-8f73-f9e7d44318af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6804","nonce":42954}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6805","nonce":42954}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6807","nonce":42954}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6806","nonce":42954}]},"public_addr":"172.21.15.102:6804/42954","cluster_addr":"172.21.15.102:6805/42954","heartbeat_back_addr":"172.21.15.102:6807/42954","heartbeat_front_addr":"172.21.15.102:6806/42954","state":["exists","up"]},{"osd":3,"uuid":"4fc9d20c-1ad0-484f-aafa-2c9a7574245b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6800","nonce":42953}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6801","nonce":42953}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6803","nonce":42953}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6802","nonce":42953}]},"public_addr":"172.21.15.102:6800/42953","cluster_addr":"172.21.15.102:6801/42953","heartbeat_back_addr":"172.21.15.102:6803/42953","heartbeat_front_addr":"172.21.15.102:6802/42953","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2024-04-17T22:31:36.609 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.623 7fe5efc1b700 1 -- 172.21.15.102:0/3607202077 >> v1:172.21.15.102:6816/42360 conn(0x7fe5d00145a0 legacy=0x7fe5d0016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:36.609 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.623 7fe5efc1b700 1 -- 172.21.15.102:0/3607202077 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fe5e80d5de0 msgr2=0x7fe5e80d6230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:36.609 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.623 7fe5efc1b700 1 --2- 172.21.15.102:0/3607202077 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fe5e80d5de0 0x7fe5e80d6230 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 rx=0x7fe5e80c4f20 tx=0x7fe5dc00ec80).stop 2024-04-17T22:31:36.609 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.623 7fe5efc1b700 1 -- 172.21.15.102:0/3607202077 shutdown_connections 2024-04-17T22:31:36.609 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.623 7fe5efc1b700 1 --2- 172.21.15.102:0/3607202077 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe5e81b44b0 0x7fe5e81b4900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:36.610 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.623 7fe5efc1b700 1 --2- 172.21.15.102:0/3607202077 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fe5e80d5de0 0x7fe5e80d6230 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:36.610 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.623 7fe5efc1b700 1 --2- 172.21.15.102:0/3607202077 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe5e80d6860 0x7fe5e81b3e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:36.610 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.623 7fe5efc1b700 1 -- 172.21.15.102:0/3607202077 >> 172.21.15.102:0/3607202077 conn(0x7fe5e80e0f20 msgr2=0x7fe5e80ca420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:36.610 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.623 7fe5efc1b700 1 -- 172.21.15.102:0/3607202077 shutdown_connections 2024-04-17T22:31:36.610 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.623 7fe5efc1b700 1 -- 172.21.15.102:0/3607202077 wait complete. 2024-04-17T22:31:36.620 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2024-04-17 22:28:57.478545', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '12', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-17 22:31:26.214102', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-04-17 22:31:27.183113', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-17T22:31:36.620 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2024-04-17T22:31:36.750 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.762 7f6c30c43700 1 Processor -- start 2024-04-17T22:31:36.750 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.762 7f6c30c43700 1 -- start start 2024-04-17T22:31:36.750 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.762 7f6c30c43700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f6c2c0f5840 0x7f6c2c10b010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:36.751 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.762 7f6c30c43700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f6c2c10b640 0x7f6c2c19faf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:36.751 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.762 7f6c30c43700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6c2c10ba90 0x7f6c2c1a0120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:36.751 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.762 7f6c30c43700 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c2c10bee0 con 0x7f6c2c10b640 2024-04-17T22:31:36.751 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.762 7f6c30c43700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c2c0fa370 con 0x7f6c2c10ba90 2024-04-17T22:31:36.751 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.762 7f6c30c43700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6c2c103b40 con 0x7f6c2c0f5840 2024-04-17T22:31:36.751 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.762 7f6c29850700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f6c2c0f5840 0x7f6c2c10b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:36.752 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.762 7f6c2a051700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6c2c10ba90 0x7f6c2c1a0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:36.752 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.762 7f6c23fff700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f6c2c10b640 0x7f6c2c19faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:36.752 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.762 7f6c23fff700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f6c2c10b640 0x7f6c2c19faf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:57434/0 (socket says 172.21.15.102:57434) 2024-04-17T22:31:36.752 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.762 7f6c2a051700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6c2c10ba90 0x7f6c2c1a0120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:40186/0 (socket says 172.21.15.102:40186) 2024-04-17T22:31:36.752 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.762 7f6c29850700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f6c2c0f5840 0x7f6c2c10b010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:48140/0 (socket says 172.21.15.102:48140) 2024-04-17T22:31:36.752 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.762 7f6c23fff700 1 -- 172.21.15.102:0/3191943707 learned_addr learned my addr 172.21.15.102:0/3191943707 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:31:36.752 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.763 7f6c23fff700 1 -- 172.21.15.102:0/3191943707 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f6c2c0f5840 msgr2=0x7f6c2c10b010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:31:36.752 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.763 7f6c23fff700 1 --2- 172.21.15.102:0/3191943707 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f6c2c0f5840 0x7f6c2c10b010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:36.752 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.763 7f6c23fff700 1 -- 172.21.15.102:0/3191943707 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6c2c10ba90 msgr2=0x7f6c2c1a0120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:31:36.752 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.763 7f6c23fff700 1 --2- 172.21.15.102:0/3191943707 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6c2c10ba90 0x7f6c2c1a0120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:36.753 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.763 7f6c23fff700 1 -- 172.21.15.102:0/3191943707 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c2c1a0750 con 0x7f6c2c10b640 2024-04-17T22:31:36.753 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.763 7f6c2a051700 1 --2- 172.21.15.102:0/3191943707 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6c2c10ba90 0x7f6c2c1a0120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:36.753 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.763 7f6c23fff700 1 --2- 172.21.15.102:0/3191943707 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6c2c10b640 0x7f6c2c19faf0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 rx=0x7f6c2c0fa520 tx=0x7f6c14009a80).ready entity=mon.0 client_cookie=1732f107c6111dc3 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:36.753 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.763 7f6c2904f700 1 -- 172.21.15.102:0/3191943707 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f6c2c1a0750 con 0x7f6c2c10b640 2024-04-17T22:31:36.753 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.763 7f6c2904f700 1 -- 172.21.15.102:0/3191943707 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c1400f510 con 0x7f6c2c10b640 2024-04-17T22:31:36.753 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.763 7f6c2904f700 1 -- 172.21.15.102:0/3191943707 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f6c1400f6f0 con 0x7f6c2c10b640 2024-04-17T22:31:36.753 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.763 7f6c30c43700 1 -- 172.21.15.102:0/3191943707 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6c2c10b640 msgr2=0x7f6c2c19faf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:36.753 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.763 7f6c30c43700 1 --2- 172.21.15.102:0/3191943707 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6c2c10b640 0x7f6c2c19faf0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 rx=0x7f6c2c0fa520 tx=0x7f6c14009a80).stop 2024-04-17T22:31:36.753 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.763 7f6c30c43700 1 -- 172.21.15.102:0/3191943707 shutdown_connections 2024-04-17T22:31:36.753 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.763 7f6c30c43700 1 --2- 172.21.15.102:0/3191943707 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6c2c10ba90 0x7f6c2c1a0120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:36.753 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.763 7f6c30c43700 1 --2- 172.21.15.102:0/3191943707 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f6c2c0f5840 0x7f6c2c10b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:36.753 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.763 7f6c30c43700 1 --2- 172.21.15.102:0/3191943707 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6c2c10b640 0x7f6c2c19faf0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:36.754 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.763 7f6c30c43700 1 -- 172.21.15.102:0/3191943707 >> 172.21.15.102:0/3191943707 conn(0x7f6c2c1138a0 msgr2=0x7f6c2c0f4f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:36.754 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.764 7f6c30c43700 1 -- 172.21.15.102:0/3191943707 shutdown_connections 2024-04-17T22:31:36.754 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.764 7f6c30c43700 1 -- 172.21.15.102:0/3191943707 wait complete. 2024-04-17T22:31:36.774 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.786 7f6c30c43700 1 Processor -- start 2024-04-17T22:31:36.775 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.786 7f6c30c43700 1 -- start start 2024-04-17T22:31:36.775 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.786 7f6c30c43700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6c2c0c9660 0x7f6c2c1a9070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:36.775 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.786 7f6c30c43700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6c2c1a96a0 0x7f6c2c1af920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:36.775 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.786 7f6c30c43700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f6c2c1a9af0 0x7f6c2c1aff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:36.775 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.786 7f6c30c43700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c2c0e2790 con 0x7f6c2c0c9660 2024-04-17T22:31:36.775 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.786 7f6c30c43700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c2c0d23b0 con 0x7f6c2c1a96a0 2024-04-17T22:31:36.775 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.786 7f6c30c43700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6c2c0d25e0 con 0x7f6c2c1a9af0 2024-04-17T22:31:36.775 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.787 7f6c23fff700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6c2c1a96a0 0x7f6c2c1af920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:36.775 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.787 7f6c2a051700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f6c2c1a9af0 0x7f6c2c1aff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:36.775 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.787 7f6c23fff700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6c2c1a96a0 0x7f6c2c1af920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:40200/0 (socket says 172.21.15.102:40200) 2024-04-17T22:31:36.775 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.787 7f6c2a051700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f6c2c1a9af0 0x7f6c2c1aff50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:48152/0 (socket says 172.21.15.102:48152) 2024-04-17T22:31:36.775 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.787 7f6c23fff700 1 -- 172.21.15.102:0/3912852439 learned_addr learned my addr 172.21.15.102:0/3912852439 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:31:36.776 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.787 7f6c29850700 1 --2- 172.21.15.102:0/3912852439 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6c2c0c9660 0x7f6c2c1a9070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:36.776 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.787 7f6c23fff700 1 -- 172.21.15.102:0/3912852439 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f6c2c1a9af0 msgr2=0x7f6c2c1aff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:36.776 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.787 7f6c23fff700 1 --2- 172.21.15.102:0/3912852439 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f6c2c1a9af0 0x7f6c2c1aff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:36.776 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.787 7f6c23fff700 1 -- 172.21.15.102:0/3912852439 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6c2c0c9660 msgr2=0x7f6c2c1a9070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:36.776 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.787 7f6c23fff700 1 --2- 172.21.15.102:0/3912852439 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6c2c0c9660 0x7f6c2c1a9070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:36.776 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.787 7f6c23fff700 1 -- 172.21.15.102:0/3912852439 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c2c1b0580 con 0x7f6c2c1a96a0 2024-04-17T22:31:36.776 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.787 7f6c2a051700 1 --2- 172.21.15.102:0/3912852439 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f6c2c1a9af0 0x7f6c2c1aff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:36.776 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.787 7f6c29850700 1 --2- 172.21.15.102:0/3912852439 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6c2c0c9660 0x7f6c2c1a9070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:36.776 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.787 7f6c23fff700 1 --2- 172.21.15.102:0/3912852439 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6c2c1a96a0 0x7f6c2c1af920 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 rx=0x7f6c2c0d2c30 tx=0x7f6c140068c0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:36.776 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.787 7f6c22ffd700 1 -- 172.21.15.102:0/3912852439 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f6c2c1b0580 con 0x7f6c2c1a96a0 2024-04-17T22:31:36.776 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.787 7f6c22ffd700 1 -- 172.21.15.102:0/3912852439 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c1401c650 con 0x7f6c2c1a96a0 2024-04-17T22:31:36.776 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.787 7f6c30c43700 1 -- 172.21.15.102:0/3912852439 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c2c1b0840 con 0x7f6c2c1a96a0 2024-04-17T22:31:36.777 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.787 7f6c22ffd700 1 -- 172.21.15.102:0/3912852439 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f6c1401ca20 con 0x7f6c2c1a96a0 2024-04-17T22:31:36.777 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.787 7f6c30c43700 1 -- 172.21.15.102:0/3912852439 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c2c1b7270 con 0x7f6c2c1a96a0 2024-04-17T22:31:36.777 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.788 7f6c22ffd700 1 -- 172.21.15.102:0/3912852439 <== mon.1 v2:172.21.15.102:3301/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7f6c14019030 con 0x7f6c2c1a96a0 2024-04-17T22:31:36.777 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.788 7f6c22ffd700 1 -- 172.21.15.102:0/3912852439 <== mon.1 v2:172.21.15.102:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 3194+0+0 (secure 0 0 0) 0x7f6c14018040 con 0x7f6c2c1a96a0 2024-04-17T22:31:36.777 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.788 7f6c30c43700 1 -- 172.21.15.102:0/3912852439 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c2c0f0800 con 0x7f6c2c1a96a0 2024-04-17T22:31:36.777 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.791 7f6c22ffd700 1 -- 172.21.15.102:0/3912852439 <== mon.1 v2:172.21.15.102:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+110455 (secure 0 0 0) 0x7f6c1401c440 con 0x7f6c2c1a96a0 2024-04-17T22:31:36.884 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.898 7f6c30c43700 1 -- 172.21.15.102:0/3912852439 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f6c2c0abc60 con 0x7f6c2c1a96a0 2024-04-17T22:31:36.884 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.898 7f6c22ffd700 1 -- 172.21.15.102:0/3912852439 <== mon.1 v2:172.21.15.102:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v25) v1 ==== 92+0+10 (secure 0 0 0) 0x7f6c1400f300 con 0x7f6c2c1a96a0 2024-04-17T22:31:36.885 INFO:teuthology.orchestra.run.smithi102.stdout:pg_num: 8 2024-04-17T22:31:36.886 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.899 7f6c30c43700 1 -- 172.21.15.102:0/3912852439 >> v1:172.21.15.102:6816/42360 conn(0x7f6c100145a0 legacy=0x7f6c10016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:36.886 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.899 7f6c30c43700 1 -- 172.21.15.102:0/3912852439 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6c2c1a96a0 msgr2=0x7f6c2c1af920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:36.886 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.899 7f6c30c43700 1 --2- 172.21.15.102:0/3912852439 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6c2c1a96a0 0x7f6c2c1af920 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 rx=0x7f6c2c0d2c30 tx=0x7f6c140068c0).stop 2024-04-17T22:31:36.887 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.899 7f6c30c43700 1 -- 172.21.15.102:0/3912852439 shutdown_connections 2024-04-17T22:31:36.887 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.899 7f6c30c43700 1 --2- 172.21.15.102:0/3912852439 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f6c2c1a9af0 0x7f6c2c1aff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:36.887 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.899 7f6c30c43700 1 --2- 172.21.15.102:0/3912852439 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6c2c0c9660 0x7f6c2c1a9070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:36.887 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.899 7f6c30c43700 1 --2- 172.21.15.102:0/3912852439 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6c2c1a96a0 0x7f6c2c1af920 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:36.887 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.899 7f6c30c43700 1 -- 172.21.15.102:0/3912852439 >> 172.21.15.102:0/3912852439 conn(0x7f6c2c0d5ba0 msgr2=0x7f6c2c0ce8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:36.887 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.899 7f6c30c43700 1 -- 172.21.15.102:0/3912852439 shutdown_connections 2024-04-17T22:31:36.887 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:36.899 7f6c30c43700 1 -- 172.21.15.102:0/3912852439 wait complete. 2024-04-17T22:31:36.896 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-17T22:31:37.025 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.037 7f3f10fec700 1 Processor -- start 2024-04-17T22:31:37.026 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.037 7f3f10fec700 1 -- start start 2024-04-17T22:31:37.026 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.037 7f3f10fec700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f3f0c10f040 0x7f3f0c1a83c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:37.026 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.037 7f3f10fec700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f3f0c1a89f0 0x7f3f0c1aae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:37.026 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.037 7f3f10fec700 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f0c103670 con 0x7f3f0c0f5be0 2024-04-17T22:31:37.026 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.037 7f3f10fec700 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f0c110e50 con 0x7f3f0c10f040 2024-04-17T22:31:37.028 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.037 7f3f10fec700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f0c1ac490 con 0x7f3f0c1a89f0 2024-04-17T22:31:37.028 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.037 7f3f0a5a0700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f3f0c1a89f0 0x7f3f0c1aae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:37.028 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.037 7f3f03fff700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f3f0c10f040 0x7f3f0c1a83c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:37.028 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.037 7f3f0a5a0700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f3f0c1a89f0 0x7f3f0c1aae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:40204/0 (socket says 172.21.15.102:40204) 2024-04-17T22:31:37.028 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.037 7f3f0a5a0700 1 -- 172.21.15.102:0/4028616938 learned_addr learned my addr 172.21.15.102:0/4028616938 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:31:37.028 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.038 7f3f03fff700 1 -- 172.21.15.102:0/4028616938 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f3f0c1a89f0 msgr2=0x7f3f0c1aae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:31:37.028 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.038 7f3f03fff700 1 --2- 172.21.15.102:0/4028616938 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f3f0c1a89f0 0x7f3f0c1aae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:37.028 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.038 7f3f03fff700 1 -- 172.21.15.102:0/4028616938 >> v1:172.21.15.102:6789/0 conn(0x7f3f0c0f5be0 legacy=0x7f3f0c10e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:37.029 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.038 7f3f03fff700 1 -- 172.21.15.102:0/4028616938 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f0c1ac640 con 0x7f3f0c10f040 2024-04-17T22:31:37.029 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.038 7f3f0959e700 1 -- 172.21.15.102:0/4028616938 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4212953248 0 0) 0x7f3efc001110 con 0x7f3f0c0f5be0 2024-04-17T22:31:37.029 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.038 7f3f0a5a0700 1 --2- 172.21.15.102:0/4028616938 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f3f0c1a89f0 0x7f3f0c1aae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:37.029 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.038 7f3f03fff700 1 --2- 172.21.15.102:0/4028616938 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3f0c10f040 0x7f3f0c1a83c0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 rx=0x7f3f0c113420 tx=0x7f3ef400ba20).ready entity=mon.0 client_cookie=58fdc80f16bce337 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:37.029 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.038 7f3f0959e700 1 -- 172.21.15.102:0/4028616938 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f3f0c1ac640 con 0x7f3f0c10f040 2024-04-17T22:31:37.029 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.038 7f3f0959e700 1 -- 172.21.15.102:0/4028616938 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ef400d5c0 con 0x7f3f0c10f040 2024-04-17T22:31:37.029 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.038 7f3f0959e700 1 -- 172.21.15.102:0/4028616938 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f3ef400d990 con 0x7f3f0c10f040 2024-04-17T22:31:37.029 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.038 7f3f10fec700 1 -- 172.21.15.102:0/4028616938 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3f0c10f040 msgr2=0x7f3f0c1a83c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:37.029 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.038 7f3f10fec700 1 --2- 172.21.15.102:0/4028616938 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3f0c10f040 0x7f3f0c1a83c0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 rx=0x7f3f0c113420 tx=0x7f3ef400ba20).stop 2024-04-17T22:31:37.029 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.039 7f3f10fec700 1 -- 172.21.15.102:0/4028616938 shutdown_connections 2024-04-17T22:31:37.029 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.039 7f3f10fec700 1 --2- 172.21.15.102:0/4028616938 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f3f0c1a89f0 0x7f3f0c1aae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:37.030 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.039 7f3f10fec700 1 --2- 172.21.15.102:0/4028616938 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3f0c10f040 0x7f3f0c1a83c0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:37.030 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.039 7f3f10fec700 1 -- 172.21.15.102:0/4028616938 >> 172.21.15.102:0/4028616938 conn(0x7f3f0c113a90 msgr2=0x7f3f0c0f5140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:37.030 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.039 7f3f10fec700 1 -- 172.21.15.102:0/4028616938 shutdown_connections 2024-04-17T22:31:37.030 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.039 7f3f10fec700 1 -- 172.21.15.102:0/4028616938 wait complete. 2024-04-17T22:31:37.050 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.061 7f3f10fec700 1 Processor -- start 2024-04-17T22:31:37.050 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.061 7f3f10fec700 1 -- start start 2024-04-17T22:31:37.050 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.061 7f3f10fec700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f3f0c0c50c0 0x7f3f0c0ce3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:37.050 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.061 7f3f10fec700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3f0c0ce9d0 0x7f3f0c0d2ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:37.050 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.061 7f3f10fec700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f3f0c0cee20 0x7f3f0c0d3100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:37.050 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.061 7f3f10fec700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f0c0d4e90 con 0x7f3f0c0ce9d0 2024-04-17T22:31:37.050 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.061 7f3f10fec700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f0c0d3730 con 0x7f3f0c0c50c0 2024-04-17T22:31:37.050 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.061 7f3f10fec700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3f0c0d3960 con 0x7f3f0c0cee20 2024-04-17T22:31:37.050 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.062 7f3f09d9f700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f3f0c0c50c0 0x7f3f0c0ce3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:37.050 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.062 7f3f0a5a0700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f3f0c0cee20 0x7f3f0c0d3100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:37.051 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.062 7f3f03fff700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3f0c0ce9d0 0x7f3f0c0d2ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:37.051 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.062 7f3f09d9f700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f3f0c0c50c0 0x7f3f0c0ce3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:40218/0 (socket says 172.21.15.102:40218) 2024-04-17T22:31:37.051 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.062 7f3f0a5a0700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f3f0c0cee20 0x7f3f0c0d3100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:48154/0 (socket says 172.21.15.102:48154) 2024-04-17T22:31:37.051 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.062 7f3f03fff700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3f0c0ce9d0 0x7f3f0c0d2ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:57456/0 (socket says 172.21.15.102:57456) 2024-04-17T22:31:37.051 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.062 7f3f09d9f700 1 -- 172.21.15.102:0/367310402 learned_addr learned my addr 172.21.15.102:0/367310402 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:31:37.051 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.062 7f3f03fff700 1 -- 172.21.15.102:0/367310402 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f3f0c0cee20 msgr2=0x7f3f0c0d3100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:37.051 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.062 7f3f03fff700 1 --2- 172.21.15.102:0/367310402 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f3f0c0cee20 0x7f3f0c0d3100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:37.051 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.062 7f3f03fff700 1 -- 172.21.15.102:0/367310402 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f3f0c0c50c0 msgr2=0x7f3f0c0ce3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:37.051 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.062 7f3f03fff700 1 --2- 172.21.15.102:0/367310402 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f3f0c0c50c0 0x7f3f0c0ce3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:37.051 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.062 7f3f03fff700 1 -- 172.21.15.102:0/367310402 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f0c1bb5e0 con 0x7f3f0c0ce9d0 2024-04-17T22:31:37.052 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.062 7f3f09d9f700 1 --2- 172.21.15.102:0/367310402 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f3f0c0c50c0 0x7f3f0c0ce3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:37.052 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.062 7f3f0a5a0700 1 --2- 172.21.15.102:0/367310402 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f3f0c0cee20 0x7f3f0c0d3100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:37.052 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.062 7f3f03fff700 1 --2- 172.21.15.102:0/367310402 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3f0c0ce9d0 0x7f3f0c0d2ad0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 rx=0x7f3f0c0dc550 tx=0x7f3ef40068c0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:37.052 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.062 7f3f02ffd700 1 -- 172.21.15.102:0/367310402 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f3f0c1bb5e0 con 0x7f3f0c0ce9d0 2024-04-17T22:31:37.052 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.062 7f3f02ffd700 1 -- 172.21.15.102:0/367310402 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ef4011980 con 0x7f3f0c0ce9d0 2024-04-17T22:31:37.052 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.062 7f3f02ffd700 1 -- 172.21.15.102:0/367310402 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f3ef4011b60 con 0x7f3f0c0ce9d0 2024-04-17T22:31:37.052 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.062 7f3f10fec700 1 -- 172.21.15.102:0/367310402 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f0c1bb7e0 con 0x7f3f0c0ce9d0 2024-04-17T22:31:37.052 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.062 7f3f10fec700 1 -- 172.21.15.102:0/367310402 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f0c1bbbc0 con 0x7f3f0c0ce9d0 2024-04-17T22:31:37.052 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.063 7f3f02ffd700 1 -- 172.21.15.102:0/367310402 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7f3ef4018370 con 0x7f3f0c0ce9d0 2024-04-17T22:31:37.052 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.063 7f3f02ffd700 1 -- 172.21.15.102:0/367310402 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 3194+0+0 (secure 0 0 0) 0x7f3ef400adc0 con 0x7f3f0c0ce9d0 2024-04-17T22:31:37.052 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.063 7f3f10fec700 1 -- 172.21.15.102:0/367310402 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f0c0f09f0 con 0x7f3f0c0ce9d0 2024-04-17T22:31:37.053 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.067 7f3f02ffd700 1 -- 172.21.15.102:0/367310402 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+110455 (secure 0 0 0) 0x7f3ef4011770 con 0x7f3f0c0ce9d0 2024-04-17T22:31:37.156 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.170 7f3f10fec700 1 -- 172.21.15.102:0/367310402 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f3f0c1ae660 con 0x7f3f0c0ce9d0 2024-04-17T22:31:37.157 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.171 7f3f02ffd700 1 -- 172.21.15.102:0/367310402 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+11 (secure 0 0 0) 0x7f3ef400d3b0 con 0x7f3f0c0ce9d0 2024-04-17T22:31:37.157 INFO:teuthology.orchestra.run.smithi102.stdout:pg_num: 64 2024-04-17T22:31:37.158 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.172 7f3f10fec700 1 -- 172.21.15.102:0/367310402 >> v1:172.21.15.102:6816/42360 conn(0x7f3ef00145a0 legacy=0x7f3ef0016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:37.158 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.172 7f3f10fec700 1 -- 172.21.15.102:0/367310402 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3f0c0ce9d0 msgr2=0x7f3f0c0d2ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:37.158 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.172 7f3f10fec700 1 --2- 172.21.15.102:0/367310402 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3f0c0ce9d0 0x7f3f0c0d2ad0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 rx=0x7f3f0c0dc550 tx=0x7f3ef40068c0).stop 2024-04-17T22:31:37.158 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.172 7f3f10fec700 1 -- 172.21.15.102:0/367310402 shutdown_connections 2024-04-17T22:31:37.159 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.172 7f3f10fec700 1 --2- 172.21.15.102:0/367310402 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f3f0c0cee20 0x7f3f0c0d3100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:37.159 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.172 7f3f10fec700 1 --2- 172.21.15.102:0/367310402 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f3f0c0c50c0 0x7f3f0c0ce3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:37.159 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.172 7f3f10fec700 1 --2- 172.21.15.102:0/367310402 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3f0c0ce9d0 0x7f3f0c0d2ad0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:37.159 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.172 7f3f10fec700 1 -- 172.21.15.102:0/367310402 >> 172.21.15.102:0/367310402 conn(0x7f3f0c0c5a00 msgr2=0x7f3f0c0bd930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:37.159 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.172 7f3f10fec700 1 -- 172.21.15.102:0/367310402 shutdown_connections 2024-04-17T22:31:37.159 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.172 7f3f10fec700 1 -- 172.21.15.102:0/367310402 wait complete. 2024-04-17T22:31:37.169 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-17T22:31:37.297 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.309 7f9e06766700 1 Processor -- start 2024-04-17T22:31:37.297 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.309 7f9e06766700 1 -- start start 2024-04-17T22:31:37.297 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.309 7f9e06766700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f9e0010e790 0x7f9e0010ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:37.297 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.309 7f9e06766700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f9e001038e0 0x7f9e001a83f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:37.297 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.309 7f9e06766700 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e000ad030 con 0x7f9e0010f210 2024-04-17T22:31:37.297 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.309 7f9e06766700 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e00104630 con 0x7f9e001038e0 2024-04-17T22:31:37.297 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.309 7f9e06766700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e000f2ae0 con 0x7f9e0010e790 2024-04-17T22:31:37.298 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.309 7f9dffbb1700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f9e001038e0 0x7f9e001a83f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:37.298 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.309 7f9dff3b0700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f9e0010e790 0x7f9e0010ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:37.298 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.309 7f9dffbb1700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f9e001038e0 0x7f9e001a83f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:57468/0 (socket says 172.21.15.102:57468) 2024-04-17T22:31:37.298 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.309 7f9dfebaf700 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f9e0010f210 0x7f9e001030e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:45694/0 (socket says 172.21.15.102:45694) 2024-04-17T22:31:37.298 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.309 7f9dffbb1700 1 -- 172.21.15.102:0/2429680487 learned_addr learned my addr 172.21.15.102:0/2429680487 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:31:37.298 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.310 7f9dff3b0700 1 -- 172.21.15.102:0/2429680487 >> v2:172.21.15.102:3300/0 conn(0x7f9e001038e0 msgr2=0x7f9e001a83f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:31:37.298 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.310 7f9dff3b0700 1 --2- 172.21.15.102:0/2429680487 >> v2:172.21.15.102:3300/0 conn(0x7f9e001038e0 0x7f9e001a83f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:37.298 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.310 7f9dff3b0700 1 -- 172.21.15.102:0/2429680487 >> v1:172.21.15.102:6789/0 conn(0x7f9e0010f210 legacy=0x7f9e001030e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:37.298 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.310 7f9dff3b0700 1 -- 172.21.15.102:0/2429680487 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e00103d30 con 0x7f9e0010e790 2024-04-17T22:31:37.298 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.310 7f9dffbb1700 1 --2- 172.21.15.102:0/2429680487 >> v2:172.21.15.102:3300/0 conn(0x7f9e001038e0 0x7f9e001a83f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:37.298 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.310 7f9dff3b0700 1 --2- 172.21.15.102:0/2429680487 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f9e0010e790 0x7f9e0010ebe0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 rx=0x7f9df0002f70 tx=0x7f9df000b970).ready entity=mon.1 client_cookie=f40a25f7bc61f67 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:37.298 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.310 7f9dfe3ae700 1 -- 172.21.15.102:0/2429680487 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9e00103d30 con 0x7f9e0010e790 2024-04-17T22:31:37.298 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.310 7f9dfe3ae700 1 -- 172.21.15.102:0/2429680487 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9df000d5c0 con 0x7f9e0010e790 2024-04-17T22:31:37.299 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.310 7f9dfe3ae700 1 -- 172.21.15.102:0/2429680487 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9df000d990 con 0x7f9e0010e790 2024-04-17T22:31:37.299 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.310 7f9e06766700 1 -- 172.21.15.102:0/2429680487 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f9e0010e790 msgr2=0x7f9e0010ebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:37.299 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.310 7f9e06766700 1 --2- 172.21.15.102:0/2429680487 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f9e0010e790 0x7f9e0010ebe0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 rx=0x7f9df0002f70 tx=0x7f9df000b970).stop 2024-04-17T22:31:37.299 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.311 7f9e06766700 1 -- 172.21.15.102:0/2429680487 shutdown_connections 2024-04-17T22:31:37.299 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.311 7f9e06766700 1 --2- 172.21.15.102:0/2429680487 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f9e0010e790 0x7f9e0010ebe0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:37.299 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.311 7f9e06766700 1 --2- 172.21.15.102:0/2429680487 >> v2:172.21.15.102:3300/0 conn(0x7f9e001038e0 0x7f9e001a83f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:37.299 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.311 7f9e06766700 1 -- 172.21.15.102:0/2429680487 >> 172.21.15.102:0/2429680487 conn(0x7f9e00113000 msgr2=0x7f9e00113450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:37.299 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.311 7f9e06766700 1 -- 172.21.15.102:0/2429680487 shutdown_connections 2024-04-17T22:31:37.299 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.311 7f9e06766700 1 -- 172.21.15.102:0/2429680487 wait complete. 2024-04-17T22:31:37.321 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.333 7f9e06766700 1 Processor -- start 2024-04-17T22:31:37.322 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.334 7f9e06766700 1 -- start start 2024-04-17T22:31:37.322 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.334 7f9e06766700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f9e000cdda0 0x7f9e000ce330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:37.322 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.334 7f9e06766700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9e000ce960 0x7f9e001b1ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:37.322 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.334 7f9e06766700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f9e000cedb0 0x7f9e001b22d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:37.322 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.334 7f9e06766700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e000cd910 con 0x7f9e000ce960 2024-04-17T22:31:37.322 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.334 7f9e06766700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e000d3fd0 con 0x7f9e000cdda0 2024-04-17T22:31:37.322 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.334 7f9e06766700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9e000cd610 con 0x7f9e000cedb0 2024-04-17T22:31:37.322 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.334 7f9dff3b0700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f9e000cdda0 0x7f9e000ce330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:37.322 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.334 7f9dfebaf700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9e000ce960 0x7f9e001b1ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:37.322 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.334 7f9dff3b0700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f9e000cdda0 0x7f9e000ce330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:40236/0 (socket says 172.21.15.102:40236) 2024-04-17T22:31:37.322 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.334 7f9dff3b0700 1 -- 172.21.15.102:0/4077989201 learned_addr learned my addr 172.21.15.102:0/4077989201 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:31:37.322 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.334 7f9dffbb1700 1 --2- 172.21.15.102:0/4077989201 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f9e000cedb0 0x7f9e001b22d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:37.323 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.334 7f9dff3b0700 1 -- 172.21.15.102:0/4077989201 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f9e000cedb0 msgr2=0x7f9e001b22d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:37.323 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.334 7f9dff3b0700 1 --2- 172.21.15.102:0/4077989201 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f9e000cedb0 0x7f9e001b22d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:37.323 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.334 7f9dff3b0700 1 -- 172.21.15.102:0/4077989201 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9e000ce960 msgr2=0x7f9e001b1ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:37.323 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.334 7f9dff3b0700 1 --2- 172.21.15.102:0/4077989201 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9e000ce960 0x7f9e001b1ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:37.323 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.334 7f9dff3b0700 1 -- 172.21.15.102:0/4077989201 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e001b2900 con 0x7f9e000cdda0 2024-04-17T22:31:37.323 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.334 7f9dff3b0700 1 --2- 172.21.15.102:0/4077989201 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f9e000cdda0 0x7f9e000ce330 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 rx=0x7f9e000bcf60 tx=0x7f9df00068c0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:37.323 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.335 7f9dfcbab700 1 -- 172.21.15.102:0/4077989201 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9e001b2900 con 0x7f9e000cdda0 2024-04-17T22:31:37.323 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.335 7f9dfcbab700 1 -- 172.21.15.102:0/4077989201 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9df00119c0 con 0x7f9e000cdda0 2024-04-17T22:31:37.323 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.335 7f9e06766700 1 -- 172.21.15.102:0/4077989201 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e001b9100 con 0x7f9e000cdda0 2024-04-17T22:31:37.323 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.335 7f9dfcbab700 1 -- 172.21.15.102:0/4077989201 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9df0017040 con 0x7f9e000cdda0 2024-04-17T22:31:37.323 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.335 7f9e06766700 1 -- 172.21.15.102:0/4077989201 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e001b94e0 con 0x7f9e000cdda0 2024-04-17T22:31:37.323 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.335 7f9dfcbab700 1 -- 172.21.15.102:0/4077989201 <== mon.1 v2:172.21.15.102:3301/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7f9df00273b0 con 0x7f9e000cdda0 2024-04-17T22:31:37.323 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.335 7f9dfcbab700 1 -- 172.21.15.102:0/4077989201 <== mon.1 v2:172.21.15.102:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 3194+0+0 (secure 0 0 0) 0x7f9df0018070 con 0x7f9e000cdda0 2024-04-17T22:31:37.323 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.335 7f9e06766700 1 -- 172.21.15.102:0/4077989201 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e000f09f0 con 0x7f9e000cdda0 2024-04-17T22:31:37.324 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.339 7f9dfcbab700 1 -- 172.21.15.102:0/4077989201 <== mon.1 v2:172.21.15.102:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+110455 (secure 0 0 0) 0x7f9df00117b0 con 0x7f9e000cdda0 2024-04-17T22:31:37.429 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.442 7f9e06766700 1 -- 172.21.15.102:0/4077989201 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f9e000ccd80 con 0x7f9e000cdda0 2024-04-17T22:31:37.429 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.443 7f9dfcbab700 1 -- 172.21.15.102:0/4077989201 <== mon.1 v2:172.21.15.102:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+11 (secure 0 0 0) 0x7f9df000d3b0 con 0x7f9e000cdda0 2024-04-17T22:31:37.429 INFO:teuthology.orchestra.run.smithi102.stdout:pg_num: 64 2024-04-17T22:31:37.431 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.444 7f9e06766700 1 -- 172.21.15.102:0/4077989201 >> v1:172.21.15.102:6816/42360 conn(0x7f9de00145a0 legacy=0x7f9de0016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:37.431 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.444 7f9e06766700 1 -- 172.21.15.102:0/4077989201 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f9e000cdda0 msgr2=0x7f9e000ce330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:37.431 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.444 7f9e06766700 1 --2- 172.21.15.102:0/4077989201 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f9e000cdda0 0x7f9e000ce330 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 rx=0x7f9e000bcf60 tx=0x7f9df00068c0).stop 2024-04-17T22:31:37.431 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.444 7f9e06766700 1 -- 172.21.15.102:0/4077989201 shutdown_connections 2024-04-17T22:31:37.431 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.444 7f9e06766700 1 --2- 172.21.15.102:0/4077989201 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f9e000cedb0 0x7f9e001b22d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:37.431 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.444 7f9e06766700 1 --2- 172.21.15.102:0/4077989201 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f9e000cdda0 0x7f9e000ce330 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:37.431 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.444 7f9e06766700 1 --2- 172.21.15.102:0/4077989201 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9e000ce960 0x7f9e001b1ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:37.431 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.444 7f9e06766700 1 -- 172.21.15.102:0/4077989201 >> 172.21.15.102:0/4077989201 conn(0x7f9e000da810 msgr2=0x7f9e000e0d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:37.431 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.444 7f9e06766700 1 -- 172.21.15.102:0/4077989201 shutdown_connections 2024-04-17T22:31:37.431 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.444 7f9e06766700 1 -- 172.21.15.102:0/4077989201 wait complete. 2024-04-17T22:31:37.441 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T22:31:37.572 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.582 7fb1625d3700 1 Processor -- start 2024-04-17T22:31:37.573 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.582 7fb1625d3700 1 -- start start 2024-04-17T22:31:37.573 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.582 7fb1625d3700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb15c0f5770 0x7fb15c10c110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:37.573 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.582 7fb1625d3700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fb15c10c740 0x7fb15c1a0c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:37.573 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.582 7fb1625d3700 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb15c1a1a30 con 0x7fb15c10cb90 2024-04-17T22:31:37.573 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.582 7fb1625d3700 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb15c0f93e0 con 0x7fb15c10c740 2024-04-17T22:31:37.573 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.582 7fb1625d3700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb15c1a1c50 con 0x7fb15c0f5770 2024-04-17T22:31:37.574 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.582 7fb15abaf700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fb15c10c740 0x7fb15c1a0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:37.574 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.582 7fb15b3b0700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb15c0f5770 0x7fb15c10c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:37.574 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.582 7fb15bbb1700 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7fb15c10cb90 0x7fb15c1a1230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:45696/0 (socket says 172.21.15.102:45696) 2024-04-17T22:31:37.574 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.582 7fb15bbb1700 1 -- 172.21.15.102:0/321943380 learned_addr learned my addr 172.21.15.102:0/321943380 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-17T22:31:37.574 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.582 7fb15abaf700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fb15c10c740 0x7fb15c1a0c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:57486/0 (socket says 172.21.15.102:57486) 2024-04-17T22:31:37.574 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.582 7fb15b3b0700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb15c0f5770 0x7fb15c10c110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:40246/0 (socket says 172.21.15.102:40246) 2024-04-17T22:31:37.575 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.582 7fb15b3b0700 1 -- 172.21.15.102:0/321943380 >> v2:172.21.15.102:3300/0 conn(0x7fb15c10c740 msgr2=0x7fb15c1a0c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:31:37.575 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.582 7fb15b3b0700 1 --2- 172.21.15.102:0/321943380 >> v2:172.21.15.102:3300/0 conn(0x7fb15c10c740 0x7fb15c1a0c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:37.575 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.582 7fb15b3b0700 1 -- 172.21.15.102:0/321943380 >> v1:172.21.15.102:6789/0 conn(0x7fb15c10cb90 legacy=0x7fb15c1a1230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:37.575 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.583 7fb15a3ae700 1 -- 172.21.15.102:0/321943380 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1740608038 0 0) 0x7fb14c001110 con 0x7fb15c10cb90 2024-04-17T22:31:37.575 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.583 7fb15b3b0700 1 -- 172.21.15.102:0/321943380 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb15c1aa330 con 0x7fb15c0f5770 2024-04-17T22:31:37.575 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.583 7fb15abaf700 1 --2- 172.21.15.102:0/321943380 >> v2:172.21.15.102:3300/0 conn(0x7fb15c10c740 0x7fb15c1a0c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:37.576 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.583 7fb15b3b0700 1 --2- 172.21.15.102:0/321943380 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb15c0f5770 0x7fb15c10c110 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 rx=0x7fb1540032f0 tx=0x7fb154009c90).ready entity=mon.1 client_cookie=56d7042aa14a5375 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:37.576 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.583 7fb15a3ae700 1 -- 172.21.15.102:0/321943380 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb15c1aa330 con 0x7fb15c0f5770 2024-04-17T22:31:37.576 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.583 7fb15a3ae700 1 -- 172.21.15.102:0/321943380 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb15400b820 con 0x7fb15c0f5770 2024-04-17T22:31:37.576 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.583 7fb15a3ae700 1 -- 172.21.15.102:0/321943380 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb15400ba00 con 0x7fb15c0f5770 2024-04-17T22:31:37.576 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.583 7fb1625d3700 1 -- 172.21.15.102:0/321943380 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb15c0f5770 msgr2=0x7fb15c10c110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:37.576 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.583 7fb1625d3700 1 --2- 172.21.15.102:0/321943380 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb15c0f5770 0x7fb15c10c110 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 rx=0x7fb1540032f0 tx=0x7fb154009c90).stop 2024-04-17T22:31:37.577 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.583 7fb1625d3700 1 -- 172.21.15.102:0/321943380 shutdown_connections 2024-04-17T22:31:37.577 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.583 7fb1625d3700 1 --2- 172.21.15.102:0/321943380 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb15c0f5770 0x7fb15c10c110 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:37.577 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.583 7fb1625d3700 1 --2- 172.21.15.102:0/321943380 >> v2:172.21.15.102:3300/0 conn(0x7fb15c10c740 0x7fb15c1a0c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:37.577 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.583 7fb1625d3700 1 -- 172.21.15.102:0/321943380 >> 172.21.15.102:0/321943380 conn(0x7fb15c1138c0 msgr2=0x7fb15c0f4f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:37.578 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.583 7fb1625d3700 1 -- 172.21.15.102:0/321943380 shutdown_connections 2024-04-17T22:31:37.578 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.583 7fb1625d3700 1 -- 172.21.15.102:0/321943380 wait complete. 2024-04-17T22:31:37.592 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.606 7fb1625d3700 1 Processor -- start 2024-04-17T22:31:37.592 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.606 7fb1625d3700 1 -- start start 2024-04-17T22:31:37.595 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.607 7fb1625d3700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb15c0dce80 0x7fb15c1a6090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:37.595 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.607 7fb1625d3700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb15c1a66c0 0x7fb15c0cb910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:37.595 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.607 7fb1625d3700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb15c1a6b10 0x7fb15c0cbf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:37.595 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.607 7fb1625d3700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb15c0d5970 con 0x7fb15c1a66c0 2024-04-17T22:31:37.595 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.607 7fb1625d3700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb15c0cc570 con 0x7fb15c1a6b10 2024-04-17T22:31:37.595 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.607 7fb1625d3700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb15c0cc7a0 con 0x7fb15c0dce80 2024-04-17T22:31:37.595 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.607 7fb15b3b0700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb15c0dce80 0x7fb15c1a6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:37.595 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.607 7fb15bbb1700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb15c1a6b10 0x7fb15c0cbf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:37.595 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.607 7fb15abaf700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb15c1a66c0 0x7fb15c0cb910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:37.595 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.607 7fb15b3b0700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb15c0dce80 0x7fb15c1a6090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:48180/0 (socket says 172.21.15.102:48180) 2024-04-17T22:31:37.595 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.607 7fb15b3b0700 1 -- 172.21.15.102:0/1278679208 learned_addr learned my addr 172.21.15.102:0/1278679208 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:31:37.595 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.607 7fb15b3b0700 1 -- 172.21.15.102:0/1278679208 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb15c1a6b10 msgr2=0x7fb15c0cbf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:37.596 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.607 7fb15b3b0700 1 --2- 172.21.15.102:0/1278679208 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb15c1a6b10 0x7fb15c0cbf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:37.596 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.607 7fb15b3b0700 1 -- 172.21.15.102:0/1278679208 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb15c1a66c0 msgr2=0x7fb15c0cb910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:37.596 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.607 7fb15b3b0700 1 --2- 172.21.15.102:0/1278679208 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb15c1a66c0 0x7fb15c0cb910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:37.596 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.607 7fb15b3b0700 1 -- 172.21.15.102:0/1278679208 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb15c1b92f0 con 0x7fb15c0dce80 2024-04-17T22:31:37.596 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.607 7fb15abaf700 1 --2- 172.21.15.102:0/1278679208 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb15c1a66c0 0x7fb15c0cb910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:37.596 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.608 7fb15b3b0700 1 --2- 172.21.15.102:0/1278679208 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb15c0dce80 0x7fb15c1a6090 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 rx=0x7fb1540032f0 tx=0x7fb154006c40).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:37.596 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.608 7fb158bab700 1 -- 172.21.15.102:0/1278679208 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb15c1b92f0 con 0x7fb15c0dce80 2024-04-17T22:31:37.596 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.608 7fb158bab700 1 -- 172.21.15.102:0/1278679208 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb15401f020 con 0x7fb15c0dce80 2024-04-17T22:31:37.596 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.608 7fb1625d3700 1 -- 172.21.15.102:0/1278679208 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb15c1b94f0 con 0x7fb15c0dce80 2024-04-17T22:31:37.596 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.608 7fb158bab700 1 -- 172.21.15.102:0/1278679208 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb15401b430 con 0x7fb15c0dce80 2024-04-17T22:31:37.596 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.608 7fb1625d3700 1 -- 172.21.15.102:0/1278679208 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb15c1b98d0 con 0x7fb15c0dce80 2024-04-17T22:31:37.596 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.608 7fb158bab700 1 -- 172.21.15.102:0/1278679208 <== mon.2 v2:172.21.15.102:3302/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7fb1540099b0 con 0x7fb15c0dce80 2024-04-17T22:31:37.596 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.608 7fb158bab700 1 -- 172.21.15.102:0/1278679208 <== mon.2 v2:172.21.15.102:3302/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 3194+0+0 (secure 0 0 0) 0x7fb1540146d0 con 0x7fb15c0dce80 2024-04-17T22:31:37.596 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.609 7fb1625d3700 1 -- 172.21.15.102:0/1278679208 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb15c0f0820 con 0x7fb15c0dce80 2024-04-17T22:31:37.598 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.612 7fb158bab700 1 -- 172.21.15.102:0/1278679208 <== mon.2 v2:172.21.15.102:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+110455 (secure 0 0 0) 0x7fb154011b60 con 0x7fb15c0dce80 2024-04-17T22:31:37.732 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.746 7fb1625d3700 1 -- 172.21.15.102:0/1278679208 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb15c0efef0 con 0x7fb15c0dce80 2024-04-17T22:31:37.733 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.747 7fb158bab700 1 -- 172.21.15.102:0/1278679208 <== mon.2 v2:172.21.15.102:3302/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) v1 ==== 94+0+1704 (secure 0 0 0) 0x7fb15400b610 con 0x7fb15c0dce80 2024-04-17T22:31:37.733 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-17T22:31:37.733 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":11,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[],"filesystems":[{"mdsmap":{"epoch":11,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17 22:31:28.204771","modified":"2024-04-17 22:31:30.213118","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"-1 (unspecified)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5319},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5319":{"gid":5319,"name":"a","rank":0,"incarnation":9,"state":"up:active","state_seq":3,"addr":"172.21.15.102:6817/479367134","addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6817","nonce":479367134}]},"export_targets":[],"features":4611087854035861503,"flags":0}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2024-04-17T22:31:37.734 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.748 7fb1625d3700 1 -- 172.21.15.102:0/1278679208 >> v1:172.21.15.102:6816/42360 conn(0x7fb13c0145a0 legacy=0x7fb13c016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:37.734 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.748 7fb1625d3700 1 -- 172.21.15.102:0/1278679208 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb15c0dce80 msgr2=0x7fb15c1a6090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:37.734 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.748 7fb1625d3700 1 --2- 172.21.15.102:0/1278679208 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb15c0dce80 0x7fb15c1a6090 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 rx=0x7fb1540032f0 tx=0x7fb154006c40).stop 2024-04-17T22:31:37.734 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.748 7fb1625d3700 1 -- 172.21.15.102:0/1278679208 shutdown_connections 2024-04-17T22:31:37.735 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.748 7fb1625d3700 1 --2- 172.21.15.102:0/1278679208 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb15c1a6b10 0x7fb15c0cbf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:37.735 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.748 7fb1625d3700 1 --2- 172.21.15.102:0/1278679208 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb15c0dce80 0x7fb15c1a6090 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:37.735 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.748 7fb1625d3700 1 --2- 172.21.15.102:0/1278679208 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb15c1a66c0 0x7fb15c0cb910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:37.735 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.748 7fb1625d3700 1 -- 172.21.15.102:0/1278679208 >> 172.21.15.102:0/1278679208 conn(0x7fb15c0c9cf0 msgr2=0x7fb15c0ceb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:37.735 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.748 7fb1625d3700 1 -- 172.21.15.102:0/1278679208 shutdown_connections 2024-04-17T22:31:37.735 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.748 7fb1625d3700 1 -- 172.21.15.102:0/1278679208 wait complete. 2024-04-17T22:31:37.735 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 11 2024-04-17T22:31:37.745 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T22:31:37.871 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.883 7f92061b2700 1 Processor -- start 2024-04-17T22:31:37.871 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.883 7f92061b2700 1 -- start start 2024-04-17T22:31:37.871 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.883 7f92061b2700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f92000f5860 0x7f920010b030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:37.871 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.883 7f92061b2700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f920010b660 0x7f920019fb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:37.871 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.883 7f92061b2700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f920010bab0 0x7f92001a0140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:37.871 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.883 7f92061b2700 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f920010bf00 con 0x7f92000f5860 2024-04-17T22:31:37.871 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.883 7f92061b2700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92000fa390 con 0x7f920010b660 2024-04-17T22:31:37.872 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.883 7f92061b2700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9200103b60 con 0x7f920010bab0 2024-04-17T22:31:37.872 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.883 7f91fedc6700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f92000f5860 0x7f920010b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:37.872 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.883 7f91ff5c7700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f920010bab0 0x7f92001a0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:37.872 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.883 7f91fe5c5700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f920010b660 0x7f920019fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:37.872 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.883 7f91fedc6700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f92000f5860 0x7f920010b030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:57500/0 (socket says 172.21.15.102:57500) 2024-04-17T22:31:37.872 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.883 7f91fedc6700 1 -- 172.21.15.102:0/3253766309 learned_addr learned my addr 172.21.15.102:0/3253766309 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:31:37.872 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.884 7f91ff5c7700 1 -- 172.21.15.102:0/3253766309 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f920010b660 msgr2=0x7f920019fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:31:37.872 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.884 7f91ff5c7700 1 --2- 172.21.15.102:0/3253766309 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f920010b660 0x7f920019fb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:37.872 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.884 7f91ff5c7700 1 -- 172.21.15.102:0/3253766309 >> v2:172.21.15.102:3300/0 conn(0x7f92000f5860 msgr2=0x7f920010b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:31:37.872 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.884 7f91ff5c7700 1 --2- 172.21.15.102:0/3253766309 >> v2:172.21.15.102:3300/0 conn(0x7f92000f5860 0x7f920010b030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:37.872 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.884 7f91ff5c7700 1 -- 172.21.15.102:0/3253766309 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92001a0770 con 0x7f920010bab0 2024-04-17T22:31:37.872 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.884 7f91fe5c5700 1 --2- 172.21.15.102:0/3253766309 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f920010b660 0x7f920019fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:37.872 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.884 7f91ff5c7700 1 --2- 172.21.15.102:0/3253766309 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f920010bab0 0x7f92001a0140 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 rx=0x7f92000f46c0 tx=0x7f91f800d2f0).ready entity=mon.2 client_cookie=5048a169df993f97 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:37.872 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.884 7f91fddc4700 1 -- 172.21.15.102:0/3253766309 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f92001a0770 con 0x7f920010bab0 2024-04-17T22:31:37.872 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.884 7f91fddc4700 1 -- 172.21.15.102:0/3253766309 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91f800de10 con 0x7f920010bab0 2024-04-17T22:31:37.873 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.884 7f91fddc4700 1 -- 172.21.15.102:0/3253766309 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f91f8010660 con 0x7f920010bab0 2024-04-17T22:31:37.873 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.884 7f92061b2700 1 -- 172.21.15.102:0/3253766309 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f920010bab0 msgr2=0x7f92001a0140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:37.873 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.884 7f92061b2700 1 --2- 172.21.15.102:0/3253766309 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f920010bab0 0x7f92001a0140 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 rx=0x7f92000f46c0 tx=0x7f91f800d2f0).stop 2024-04-17T22:31:37.873 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.884 7f92061b2700 1 -- 172.21.15.102:0/3253766309 shutdown_connections 2024-04-17T22:31:37.873 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.884 7f92061b2700 1 --2- 172.21.15.102:0/3253766309 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f920010bab0 0x7f92001a0140 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:37.873 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.884 7f92061b2700 1 --2- 172.21.15.102:0/3253766309 >> v2:172.21.15.102:3300/0 conn(0x7f92000f5860 0x7f920010b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:37.873 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.884 7f92061b2700 1 --2- 172.21.15.102:0/3253766309 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f920010b660 0x7f920019fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:37.873 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.884 7f92061b2700 1 -- 172.21.15.102:0/3253766309 >> 172.21.15.102:0/3253766309 conn(0x7f92001138c0 msgr2=0x7f92000f4f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:37.873 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.885 7f92061b2700 1 -- 172.21.15.102:0/3253766309 shutdown_connections 2024-04-17T22:31:37.873 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.885 7f92061b2700 1 -- 172.21.15.102:0/3253766309 wait complete. 2024-04-17T22:31:37.893 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.907 7f92061b2700 1 Processor -- start 2024-04-17T22:31:37.893 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.907 7f92061b2700 1 -- start start 2024-04-17T22:31:37.895 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.907 7f92061b2700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f92000d50f0 0x7f92001ad060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:37.896 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.907 7f92061b2700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f92001ad690 0x7f92001b39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:37.896 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.908 7f92061b2700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f92001adae0 0x7f92001b4000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:37.896 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.908 7f92061b2700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92000d4770 con 0x7f92001adae0 2024-04-17T22:31:37.896 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.908 7f92061b2700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92000d4350 con 0x7f92000d50f0 2024-04-17T22:31:37.896 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.908 7f92061b2700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f92001b4630 con 0x7f92001ad690 2024-04-17T22:31:37.896 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.908 7f91fedc6700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f92000d50f0 0x7f92001ad060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:37.896 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.908 7f91fe5c5700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f92001ad690 0x7f92001b39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:37.896 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.908 7f91fedc6700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f92000d50f0 0x7f92001ad060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:40280/0 (socket says 172.21.15.102:40280) 2024-04-17T22:31:37.896 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.908 7f91fe5c5700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f92001ad690 0x7f92001b39d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:48202/0 (socket says 172.21.15.102:48202) 2024-04-17T22:31:37.896 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.908 7f91fedc6700 1 -- 172.21.15.102:0/1986871649 learned_addr learned my addr 172.21.15.102:0/1986871649 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:31:37.896 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.908 7f91ff5c7700 1 --2- 172.21.15.102:0/1986871649 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f92001adae0 0x7f92001b4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:37.896 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.908 7f91fedc6700 1 -- 172.21.15.102:0/1986871649 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f92001ad690 msgr2=0x7f92001b39d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:37.897 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.908 7f91fedc6700 1 --2- 172.21.15.102:0/1986871649 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f92001ad690 0x7f92001b39d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:37.897 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.908 7f91fedc6700 1 -- 172.21.15.102:0/1986871649 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f92001adae0 msgr2=0x7f92001b4000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:37.897 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.908 7f91fedc6700 1 --2- 172.21.15.102:0/1986871649 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f92001adae0 0x7f92001b4000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:37.897 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.908 7f91fedc6700 1 -- 172.21.15.102:0/1986871649 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92001b4860 con 0x7f92000d50f0 2024-04-17T22:31:37.897 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.908 7f91fe5c5700 1 --2- 172.21.15.102:0/1986871649 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f92001ad690 0x7f92001b39d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:37.897 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.908 7f91ff5c7700 1 --2- 172.21.15.102:0/1986871649 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f92001adae0 0x7f92001b4000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:37.897 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.908 7f91fedc6700 1 --2- 172.21.15.102:0/1986871649 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f92000d50f0 0x7f92001ad060 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 rx=0x7f91f0006b50 tx=0x7f91f000eca0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:37.897 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.908 7f91f7fff700 1 -- 172.21.15.102:0/1986871649 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f92001b4860 con 0x7f92000d50f0 2024-04-17T22:31:37.897 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.908 7f91f7fff700 1 -- 172.21.15.102:0/1986871649 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91f000c7b0 con 0x7f92000d50f0 2024-04-17T22:31:37.897 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.908 7f91f7fff700 1 -- 172.21.15.102:0/1986871649 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f91f000f040 con 0x7f92000d50f0 2024-04-17T22:31:37.897 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.908 7f92061b2700 1 -- 172.21.15.102:0/1986871649 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92001bb150 con 0x7f92000d50f0 2024-04-17T22:31:37.897 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.908 7f92061b2700 1 -- 172.21.15.102:0/1986871649 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92001bb530 con 0x7f92000d50f0 2024-04-17T22:31:37.897 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.909 7f91f7fff700 1 -- 172.21.15.102:0/1986871649 <== mon.1 v2:172.21.15.102:3301/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7f91f0022060 con 0x7f92000d50f0 2024-04-17T22:31:37.898 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.909 7f91f7fff700 1 -- 172.21.15.102:0/1986871649 <== mon.1 v2:172.21.15.102:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 3194+0+0 (secure 0 0 0) 0x7f91f0010040 con 0x7f92000d50f0 2024-04-17T22:31:37.898 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.909 7f92061b2700 1 -- 172.21.15.102:0/1986871649 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92000f0820 con 0x7f92000d50f0 2024-04-17T22:31:37.898 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:37.912 7f91f7fff700 1 -- 172.21.15.102:0/1986871649 <== mon.1 v2:172.21.15.102:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+110455 (secure 0 0 0) 0x7f91f000c5a0 con 0x7f92000d50f0 2024-04-17T22:31:38.004 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.018 7f92061b2700 1 -- 172.21.15.102:0/1986871649 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f92001ae0f0 con 0x7f92000d50f0 2024-04-17T22:31:38.005 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.019 7f91f7fff700 1 -- 172.21.15.102:0/1986871649 <== mon.1 v2:172.21.15.102:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+8764 (secure 0 0 0) 0x7f91f0000c30 con 0x7f92000d50f0 2024-04-17T22:31:38.005 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-17T22:31:38.005 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":25,"fsid":"cb63e789-091d-49b1-b120-55f9db526871","created":"2024-04-17 22:28:48.209635","modified":"2024-04-17 22:31:29.205644","last_up_change":"2024-04-17 22:28:54.236215","last_in_change":"2024-04-17 22:28:54.236215","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":4,"require_min_compat_client":"nautilus","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-04-17 22:28:57.478545","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-17 22:31:26.214102","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-17 22:31:27.183113","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3351ef85-ade4-4b90-a8f4-213f4e00ba01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6812","nonce":42955}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6813","nonce":42955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6815","nonce":42955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6814","nonce":42955}]},"public_addr":"172.21.15.102:6812/42955","cluster_addr":"172.21.15.102:6813/42955","heartbeat_back_addr":"172.21.15.102:6815/42955","heartbeat_front_addr":"172.21.15.102:6814/42955","state":["exists","up"]},{"osd":1,"uuid":"04f67eac-f36b-48f5-982d-4cfc776596d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6808","nonce":42958}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6809","nonce":42958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6811","nonce":42958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6810","nonce":42958}]},"public_addr":"172.21.15.102:6808/42958","cluster_addr":"172.21.15.102:6809/42958","heartbeat_back_addr":"172.21.15.102:6811/42958","heartbeat_front_addr":"172.21.15.102:6810/42958","state":["exists","up"]},{"osd":2,"uuid":"d524204b-70b2-40a8-8f73-f9e7d44318af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6804","nonce":42954}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6805","nonce":42954}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6807","nonce":42954}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6806","nonce":42954}]},"public_addr":"172.21.15.102:6804/42954","cluster_addr":"172.21.15.102:6805/42954","heartbeat_back_addr":"172.21.15.102:6807/42954","heartbeat_front_addr":"172.21.15.102:6806/42954","state":["exists","up"]},{"osd":3,"uuid":"4fc9d20c-1ad0-484f-aafa-2c9a7574245b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6800","nonce":42953}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6801","nonce":42953}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6803","nonce":42953}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6802","nonce":42953}]},"public_addr":"172.21.15.102:6800/42953","cluster_addr":"172.21.15.102:6801/42953","heartbeat_back_addr":"172.21.15.102:6803/42953","heartbeat_front_addr":"172.21.15.102:6802/42953","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2024-04-17T22:31:38.007 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.020 7f92061b2700 1 -- 172.21.15.102:0/1986871649 >> v1:172.21.15.102:6816/42360 conn(0x7f91e40145a0 legacy=0x7f91e4016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:38.007 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.020 7f92061b2700 1 -- 172.21.15.102:0/1986871649 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f92000d50f0 msgr2=0x7f92001ad060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:38.007 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.020 7f92061b2700 1 --2- 172.21.15.102:0/1986871649 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f92000d50f0 0x7f92001ad060 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 rx=0x7f91f0006b50 tx=0x7f91f000eca0).stop 2024-04-17T22:31:38.007 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.020 7f92061b2700 1 -- 172.21.15.102:0/1986871649 shutdown_connections 2024-04-17T22:31:38.007 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.020 7f92061b2700 1 --2- 172.21.15.102:0/1986871649 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f92001adae0 0x7f92001b4000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:38.007 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.020 7f92061b2700 1 --2- 172.21.15.102:0/1986871649 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f92000d50f0 0x7f92001ad060 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:38.007 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.020 7f92061b2700 1 --2- 172.21.15.102:0/1986871649 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f92001ad690 0x7f92001b39d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:38.007 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.020 7f92061b2700 1 -- 172.21.15.102:0/1986871649 >> 172.21.15.102:0/1986871649 conn(0x7f92000c3e10 msgr2=0x7f92000cd520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:38.007 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.020 7f92061b2700 1 -- 172.21.15.102:0/1986871649 shutdown_connections 2024-04-17T22:31:38.007 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.020 7f92061b2700 1 -- 172.21.15.102:0/1986871649 wait complete. 2024-04-17T22:31:38.017 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-17T22:31:38.017 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T22:31:38.144 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.156 7f0a95736700 1 Processor -- start 2024-04-17T22:31:38.145 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.156 7f0a95736700 1 -- start start 2024-04-17T22:31:38.145 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.157 7f0a95736700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f0a9010e1d0 0x7f0a9010e620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:38.145 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.157 7f0a95736700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f0a9010ec50 0x7f0a900f16c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:38.145 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.157 7f0a95736700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f0a900f1cf0 0x7f0a901aa460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:38.145 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.157 7f0a95736700 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a900f9ea0 con 0x7f0a9010ec50 2024-04-17T22:31:38.145 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.157 7f0a95736700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a900f8fc0 con 0x7f0a900f1cf0 2024-04-17T22:31:38.145 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.157 7f0a95736700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0a900fa140 con 0x7f0a9010e1d0 2024-04-17T22:31:38.145 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.157 7f0a8eced700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f0a900f1cf0 0x7f0a901aa460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:38.145 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.157 7f0a8e4ec700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f0a9010e1d0 0x7f0a9010e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:38.145 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.157 7f0a8dceb700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f0a9010ec50 0x7f0a900f16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:38.146 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.157 7f0a8eced700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f0a900f1cf0 0x7f0a901aa460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:40284/0 (socket says 172.21.15.102:40284) 2024-04-17T22:31:38.146 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.157 7f0a8eced700 1 -- 172.21.15.102:0/1353331750 learned_addr learned my addr 172.21.15.102:0/1353331750 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:31:38.146 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.157 7f0a8eced700 1 -- 172.21.15.102:0/1353331750 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f0a9010e1d0 msgr2=0x7f0a9010e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:31:38.146 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.157 7f0a8eced700 1 --2- 172.21.15.102:0/1353331750 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f0a9010e1d0 0x7f0a9010e620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:38.146 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.157 7f0a8eced700 1 -- 172.21.15.102:0/1353331750 >> v2:172.21.15.102:3300/0 conn(0x7f0a9010ec50 msgr2=0x7f0a900f16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:31:38.146 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.157 7f0a8eced700 1 --2- 172.21.15.102:0/1353331750 >> v2:172.21.15.102:3300/0 conn(0x7f0a9010ec50 0x7f0a900f16c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:38.146 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.157 7f0a8eced700 1 -- 172.21.15.102:0/1353331750 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a900f2140 con 0x7f0a900f1cf0 2024-04-17T22:31:38.146 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.157 7f0a8e4ec700 1 --2- 172.21.15.102:0/1353331750 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f0a9010e1d0 0x7f0a9010e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:38.146 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.157 7f0a8dceb700 1 --2- 172.21.15.102:0/1353331750 >> v2:172.21.15.102:3300/0 conn(0x7f0a9010ec50 0x7f0a900f16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:38.146 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.157 7f0a8eced700 1 --2- 172.21.15.102:0/1353331750 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f0a900f1cf0 0x7f0a901aa460 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 rx=0x7f0a900f4400 tx=0x7f0a88009e40).ready entity=mon.1 client_cookie=ef0bad78932895ab server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:38.146 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.157 7f0a8d4ea700 1 -- 172.21.15.102:0/1353331750 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f0a900f2140 con 0x7f0a900f1cf0 2024-04-17T22:31:38.146 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.157 7f0a8d4ea700 1 -- 172.21.15.102:0/1353331750 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a8800b830 con 0x7f0a900f1cf0 2024-04-17T22:31:38.147 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.157 7f0a8d4ea700 1 -- 172.21.15.102:0/1353331750 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f0a8800ba10 con 0x7f0a900f1cf0 2024-04-17T22:31:38.147 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.157 7f0a95736700 1 -- 172.21.15.102:0/1353331750 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f0a900f1cf0 msgr2=0x7f0a901aa460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:38.147 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.157 7f0a95736700 1 --2- 172.21.15.102:0/1353331750 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f0a900f1cf0 0x7f0a901aa460 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 rx=0x7f0a900f4400 tx=0x7f0a88009e40).stop 2024-04-17T22:31:38.147 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.158 7f0a95736700 1 -- 172.21.15.102:0/1353331750 shutdown_connections 2024-04-17T22:31:38.147 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.158 7f0a95736700 1 --2- 172.21.15.102:0/1353331750 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f0a900f1cf0 0x7f0a901aa460 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:38.147 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.158 7f0a95736700 1 --2- 172.21.15.102:0/1353331750 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f0a9010e1d0 0x7f0a9010e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:38.147 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.158 7f0a95736700 1 --2- 172.21.15.102:0/1353331750 >> v2:172.21.15.102:3300/0 conn(0x7f0a9010ec50 0x7f0a900f16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:38.147 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.158 7f0a95736700 1 -- 172.21.15.102:0/1353331750 >> 172.21.15.102:0/1353331750 conn(0x7f0a90113830 msgr2=0x7f0a900f4ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:38.147 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.158 7f0a95736700 1 -- 172.21.15.102:0/1353331750 shutdown_connections 2024-04-17T22:31:38.147 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.158 7f0a95736700 1 -- 172.21.15.102:0/1353331750 wait complete. 2024-04-17T22:31:38.169 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.181 7f0a95736700 1 Processor -- start 2024-04-17T22:31:38.169 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.181 7f0a95736700 1 -- start start 2024-04-17T22:31:38.169 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.181 7f0a95736700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f0a901ad230 0x7f0a901ad680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:38.170 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.181 7f0a95736700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f0a901adcb0 0x7f0a901b3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:38.170 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.181 7f0a95736700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0a901b42f0 0x7f0a901b4740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:38.170 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.181 7f0a95736700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a900cd0b0 con 0x7f0a901b42f0 2024-04-17T22:31:38.170 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.181 7f0a95736700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a900c4140 con 0x7f0a901ad230 2024-04-17T22:31:38.170 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.181 7f0a95736700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0a901b4d70 con 0x7f0a901adcb0 2024-04-17T22:31:38.170 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.181 7f0a8e4ec700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f0a901ad230 0x7f0a901ad680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:38.170 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.181 7f0a8eced700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0a901b42f0 0x7f0a901b4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:38.170 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.182 7f0a8e4ec700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f0a901ad230 0x7f0a901ad680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:40286/0 (socket says 172.21.15.102:40286) 2024-04-17T22:31:38.170 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.182 7f0a8e4ec700 1 -- 172.21.15.102:0/1420427990 learned_addr learned my addr 172.21.15.102:0/1420427990 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:31:38.170 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.182 7f0a8dceb700 1 --2- 172.21.15.102:0/1420427990 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f0a901adcb0 0x7f0a901b3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:38.170 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.182 7f0a8e4ec700 1 -- 172.21.15.102:0/1420427990 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f0a901adcb0 msgr2=0x7f0a901b3cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:38.171 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.182 7f0a8e4ec700 1 --2- 172.21.15.102:0/1420427990 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f0a901adcb0 0x7f0a901b3cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:38.171 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.182 7f0a8e4ec700 1 -- 172.21.15.102:0/1420427990 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0a901b42f0 msgr2=0x7f0a901b4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:38.171 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.182 7f0a8e4ec700 1 --2- 172.21.15.102:0/1420427990 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0a901b42f0 0x7f0a901b4740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:38.171 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.182 7f0a8e4ec700 1 -- 172.21.15.102:0/1420427990 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a900cd2f0 con 0x7f0a901ad230 2024-04-17T22:31:38.171 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.182 7f0a8dceb700 1 --2- 172.21.15.102:0/1420427990 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f0a901adcb0 0x7f0a901b3cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:38.171 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.182 7f0a8eced700 1 --2- 172.21.15.102:0/1420427990 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0a901b42f0 0x7f0a901b4740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:38.171 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.182 7f0a8e4ec700 1 --2- 172.21.15.102:0/1420427990 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f0a901ad230 0x7f0a901ad680 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 rx=0x7f0a900d6e00 tx=0x7f0a8000bf70).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:38.171 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.182 7f0a7f7fe700 1 -- 172.21.15.102:0/1420427990 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f0a900cd2f0 con 0x7f0a901ad230 2024-04-17T22:31:38.171 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.182 7f0a7f7fe700 1 -- 172.21.15.102:0/1420427990 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a8000c9a0 con 0x7f0a901ad230 2024-04-17T22:31:38.171 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.182 7f0a7f7fe700 1 -- 172.21.15.102:0/1420427990 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f0a8000f040 con 0x7f0a901ad230 2024-04-17T22:31:38.171 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.182 7f0a95736700 1 -- 172.21.15.102:0/1420427990 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a900cf430 con 0x7f0a901ad230 2024-04-17T22:31:38.171 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.182 7f0a95736700 1 -- 172.21.15.102:0/1420427990 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a901bb590 con 0x7f0a901ad230 2024-04-17T22:31:38.172 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.183 7f0a7f7fe700 1 -- 172.21.15.102:0/1420427990 <== mon.1 v2:172.21.15.102:3301/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7f0a80011070 con 0x7f0a901ad230 2024-04-17T22:31:38.172 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.183 7f0a7f7fe700 1 -- 172.21.15.102:0/1420427990 <== mon.1 v2:172.21.15.102:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 3194+0+0 (secure 0 0 0) 0x7f0a80010040 con 0x7f0a901ad230 2024-04-17T22:31:38.172 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.183 7f0a95736700 1 -- 172.21.15.102:0/1420427990 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a900f0790 con 0x7f0a901ad230 2024-04-17T22:31:38.172 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.186 7f0a7f7fe700 1 -- 172.21.15.102:0/1420427990 <== mon.1 v2:172.21.15.102:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+110455 (secure 0 0 0) 0x7f0a8000c5a0 con 0x7f0a901ad230 2024-04-17T22:31:38.307 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.321 7f0a95736700 1 -- 172.21.15.102:0/1420427990 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0a900f54c0 con 0x7f0a901ad230 2024-04-17T22:31:38.307 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.321 7f0a7f7fe700 1 -- 172.21.15.102:0/1420427990 <== mon.1 v2:172.21.15.102:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) v1 ==== 94+0+1704 (secure 0 0 0) 0x7f0a80000c30 con 0x7f0a901ad230 2024-04-17T22:31:38.308 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-17T22:31:38.308 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":11,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[],"filesystems":[{"mdsmap":{"epoch":11,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17 22:31:28.204771","modified":"2024-04-17 22:31:30.213118","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"-1 (unspecified)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5319},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5319":{"gid":5319,"name":"a","rank":0,"incarnation":9,"state":"up:active","state_seq":3,"addr":"172.21.15.102:6817/479367134","addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6817","nonce":479367134}]},"export_targets":[],"features":4611087854035861503,"flags":0}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2024-04-17T22:31:38.309 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.323 7f0a95736700 1 -- 172.21.15.102:0/1420427990 >> v1:172.21.15.102:6816/42360 conn(0x7f0a6c0145a0 legacy=0x7f0a6c016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:38.309 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.323 7f0a95736700 1 -- 172.21.15.102:0/1420427990 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f0a901ad230 msgr2=0x7f0a901ad680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:38.309 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.323 7f0a95736700 1 --2- 172.21.15.102:0/1420427990 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f0a901ad230 0x7f0a901ad680 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 rx=0x7f0a900d6e00 tx=0x7f0a8000bf70).stop 2024-04-17T22:31:38.309 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.323 7f0a95736700 1 -- 172.21.15.102:0/1420427990 shutdown_connections 2024-04-17T22:31:38.309 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.323 7f0a95736700 1 --2- 172.21.15.102:0/1420427990 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0a901b42f0 0x7f0a901b4740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:38.310 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.323 7f0a95736700 1 --2- 172.21.15.102:0/1420427990 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f0a901ad230 0x7f0a901ad680 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:38.310 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.323 7f0a95736700 1 --2- 172.21.15.102:0/1420427990 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f0a901adcb0 0x7f0a901b3cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:38.310 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.323 7f0a95736700 1 -- 172.21.15.102:0/1420427990 >> 172.21.15.102:0/1420427990 conn(0x7f0a900e0dc0 msgr2=0x7f0a900daae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:38.310 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.323 7f0a95736700 1 -- 172.21.15.102:0/1420427990 shutdown_connections 2024-04-17T22:31:38.310 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.323 7f0a95736700 1 -- 172.21.15.102:0/1420427990 wait complete. 2024-04-17T22:31:38.310 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 11 2024-04-17T22:31:38.320 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T22:31:38.447 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.458 7f14019d2700 1 Processor -- start 2024-04-17T22:31:38.447 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.458 7f14019d2700 1 -- start start 2024-04-17T22:31:38.447 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.459 7f14019d2700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f13fc0f5770 0x7f13fc10c110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:38.447 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.459 7f14019d2700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f13fc10c740 0x7f13fc1a0c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:38.447 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.459 7f14019d2700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f13fc10cb90 0x7f13fc1a1230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:38.447 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.459 7f14019d2700 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13fc0f93e0 con 0x7f13fc10c740 2024-04-17T22:31:38.447 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.459 7f14019d2700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13fc1a1860 con 0x7f13fc10cb90 2024-04-17T22:31:38.447 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.459 7f14019d2700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f13fc1a1a90 con 0x7f13fc0f5770 2024-04-17T22:31:38.447 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.459 7f13fa5c5700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f13fc0f5770 0x7f13fc10c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:38.447 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.459 7f13f9dc4700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f13fc10c740 0x7f13fc1a0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:38.448 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.459 7f13fadc6700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f13fc10cb90 0x7f13fc1a1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:38.448 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.459 7f13fa5c5700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f13fc0f5770 0x7f13fc10c110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:48242/0 (socket says 172.21.15.102:48242) 2024-04-17T22:31:38.448 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.459 7f13fa5c5700 1 -- 172.21.15.102:0/4086621784 learned_addr learned my addr 172.21.15.102:0/4086621784 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:31:38.448 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.459 7f13f9dc4700 1 -- 172.21.15.102:0/4086621784 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f13fc0f5770 msgr2=0x7f13fc10c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:31:38.448 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.459 7f13f9dc4700 1 --2- 172.21.15.102:0/4086621784 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f13fc0f5770 0x7f13fc10c110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:38.448 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.459 7f13f9dc4700 1 -- 172.21.15.102:0/4086621784 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f13fc10cb90 msgr2=0x7f13fc1a1230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:31:38.448 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.459 7f13f9dc4700 1 --2- 172.21.15.102:0/4086621784 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f13fc10cb90 0x7f13fc1a1230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:38.448 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.459 7f13f9dc4700 1 -- 172.21.15.102:0/4086621784 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13fc1a8310 con 0x7f13fc10c740 2024-04-17T22:31:38.448 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.459 7f13fa5c5700 1 --2- 172.21.15.102:0/4086621784 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f13fc0f5770 0x7f13fc10c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:38.448 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.459 7f13fadc6700 1 --2- 172.21.15.102:0/4086621784 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f13fc10cb90 0x7f13fc1a1230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:38.448 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.459 7f13f9dc4700 1 --2- 172.21.15.102:0/4086621784 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f13fc10c740 0x7f13fc1a0c00 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 rx=0x7f13ec002f70 tx=0x7f13ec00b9f0).ready entity=mon.0 client_cookie=7ebe1ddb6c3dbe33 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:38.448 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.459 7f13f95c3700 1 -- 172.21.15.102:0/4086621784 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f13fc1a8310 con 0x7f13fc10c740 2024-04-17T22:31:38.448 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.459 7f13f95c3700 1 -- 172.21.15.102:0/4086621784 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13ec00d570 con 0x7f13fc10c740 2024-04-17T22:31:38.448 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.460 7f13f95c3700 1 -- 172.21.15.102:0/4086621784 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f13ec00d940 con 0x7f13fc10c740 2024-04-17T22:31:38.448 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.460 7f14019d2700 1 -- 172.21.15.102:0/4086621784 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f13fc10c740 msgr2=0x7f13fc1a0c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:38.449 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.460 7f14019d2700 1 --2- 172.21.15.102:0/4086621784 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f13fc10c740 0x7f13fc1a0c00 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 rx=0x7f13ec002f70 tx=0x7f13ec00b9f0).stop 2024-04-17T22:31:38.449 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.460 7f14019d2700 1 -- 172.21.15.102:0/4086621784 shutdown_connections 2024-04-17T22:31:38.449 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.460 7f14019d2700 1 --2- 172.21.15.102:0/4086621784 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f13fc10cb90 0x7f13fc1a1230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:38.449 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.460 7f14019d2700 1 --2- 172.21.15.102:0/4086621784 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f13fc0f5770 0x7f13fc10c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:38.449 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.460 7f14019d2700 1 --2- 172.21.15.102:0/4086621784 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f13fc10c740 0x7f13fc1a0c00 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:38.449 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.460 7f14019d2700 1 -- 172.21.15.102:0/4086621784 >> 172.21.15.102:0/4086621784 conn(0x7f13fc1138c0 msgr2=0x7f13fc0f4f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:38.449 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.460 7f14019d2700 1 -- 172.21.15.102:0/4086621784 shutdown_connections 2024-04-17T22:31:38.449 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.460 7f14019d2700 1 -- 172.21.15.102:0/4086621784 wait complete. 2024-04-17T22:31:38.471 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.483 7f14019d2700 1 Processor -- start 2024-04-17T22:31:38.471 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.483 7f14019d2700 1 -- start start 2024-04-17T22:31:38.471 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.483 7f14019d2700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f13fc0cdc40 0x7f13fc0d5f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:38.471 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.483 7f14019d2700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f13fc0d6550 0x7f13fc1b1d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:38.471 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.483 7f14019d2700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f13fc0d69a0 0x7f13fc1b2380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:38.471 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.483 7f14019d2700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13fc0cd140 con 0x7f13fc0d69a0 2024-04-17T22:31:38.471 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.483 7f14019d2700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13fc0cd320 con 0x7f13fc0cdc40 2024-04-17T22:31:38.472 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.483 7f14019d2700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f13fc1b29b0 con 0x7f13fc0d6550 2024-04-17T22:31:38.472 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.483 7f13fadc6700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f13fc0d69a0 0x7f13fc1b2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:38.472 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.483 7f13fa5c5700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f13fc0cdc40 0x7f13fc0d5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:38.472 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.483 7f13fadc6700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f13fc0d69a0 0x7f13fc1b2380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:57544/0 (socket says 172.21.15.102:57544) 2024-04-17T22:31:38.472 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.483 7f13fadc6700 1 -- 172.21.15.102:0/3696458493 learned_addr learned my addr 172.21.15.102:0/3696458493 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:31:38.472 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.483 7f13f9dc4700 1 --2- 172.21.15.102:0/3696458493 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f13fc0d6550 0x7f13fc1b1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:38.472 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.484 7f13fa5c5700 1 -- 172.21.15.102:0/3696458493 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f13fc0d6550 msgr2=0x7f13fc1b1d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:38.472 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.484 7f13fa5c5700 1 --2- 172.21.15.102:0/3696458493 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f13fc0d6550 0x7f13fc1b1d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:38.472 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.484 7f13fa5c5700 1 -- 172.21.15.102:0/3696458493 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f13fc0d69a0 msgr2=0x7f13fc1b2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:38.472 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.484 7f13fa5c5700 1 --2- 172.21.15.102:0/3696458493 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f13fc0d69a0 0x7f13fc1b2380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:38.472 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.484 7f13fa5c5700 1 -- 172.21.15.102:0/3696458493 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13fc1b2be0 con 0x7f13fc0cdc40 2024-04-17T22:31:38.472 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.484 7f13fadc6700 1 --2- 172.21.15.102:0/3696458493 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f13fc0d69a0 0x7f13fc1b2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:38.472 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.484 7f13fa5c5700 1 --2- 172.21.15.102:0/3696458493 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f13fc0cdc40 0x7f13fc0d5f20 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 rx=0x7f13f000c000 tx=0x7f13f000eca0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:38.472 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.484 7f13eb7fe700 1 -- 172.21.15.102:0/3696458493 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f13fc1b2be0 con 0x7f13fc0cdc40 2024-04-17T22:31:38.473 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.484 7f13eb7fe700 1 -- 172.21.15.102:0/3696458493 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13f000c5a0 con 0x7f13fc0cdc40 2024-04-17T22:31:38.473 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.484 7f14019d2700 1 -- 172.21.15.102:0/3696458493 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13fc1b94c0 con 0x7f13fc0cdc40 2024-04-17T22:31:38.473 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.484 7f13eb7fe700 1 -- 172.21.15.102:0/3696458493 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f13f000f040 con 0x7f13fc0cdc40 2024-04-17T22:31:38.473 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.484 7f14019d2700 1 -- 172.21.15.102:0/3696458493 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13fc1b98a0 con 0x7f13fc0cdc40 2024-04-17T22:31:38.473 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.484 7f13eb7fe700 1 -- 172.21.15.102:0/3696458493 <== mon.1 v2:172.21.15.102:3301/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7f13f0011070 con 0x7f13fc0cdc40 2024-04-17T22:31:38.473 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.484 7f13eb7fe700 1 -- 172.21.15.102:0/3696458493 <== mon.1 v2:172.21.15.102:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 3194+0+0 (secure 0 0 0) 0x7f13f0010040 con 0x7f13fc0cdc40 2024-04-17T22:31:38.473 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.485 7f14019d2700 1 -- 172.21.15.102:0/3696458493 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13fc0f0820 con 0x7f13fc0cdc40 2024-04-17T22:31:38.474 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.488 7f13eb7fe700 1 -- 172.21.15.102:0/3696458493 <== mon.1 v2:172.21.15.102:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+110455 (secure 0 0 0) 0x7f13f000c970 con 0x7f13fc0cdc40 2024-04-17T22:31:38.613 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.627 7f14019d2700 1 -- 172.21.15.102:0/3696458493 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f13fc1ac490 con 0x7f13fc0cdc40 2024-04-17T22:31:38.613 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.627 7f13eb7fe700 1 -- 172.21.15.102:0/3696458493 <== mon.1 v2:172.21.15.102:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) v1 ==== 94+0+1704 (secure 0 0 0) 0x7f13f0000c30 con 0x7f13fc0cdc40 2024-04-17T22:31:38.614 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-17T22:31:38.614 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":11,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[],"filesystems":[{"mdsmap":{"epoch":11,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17 22:31:28.204771","modified":"2024-04-17 22:31:30.213118","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"-1 (unspecified)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5319},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5319":{"gid":5319,"name":"a","rank":0,"incarnation":9,"state":"up:active","state_seq":3,"addr":"172.21.15.102:6817/479367134","addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6817","nonce":479367134}]},"export_targets":[],"features":4611087854035861503,"flags":0}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2024-04-17T22:31:38.615 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.628 7f14019d2700 1 -- 172.21.15.102:0/3696458493 >> v1:172.21.15.102:6816/42360 conn(0x7f13d80145a0 legacy=0x7f13d8016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:38.615 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.628 7f14019d2700 1 -- 172.21.15.102:0/3696458493 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f13fc0cdc40 msgr2=0x7f13fc0d5f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:38.615 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.628 7f14019d2700 1 --2- 172.21.15.102:0/3696458493 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f13fc0cdc40 0x7f13fc0d5f20 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 rx=0x7f13f000c000 tx=0x7f13f000eca0).stop 2024-04-17T22:31:38.615 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.629 7f14019d2700 1 -- 172.21.15.102:0/3696458493 shutdown_connections 2024-04-17T22:31:38.615 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.629 7f14019d2700 1 --2- 172.21.15.102:0/3696458493 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f13fc0d69a0 0x7f13fc1b2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:38.615 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.629 7f14019d2700 1 --2- 172.21.15.102:0/3696458493 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f13fc0cdc40 0x7f13fc0d5f20 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:38.616 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.629 7f14019d2700 1 --2- 172.21.15.102:0/3696458493 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f13fc0d6550 0x7f13fc1b1d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:38.616 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.629 7f14019d2700 1 -- 172.21.15.102:0/3696458493 >> 172.21.15.102:0/3696458493 conn(0x7f13fc0c3800 msgr2=0x7f13fc0dada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:38.616 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.629 7f14019d2700 1 -- 172.21.15.102:0/3696458493 shutdown_connections 2024-04-17T22:31:38.616 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.629 7f14019d2700 1 -- 172.21.15.102:0/3696458493 wait complete. 2024-04-17T22:31:38.616 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 11 2024-04-17T22:31:38.626 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T22:31:38.753 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.764 7f900383d700 1 Processor -- start 2024-04-17T22:31:38.753 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.765 7f900383d700 1 -- start start 2024-04-17T22:31:38.753 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.765 7f900383d700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f8ffc0f5770 0x7f8ffc10c110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:38.753 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.765 7f900383d700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f8ffc10c740 0x7f8ffc1a0c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:38.753 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.765 7f900383d700 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ffc1a1a30 con 0x7f8ffc10cb90 2024-04-17T22:31:38.753 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.765 7f900383d700 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ffc0f93e0 con 0x7f8ffc10c740 2024-04-17T22:31:38.753 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.765 7f900383d700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8ffc1a1c50 con 0x7f8ffc0f5770 2024-04-17T22:31:38.753 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.765 7f8ff3fff700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f8ffc10c740 0x7f8ffc1a0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:38.753 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.765 7f8ffbfff700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f8ffc0f5770 0x7f8ffc10c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:38.753 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.765 7f8ffbfff700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f8ffc0f5770 0x7f8ffc10c110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:48268/0 (socket says 172.21.15.102:48268) 2024-04-17T22:31:38.754 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.765 7f8ff3fff700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f8ffc10c740 0x7f8ffc1a0c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:57560/0 (socket says 172.21.15.102:57560) 2024-04-17T22:31:38.754 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.765 7f9000e39700 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f8ffc10cb90 0x7f8ffc1a1230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:45706/0 (socket says 172.21.15.102:45706) 2024-04-17T22:31:38.754 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.765 7f8ffbfff700 1 -- 172.21.15.102:0/1728123691 learned_addr learned my addr 172.21.15.102:0/1728123691 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:31:38.754 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.765 7f8ffbfff700 1 -- 172.21.15.102:0/1728123691 >> v2:172.21.15.102:3300/0 conn(0x7f8ffc10c740 msgr2=0x7f8ffc1a0c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:31:38.754 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.765 7f8ffbfff700 1 --2- 172.21.15.102:0/1728123691 >> v2:172.21.15.102:3300/0 conn(0x7f8ffc10c740 0x7f8ffc1a0c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:38.754 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.765 7f8ffbfff700 1 -- 172.21.15.102:0/1728123691 >> v1:172.21.15.102:6789/0 conn(0x7f8ffc10cb90 legacy=0x7f8ffc1a1230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:38.754 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.765 7f8ffbfff700 1 -- 172.21.15.102:0/1728123691 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ffc1aa330 con 0x7f8ffc0f5770 2024-04-17T22:31:38.754 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.765 7f8ffb7fe700 1 -- 172.21.15.102:0/1728123691 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4250208594 0 0) 0x7f8ff4001490 con 0x7f8ffc10cb90 2024-04-17T22:31:38.754 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.765 7f8ff3fff700 1 --2- 172.21.15.102:0/1728123691 >> v2:172.21.15.102:3300/0 conn(0x7f8ffc10c740 0x7f8ffc1a0c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:38.754 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.765 7f8ffbfff700 1 --2- 172.21.15.102:0/1728123691 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f8ffc0f5770 0x7f8ffc10c110 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 rx=0x7f8fec002f70 tx=0x7f8fec0099d0).ready entity=mon.2 client_cookie=3273092f72f8861 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:38.754 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.766 7f8ffb7fe700 1 -- 172.21.15.102:0/1728123691 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8ffc1aa330 con 0x7f8ffc0f5770 2024-04-17T22:31:38.754 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.766 7f8ffb7fe700 1 -- 172.21.15.102:0/1728123691 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fec00f510 con 0x7f8ffc0f5770 2024-04-17T22:31:38.754 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.766 7f8ffb7fe700 1 -- 172.21.15.102:0/1728123691 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8fec00f6f0 con 0x7f8ffc0f5770 2024-04-17T22:31:38.755 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.766 7f900383d700 1 -- 172.21.15.102:0/1728123691 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f8ffc0f5770 msgr2=0x7f8ffc10c110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:38.755 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.766 7f900383d700 1 --2- 172.21.15.102:0/1728123691 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f8ffc0f5770 0x7f8ffc10c110 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 rx=0x7f8fec002f70 tx=0x7f8fec0099d0).stop 2024-04-17T22:31:38.755 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.766 7f900383d700 1 -- 172.21.15.102:0/1728123691 shutdown_connections 2024-04-17T22:31:38.755 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.766 7f900383d700 1 --2- 172.21.15.102:0/1728123691 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f8ffc0f5770 0x7f8ffc10c110 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:38.755 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.766 7f900383d700 1 --2- 172.21.15.102:0/1728123691 >> v2:172.21.15.102:3300/0 conn(0x7f8ffc10c740 0x7f8ffc1a0c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:38.755 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.766 7f900383d700 1 -- 172.21.15.102:0/1728123691 >> 172.21.15.102:0/1728123691 conn(0x7f8ffc1138c0 msgr2=0x7f8ffc0f4f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:38.755 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.766 7f900383d700 1 -- 172.21.15.102:0/1728123691 shutdown_connections 2024-04-17T22:31:38.755 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.766 7f900383d700 1 -- 172.21.15.102:0/1728123691 wait complete. 2024-04-17T22:31:38.777 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.789 7f900383d700 1 Processor -- start 2024-04-17T22:31:38.777 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.789 7f900383d700 1 -- start start 2024-04-17T22:31:38.777 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.789 7f900383d700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f8ffc0cae50 0x7f8ffc0d6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:38.777 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.789 7f900383d700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8ffc0d69b0 0x7f8ffc1b1c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:38.777 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.789 7f900383d700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f8ffc1b2250 0x7f8ffc1b26a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:38.777 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.789 7f900383d700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ffc0ca7f0 con 0x7f8ffc0d69b0 2024-04-17T22:31:38.778 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.789 7f900383d700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ffc1b2cd0 con 0x7f8ffc1b2250 2024-04-17T22:31:38.778 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.789 7f900383d700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8ffc0d6e00 con 0x7f8ffc0cae50 2024-04-17T22:31:38.778 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.790 7f8ff3fff700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8ffc0d69b0 0x7f8ffc1b1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:38.778 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.790 7f8ff3fff700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8ffc0d69b0 0x7f8ffc1b1c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:57576/0 (socket says 172.21.15.102:57576) 2024-04-17T22:31:38.778 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.790 7f8ff3fff700 1 -- 172.21.15.102:0/2073794961 learned_addr learned my addr 172.21.15.102:0/2073794961 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:31:38.778 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.790 7f9000e39700 1 --2- 172.21.15.102:0/2073794961 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f8ffc1b2250 0x7f8ffc1b26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:38.778 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.790 7f8ffbfff700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f8ffc0cae50 0x7f8ffc0d6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:38.778 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.790 7f8ffbfff700 1 -- 172.21.15.102:0/2073794961 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f8ffc1b2250 msgr2=0x7f8ffc1b26a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:38.778 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.790 7f8ffbfff700 1 --2- 172.21.15.102:0/2073794961 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f8ffc1b2250 0x7f8ffc1b26a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:38.778 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.790 7f8ffbfff700 1 -- 172.21.15.102:0/2073794961 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8ffc0d69b0 msgr2=0x7f8ffc1b1c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:38.778 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.790 7f8ffbfff700 1 --2- 172.21.15.102:0/2073794961 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8ffc0d69b0 0x7f8ffc1b1c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:38.778 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.790 7f8ffbfff700 1 -- 172.21.15.102:0/2073794961 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ffc0cb2a0 con 0x7f8ffc0cae50 2024-04-17T22:31:38.778 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.790 7f8ff3fff700 1 --2- 172.21.15.102:0/2073794961 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8ffc0d69b0 0x7f8ffc1b1c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:38.779 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.790 7f9000e39700 1 --2- 172.21.15.102:0/2073794961 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f8ffc1b2250 0x7f8ffc1b26a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:38.779 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.790 7f8ffbfff700 1 --2- 172.21.15.102:0/2073794961 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f8ffc0cae50 0x7f8ffc0d6380 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 rx=0x7f8ffc0d75c0 tx=0x7f8fec0068c0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:38.779 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.790 7f8ff9ffb700 1 -- 172.21.15.102:0/2073794961 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8ffc0cb2a0 con 0x7f8ffc0cae50 2024-04-17T22:31:38.779 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.790 7f8ff9ffb700 1 -- 172.21.15.102:0/2073794961 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fec0117e0 con 0x7f8ffc0cae50 2024-04-17T22:31:38.779 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.790 7f900383d700 1 -- 172.21.15.102:0/2073794961 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ffc1b9100 con 0x7f8ffc0cae50 2024-04-17T22:31:38.779 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.790 7f8ff9ffb700 1 -- 172.21.15.102:0/2073794961 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8fec022070 con 0x7f8ffc0cae50 2024-04-17T22:31:38.779 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.790 7f900383d700 1 -- 172.21.15.102:0/2073794961 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ffc1b94e0 con 0x7f8ffc0cae50 2024-04-17T22:31:38.779 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.791 7f8ff9ffb700 1 -- 172.21.15.102:0/2073794961 <== mon.2 v2:172.21.15.102:3302/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7f8fec011bb0 con 0x7f8ffc0cae50 2024-04-17T22:31:38.779 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.791 7f8ff9ffb700 1 -- 172.21.15.102:0/2073794961 <== mon.2 v2:172.21.15.102:3302/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 3194+0+0 (secure 0 0 0) 0x7f8fec020130 con 0x7f8ffc0cae50 2024-04-17T22:31:38.779 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.791 7f900383d700 1 -- 172.21.15.102:0/2073794961 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ffc0f0820 con 0x7f8ffc0cae50 2024-04-17T22:31:38.781 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.794 7f8ff9ffb700 1 -- 172.21.15.102:0/2073794961 <== mon.2 v2:172.21.15.102:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+110455 (secure 0 0 0) 0x7f8fec00f300 con 0x7f8ffc0cae50 2024-04-17T22:31:38.885 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.899 7f900383d700 1 -- 172.21.15.102:0/2073794961 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8ffc0efef0 con 0x7f8ffc0cae50 2024-04-17T22:31:38.885 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.899 7f8ff9ffb700 1 -- 172.21.15.102:0/2073794961 <== mon.2 v2:172.21.15.102:3302/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+8764 (secure 0 0 0) 0x7f8fec0115d0 con 0x7f8ffc0cae50 2024-04-17T22:31:38.886 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-17T22:31:38.886 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":25,"fsid":"cb63e789-091d-49b1-b120-55f9db526871","created":"2024-04-17 22:28:48.209635","modified":"2024-04-17 22:31:29.205644","last_up_change":"2024-04-17 22:28:54.236215","last_in_change":"2024-04-17 22:28:54.236215","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":4,"require_min_compat_client":"nautilus","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-04-17 22:28:57.478545","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-17 22:31:26.214102","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-17 22:31:27.183113","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3351ef85-ade4-4b90-a8f4-213f4e00ba01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6812","nonce":42955}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6813","nonce":42955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6815","nonce":42955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6814","nonce":42955}]},"public_addr":"172.21.15.102:6812/42955","cluster_addr":"172.21.15.102:6813/42955","heartbeat_back_addr":"172.21.15.102:6815/42955","heartbeat_front_addr":"172.21.15.102:6814/42955","state":["exists","up"]},{"osd":1,"uuid":"04f67eac-f36b-48f5-982d-4cfc776596d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6808","nonce":42958}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6809","nonce":42958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6811","nonce":42958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6810","nonce":42958}]},"public_addr":"172.21.15.102:6808/42958","cluster_addr":"172.21.15.102:6809/42958","heartbeat_back_addr":"172.21.15.102:6811/42958","heartbeat_front_addr":"172.21.15.102:6810/42958","state":["exists","up"]},{"osd":2,"uuid":"d524204b-70b2-40a8-8f73-f9e7d44318af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6804","nonce":42954}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6805","nonce":42954}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6807","nonce":42954}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6806","nonce":42954}]},"public_addr":"172.21.15.102:6804/42954","cluster_addr":"172.21.15.102:6805/42954","heartbeat_back_addr":"172.21.15.102:6807/42954","heartbeat_front_addr":"172.21.15.102:6806/42954","state":["exists","up"]},{"osd":3,"uuid":"4fc9d20c-1ad0-484f-aafa-2c9a7574245b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6800","nonce":42953}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6801","nonce":42953}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6803","nonce":42953}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6802","nonce":42953}]},"public_addr":"172.21.15.102:6800/42953","cluster_addr":"172.21.15.102:6801/42953","heartbeat_back_addr":"172.21.15.102:6803/42953","heartbeat_front_addr":"172.21.15.102:6802/42953","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2024-04-17T22:31:38.887 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.900 7f900383d700 1 -- 172.21.15.102:0/2073794961 >> v1:172.21.15.102:6816/42360 conn(0x7f8fe00145a0 legacy=0x7f8fe0016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:38.887 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.900 7f900383d700 1 -- 172.21.15.102:0/2073794961 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f8ffc0cae50 msgr2=0x7f8ffc0d6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:38.887 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.900 7f900383d700 1 --2- 172.21.15.102:0/2073794961 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f8ffc0cae50 0x7f8ffc0d6380 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 rx=0x7f8ffc0d75c0 tx=0x7f8fec0068c0).stop 2024-04-17T22:31:38.887 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.900 7f900383d700 1 -- 172.21.15.102:0/2073794961 shutdown_connections 2024-04-17T22:31:38.887 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.900 7f900383d700 1 --2- 172.21.15.102:0/2073794961 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f8ffc1b2250 0x7f8ffc1b26a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:38.888 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.900 7f900383d700 1 --2- 172.21.15.102:0/2073794961 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f8ffc0cae50 0x7f8ffc0d6380 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:38.888 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.900 7f900383d700 1 --2- 172.21.15.102:0/2073794961 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8ffc0d69b0 0x7f8ffc1b1c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:38.888 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.900 7f900383d700 1 -- 172.21.15.102:0/2073794961 >> 172.21.15.102:0/2073794961 conn(0x7f8ffc0cf4c0 msgr2=0x7f8ffc0c3030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:38.888 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.900 7f900383d700 1 -- 172.21.15.102:0/2073794961 shutdown_connections 2024-04-17T22:31:38.888 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:38.900 7f900383d700 1 -- 172.21.15.102:0/2073794961 wait complete. 2024-04-17T22:31:38.898 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-17T22:31:39.024 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:39.037 7f89984fe700 1 Processor -- start 2024-04-17T22:31:39.024 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:39.037 7f89984fe700 1 -- start start 2024-04-17T22:31:39.024 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:39.037 7f89984fe700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f89900f5840 0x7f899010b010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:39.025 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:39.037 7f89984fe700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f899010b640 0x7f899019faf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:39.025 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:39.037 7f89984fe700 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8990103b40 con 0x7f899010ba90 2024-04-17T22:31:39.025 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:39.037 7f89984fe700 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f899010bee0 con 0x7f89900f5840 2024-04-17T22:31:39.025 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:39.037 7f89984fe700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f89900fa370 con 0x7f899010b640 2024-04-17T22:31:39.025 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:39.037 7f8994af8700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f899010b640 0x7f899019faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:39.025 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:39.037 7f89952f9700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f89900f5840 0x7f899010b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:39.025 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:39.037 7f89952f9700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f89900f5840 0x7f899010b010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:57586/0 (socket says 172.21.15.102:57586) 2024-04-17T22:31:39.025 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:39.037 7f8994af8700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f899010b640 0x7f899019faf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:48296/0 (socket says 172.21.15.102:48296) 2024-04-17T22:31:39.025 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:39.037 7f89952f9700 1 -- 172.21.15.102:0/111717707 learned_addr learned my addr 172.21.15.102:0/111717707 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:31:39.025 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:39.037 7f89952f9700 1 -- 172.21.15.102:0/111717707 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f899010b640 msgr2=0x7f899019faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:31:39.025 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:39.037 7f89952f9700 1 --2- 172.21.15.102:0/111717707 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f899010b640 0x7f899019faf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:39.025 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:39.037 7f89952f9700 1 -- 172.21.15.102:0/111717707 >> v1:172.21.15.102:6789/0 conn(0x7f899010ba90 legacy=0x7f89901a0120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:39.025 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:39.038 7f89952f9700 1 -- 172.21.15.102:0/111717707 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89901a0920 con 0x7f89900f5840 2024-04-17T22:31:39.025 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:39.038 7f8994af8700 1 --2- 172.21.15.102:0/111717707 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f899010b640 0x7f899019faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:39.026 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:39.038 7f89952f9700 1 --2- 172.21.15.102:0/111717707 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f89900f5840 0x7f899010b010 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 rx=0x7f8990103d60 tx=0x7f8980009a80).ready entity=mon.0 client_cookie=80a2f9c094ee0fe server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:39.026 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:39.038 7f897ffff700 1 -- 172.21.15.102:0/111717707 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f89901a0920 con 0x7f89900f5840 2024-04-17T22:31:39.026 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:39.038 7f897ffff700 1 -- 172.21.15.102:0/111717707 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f898000f8a0 con 0x7f89900f5840 2024-04-17T22:31:39.026 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:39.038 7f897ffff700 1 -- 172.21.15.102:0/111717707 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f898000fc70 con 0x7f89900f5840 2024-04-17T22:31:39.026 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:39.038 7f89984fe700 1 -- 172.21.15.102:0/111717707 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f89900f5840 msgr2=0x7f899010b010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:39.026 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:39.038 7f89984fe700 1 --2- 172.21.15.102:0/111717707 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f89900f5840 0x7f899010b010 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 rx=0x7f8990103d60 tx=0x7f8980009a80).stop 2024-04-17T22:31:39.026 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:39.038 7f89984fe700 1 -- 172.21.15.102:0/111717707 shutdown_connections 2024-04-17T22:31:39.026 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:39.038 7f89984fe700 1 --2- 172.21.15.102:0/111717707 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f89900f5840 0x7f899010b010 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:39.026 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:39.038 7f89984fe700 1 --2- 172.21.15.102:0/111717707 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f899010b640 0x7f899019faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:39.026 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:39.038 7f89984fe700 1 -- 172.21.15.102:0/111717707 >> 172.21.15.102:0/111717707 conn(0x7f89901138a0 msgr2=0x7f89900f4f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:39.026 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:39.038 7f89984fe700 1 -- 172.21.15.102:0/111717707 shutdown_connections 2024-04-17T22:31:39.026 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:39.038 7f89984fe700 1 -- 172.21.15.102:0/111717707 wait complete. 2024-04-17T22:31:39.049 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:39.061 7f89984fe700 1 Processor -- start 2024-04-17T22:31:39.049 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:39.061 7f89984fe700 1 -- start start 2024-04-17T22:31:39.049 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:39.061 7f89984fe700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f89900dd1e0 0x7f89901adbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:39.049 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:39.061 7f89984fe700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f89901ae1f0 0x7f89901b45a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:39.049 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:39.061 7f89984fe700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f89901ae640 0x7f89901b4bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:39.049 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:39.061 7f89984fe700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89900dc6e0 con 0x7f89901ae640 2024-04-17T22:31:39.049 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:39.061 7f89984fe700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89900dcbe0 con 0x7f89900dd1e0 2024-04-17T22:31:39.049 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:39.061 7f89984fe700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f89901b5200 con 0x7f89901ae1f0 2024-04-17T22:31:39.049 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:39.061 7f89952f9700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f89900dd1e0 0x7f89901adbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:39.049 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:39.061 7f8995afa700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f89901ae640 0x7f89901b4bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:39.049 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:39.061 7f89952f9700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f89900dd1e0 0x7f89901adbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:40344/0 (socket says 172.21.15.102:40344) 2024-04-17T22:31:39.049 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:39.061 7f8995afa700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f89901ae640 0x7f89901b4bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:57596/0 (socket says 172.21.15.102:57596) 2024-04-17T22:31:39.049 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:39.061 7f89952f9700 1 -- 172.21.15.102:0/3426621387 learned_addr learned my addr 172.21.15.102:0/3426621387 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:31:39.050 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:39.061 7f8994af8700 1 --2- 172.21.15.102:0/3426621387 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f89901ae1f0 0x7f89901b45a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:39.050 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:39.062 7f8995afa700 1 -- 172.21.15.102:0/3426621387 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f89901ae1f0 msgr2=0x7f89901b45a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:39.050 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:39.062 7f8995afa700 1 --2- 172.21.15.102:0/3426621387 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f89901ae1f0 0x7f89901b45a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:39.050 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:39.062 7f8995afa700 1 -- 172.21.15.102:0/3426621387 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f89900dd1e0 msgr2=0x7f89901adbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:39.050 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:39.062 7f8995afa700 1 --2- 172.21.15.102:0/3426621387 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f89900dd1e0 0x7f89901adbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:39.050 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:39.062 7f8995afa700 1 -- 172.21.15.102:0/3426621387 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89901b53e0 con 0x7f89901ae640 2024-04-17T22:31:39.050 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:39.062 7f8995afa700 1 --2- 172.21.15.102:0/3426621387 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f89901ae640 0x7f89901b4bd0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 rx=0x7f8988001250 tx=0x7f898800c950).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:39.050 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:39.062 7f897e7fc700 1 -- 172.21.15.102:0/3426621387 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f89901b53e0 con 0x7f89901ae640 2024-04-17T22:31:39.050 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:39.062 7f897e7fc700 1 -- 172.21.15.102:0/3426621387 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8988006ab0 con 0x7f89901ae640 2024-04-17T22:31:39.050 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:39.062 7f89984fe700 1 -- 172.21.15.102:0/3426621387 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89901bbda0 con 0x7f89901ae640 2024-04-17T22:31:39.050 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:39.062 7f897e7fc700 1 -- 172.21.15.102:0/3426621387 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f8988007030 con 0x7f89901ae640 2024-04-17T22:31:39.050 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:39.062 7f89984fe700 1 -- 172.21.15.102:0/3426621387 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89901bc180 con 0x7f89901ae640 2024-04-17T22:31:39.051 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:39.062 7f897e7fc700 1 -- 172.21.15.102:0/3426621387 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7f898801a020 con 0x7f89901ae640 2024-04-17T22:31:39.051 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:39.063 7f897e7fc700 1 -- 172.21.15.102:0/3426621387 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 3194+0+0 (secure 0 0 0) 0x7f898801a9a0 con 0x7f89901ae640 2024-04-17T22:31:39.051 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:39.063 7f89984fe700 1 -- 172.21.15.102:0/3426621387 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89901b5640 con 0x7f89901ae640 2024-04-17T22:31:39.052 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:39.066 7f897e7fc700 1 -- 172.21.15.102:0/3426621387 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+110455 (secure 0 0 0) 0x7f89880068a0 con 0x7f89901ae640 2024-04-17T22:31:39.186 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:39.200 7f89984fe700 1 -- 172.21.15.102:0/3426621387 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f89900f0800 con 0x7f89901ae640 2024-04-17T22:31:39.501 INFO:tasks.ceph.mon.a.smithi102.stderr:2024-04-17 22:31:39.516 7ff15d99f700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-17T22:31:39.503 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:39.518 7f897e7fc700 1 -- 172.21.15.102:0/3426621387 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v12) v1 ==== 161+0+0 (secure 0 0 0) 0x7f8990114420 con 0x7f89901ae640 2024-04-17T22:31:39.505 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:39.520 7f89984fe700 1 -- 172.21.15.102:0/3426621387 >> v1:172.21.15.102:6816/42360 conn(0x7f89700145a0 legacy=0x7f8970016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:39.506 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:39.520 7f89984fe700 1 -- 172.21.15.102:0/3426621387 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f89901ae640 msgr2=0x7f89901b4bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:39.506 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:39.520 7f89984fe700 1 --2- 172.21.15.102:0/3426621387 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f89901ae640 0x7f89901b4bd0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 rx=0x7f8988001250 tx=0x7f898800c950).stop 2024-04-17T22:31:39.506 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:39.521 7f89984fe700 1 -- 172.21.15.102:0/3426621387 shutdown_connections 2024-04-17T22:31:39.507 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:39.521 7f89984fe700 1 --2- 172.21.15.102:0/3426621387 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f89901ae640 0x7f89901b4bd0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:39.507 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:39.521 7f89984fe700 1 --2- 172.21.15.102:0/3426621387 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f89900dd1e0 0x7f89901adbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:39.507 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:39.521 7f89984fe700 1 --2- 172.21.15.102:0/3426621387 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f89901ae1f0 0x7f89901b45a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:39.507 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:39.521 7f89984fe700 1 -- 172.21.15.102:0/3426621387 >> 172.21.15.102:0/3426621387 conn(0x7f89900e0670 msgr2=0x7f89900ce820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:39.507 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:39.521 7f89984fe700 1 -- 172.21.15.102:0/3426621387 shutdown_connections 2024-04-17T22:31:39.507 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:39.521 7f89984fe700 1 -- 172.21.15.102:0/3426621387 wait complete. 2024-04-17T22:31:39.508 INFO:teuthology.orchestra.run.smithi102.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-17T22:31:39.519 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-17T22:31:39.522 INFO:tasks.ceph.mds.a.smithi102.stderr:warning: line 130: 'mon_down_mkfs_grace' in section 'mon' redefined 2024-04-17T22:31:39.572 INFO:tasks.ceph.mds.a.smithi102.stdout:starting mds.a at 2024-04-17T22:31:39.651 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:39.663 7f22474f2700 1 Processor -- start 2024-04-17T22:31:39.651 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:39.663 7f22474f2700 1 -- start start 2024-04-17T22:31:39.651 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:39.663 7f22474f2700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f22400f9650 0x7f224010e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:39.651 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:39.663 7f22474f2700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f224010ef20 0x7f22401a0df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:39.651 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:39.663 7f22474f2700 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22400ad030 con 0x7f224010f370 2024-04-17T22:31:39.651 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:39.663 7f22474f2700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2240102e20 con 0x7f224010ef20 2024-04-17T22:31:39.651 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:39.663 7f22474f2700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f22401a1c20 con 0x7f22400f9650 2024-04-17T22:31:39.651 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:39.663 7f223ffff700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f22400f9650 0x7f224010e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:39.651 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:39.663 7f2237fff700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f224010ef20 0x7f22401a0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:39.652 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:39.663 7f2244aee700 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f224010f370 0x7f22401a1420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:45742/0 (socket says 172.21.15.102:45742) 2024-04-17T22:31:39.652 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:39.663 7f2244aee700 1 -- 172.21.15.102:0/4121389365 learned_addr learned my addr 172.21.15.102:0/4121389365 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-17T22:31:39.652 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:39.663 7f223f7fe700 1 -- 172.21.15.102:0/4121389365 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3391094179 0 0) 0x7f2238001490 con 0x7f224010f370 2024-04-17T22:31:39.652 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:39.663 7f223f7fe700 1 -- 172.21.15.102:0/4121389365 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2220003270 con 0x7f224010f370 2024-04-17T22:31:39.652 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:39.663 7f2237fff700 1 -- 172.21.15.102:0/4121389365 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f22400f9650 msgr2=0x7f224010e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:31:39.652 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:39.663 7f2237fff700 1 --2- 172.21.15.102:0/4121389365 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f22400f9650 0x7f224010e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:39.652 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:39.663 7f2237fff700 1 -- 172.21.15.102:0/4121389365 >> v1:172.21.15.102:6789/0 conn(0x7f224010f370 legacy=0x7f22401a1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:39.652 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:39.663 7f2237fff700 1 -- 172.21.15.102:0/4121389365 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22401aa520 con 0x7f224010ef20 2024-04-17T22:31:39.652 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:39.663 7f223ffff700 1 --2- 172.21.15.102:0/4121389365 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f22400f9650 0x7f224010e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:39.652 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:39.664 7f2237fff700 1 --2- 172.21.15.102:0/4121389365 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f224010ef20 0x7f22401a0df0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 rx=0x7f2228002f70 tx=0x7f222800ba20).ready entity=mon.1 client_cookie=a9121c9423deb965 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:39.652 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:39.664 7f223f7fe700 1 -- 172.21.15.102:0/4121389365 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f22401aa520 con 0x7f224010ef20 2024-04-17T22:31:39.652 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:39.664 7f223f7fe700 1 -- 172.21.15.102:0/4121389365 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f222800d570 con 0x7f224010ef20 2024-04-17T22:31:39.652 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:39.664 7f223f7fe700 1 -- 172.21.15.102:0/4121389365 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f222800d940 con 0x7f224010ef20 2024-04-17T22:31:39.653 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:39.664 7f22474f2700 1 -- 172.21.15.102:0/4121389365 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f224010ef20 msgr2=0x7f22401a0df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:39.653 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:39.664 7f22474f2700 1 --2- 172.21.15.102:0/4121389365 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f224010ef20 0x7f22401a0df0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 rx=0x7f2228002f70 tx=0x7f222800ba20).stop 2024-04-17T22:31:39.653 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:39.664 7f22474f2700 1 -- 172.21.15.102:0/4121389365 shutdown_connections 2024-04-17T22:31:39.653 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:39.664 7f22474f2700 1 --2- 172.21.15.102:0/4121389365 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f22400f9650 0x7f224010e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:39.653 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:39.664 7f22474f2700 1 --2- 172.21.15.102:0/4121389365 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f224010ef20 0x7f22401a0df0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:39.653 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:39.664 7f22474f2700 1 -- 172.21.15.102:0/4121389365 >> 172.21.15.102:0/4121389365 conn(0x7f2240113000 msgr2=0x7f2240113450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:39.653 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:39.664 7f22474f2700 1 -- 172.21.15.102:0/4121389365 shutdown_connections 2024-04-17T22:31:39.653 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:39.664 7f22474f2700 1 -- 172.21.15.102:0/4121389365 wait complete. 2024-04-17T22:31:39.673 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:39.687 7f22474f2700 1 Processor -- start 2024-04-17T22:31:39.673 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:39.687 7f22474f2700 1 -- start start 2024-04-17T22:31:39.675 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:39.688 7f22474f2700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f22401ad340 0x7f22401ad790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:39.676 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:39.688 7f22474f2700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f22401addc0 0x7f22401a2660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:39.676 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:39.688 7f22474f2700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f22401a2c90 0x7f22401a30e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:39.676 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:39.688 7f22474f2700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2240089f00 con 0x7f22401addc0 2024-04-17T22:31:39.676 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:39.688 7f22474f2700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22400cb030 con 0x7f22401a2c90 2024-04-17T22:31:39.676 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:39.688 7f22474f2700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f22401a4160 con 0x7f22401ad340 2024-04-17T22:31:39.676 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:39.688 7f223ffff700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f22401ad340 0x7f22401ad790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:39.676 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:39.688 7f2244aee700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f22401a2c90 0x7f22401a30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:39.676 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:39.688 7f223ffff700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f22401ad340 0x7f22401ad790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:48326/0 (socket says 172.21.15.102:48326) 2024-04-17T22:31:39.676 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:39.688 7f223ffff700 1 -- 172.21.15.102:0/1487438737 learned_addr learned my addr 172.21.15.102:0/1487438737 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:31:39.676 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:39.688 7f2237fff700 1 --2- 172.21.15.102:0/1487438737 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f22401addc0 0x7f22401a2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:39.676 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:39.688 7f2244aee700 1 -- 172.21.15.102:0/1487438737 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f22401ad340 msgr2=0x7f22401ad790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:39.676 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:39.688 7f2244aee700 1 --2- 172.21.15.102:0/1487438737 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f22401ad340 0x7f22401ad790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:39.676 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:39.688 7f2244aee700 1 -- 172.21.15.102:0/1487438737 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f22401addc0 msgr2=0x7f22401a2660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:39.677 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:39.688 7f2244aee700 1 --2- 172.21.15.102:0/1487438737 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f22401addc0 0x7f22401a2660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:39.677 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:39.688 7f2244aee700 1 -- 172.21.15.102:0/1487438737 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22401a3b10 con 0x7f22401a2c90 2024-04-17T22:31:39.677 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:39.688 7f223ffff700 1 --2- 172.21.15.102:0/1487438737 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f22401ad340 0x7f22401ad790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:39.677 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:39.688 7f2244aee700 1 --2- 172.21.15.102:0/1487438737 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f22401a2c90 0x7f22401a30e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 rx=0x7f2238001250 tx=0x7f223800c950).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:39.677 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:39.688 7f223dffb700 1 -- 172.21.15.102:0/1487438737 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f22401a3b10 con 0x7f22401a2c90 2024-04-17T22:31:39.677 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:39.688 7f223dffb700 1 -- 172.21.15.102:0/1487438737 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2238005bb0 con 0x7f22401a2c90 2024-04-17T22:31:39.677 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:39.688 7f22474f2700 1 -- 172.21.15.102:0/1487438737 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22401bb440 con 0x7f22401a2c90 2024-04-17T22:31:39.677 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:39.688 7f223dffb700 1 -- 172.21.15.102:0/1487438737 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f22380048b0 con 0x7f22401a2c90 2024-04-17T22:31:39.677 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:39.689 7f22474f2700 1 -- 172.21.15.102:0/1487438737 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22401bb640 con 0x7f22401a2c90 2024-04-17T22:31:39.677 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:39.689 7f223dffb700 1 -- 172.21.15.102:0/1487438737 <== mon.1 v2:172.21.15.102:3301/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7f2238004ab0 con 0x7f22401a2c90 2024-04-17T22:31:39.677 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:39.689 7f223dffb700 1 -- 172.21.15.102:0/1487438737 <== mon.1 v2:172.21.15.102:3301/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 3237+0+0 (secure 0 0 0) 0x7f223800bac0 con 0x7f22401a2c90 2024-04-17T22:31:39.677 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:39.689 7f22474f2700 1 -- 172.21.15.102:0/1487438737 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22400f09f0 con 0x7f22401a2c90 2024-04-17T22:31:39.678 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:39.693 7f223dffb700 1 -- 172.21.15.102:0/1487438737 <== mon.1 v2:172.21.15.102:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+110455 (secure 0 0 0) 0x7f22380059a0 con 0x7f22401a2c90 2024-04-17T22:31:39.813 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:39.827 7f22474f2700 1 -- 172.21.15.102:0/1487438737 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f22401ae430 con 0x7f22401a2c90 2024-04-17T22:31:40.508 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:40.522 7f223dffb700 1 -- 172.21.15.102:0/1487438737 <== mon.1 v2:172.21.15.102:3301/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v13) v1 ==== 104+0+0 (secure 0 0 0) 0x7f2238000fb0 con 0x7f22401a2c90 2024-04-17T22:31:40.511 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:40.525 7f22474f2700 1 -- 172.21.15.102:0/1487438737 >> v1:172.21.15.102:6816/42360 conn(0x7f22240145a0 legacy=0x7f2224016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:40.511 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:40.525 7f22474f2700 1 -- 172.21.15.102:0/1487438737 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f22401a2c90 msgr2=0x7f22401a30e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:40.511 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:40.525 7f22474f2700 1 --2- 172.21.15.102:0/1487438737 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f22401a2c90 0x7f22401a30e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 rx=0x7f2238001250 tx=0x7f223800c950).stop 2024-04-17T22:31:40.511 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:40.525 7f22474f2700 1 -- 172.21.15.102:0/1487438737 shutdown_connections 2024-04-17T22:31:40.511 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:40.525 7f22474f2700 1 --2- 172.21.15.102:0/1487438737 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f22401a2c90 0x7f22401a30e0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:40.511 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:40.525 7f22474f2700 1 --2- 172.21.15.102:0/1487438737 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f22401ad340 0x7f22401ad790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:40.511 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:40.525 7f22474f2700 1 --2- 172.21.15.102:0/1487438737 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f22401addc0 0x7f22401a2660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:40.512 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:40.525 7f22474f2700 1 -- 172.21.15.102:0/1487438737 >> 172.21.15.102:0/1487438737 conn(0x7f22400c5c60 msgr2=0x7f22400d6a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:40.512 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:40.525 7f22474f2700 1 -- 172.21.15.102:0/1487438737 shutdown_connections 2024-04-17T22:31:40.512 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:40.525 7f22474f2700 1 -- 172.21.15.102:0/1487438737 wait complete. 2024-04-17T22:31:40.524 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-17T22:31:40.661 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:40.672 7f6849ffd700 1 Processor -- start 2024-04-17T22:31:40.661 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:40.672 7f6849ffd700 1 -- start start 2024-04-17T22:31:40.661 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:40.672 7f6849ffd700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f68440f56f0 0x7f684410e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:40.662 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:40.672 7f6849ffd700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f684410e980 0x7f68440f1430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:40.662 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:40.672 7f6849ffd700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f68440f1a60 0x7f68441a9ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:40.662 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:40.672 7f6849ffd700 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f684410edd0 con 0x7f684410e980 2024-04-17T22:31:40.662 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:40.672 7f6849ffd700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6844110960 con 0x7f68440f1a60 2024-04-17T22:31:40.662 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:40.672 7f6849ffd700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f68440f1eb0 con 0x7f68440f56f0 2024-04-17T22:31:40.662 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:40.672 7f683bfff700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f684410e980 0x7f68440f1430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:40.662 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:40.672 7f683bfff700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f684410e980 0x7f68440f1430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:57604/0 (socket says 172.21.15.102:57604) 2024-04-17T22:31:40.662 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:40.672 7f68435c7700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f68440f1a60 0x7f68441a9ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:40.662 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:40.672 7f683bfff700 1 -- 172.21.15.102:0/2062968201 learned_addr learned my addr 172.21.15.102:0/2062968201 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:31:40.662 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:40.672 7f6842dc6700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f68440f56f0 0x7f684410e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:40.662 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:40.673 7f68435c7700 1 -- 172.21.15.102:0/2062968201 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f68440f56f0 msgr2=0x7f684410e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:31:40.662 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:40.673 7f68435c7700 1 --2- 172.21.15.102:0/2062968201 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f68440f56f0 0x7f684410e350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:40.662 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:40.673 7f68435c7700 1 -- 172.21.15.102:0/2062968201 >> v2:172.21.15.102:3300/0 conn(0x7f684410e980 msgr2=0x7f68440f1430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:31:40.663 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:40.673 7f68435c7700 1 --2- 172.21.15.102:0/2062968201 >> v2:172.21.15.102:3300/0 conn(0x7f684410e980 0x7f68440f1430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:40.663 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:40.673 7f68435c7700 1 -- 172.21.15.102:0/2062968201 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68440f20e0 con 0x7f68440f1a60 2024-04-17T22:31:40.663 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:40.673 7f6842dc6700 1 --2- 172.21.15.102:0/2062968201 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f68440f56f0 0x7f684410e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:40.663 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:40.673 7f683bfff700 1 --2- 172.21.15.102:0/2062968201 >> v2:172.21.15.102:3300/0 conn(0x7f684410e980 0x7f68440f1430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:40.663 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:40.673 7f68435c7700 1 --2- 172.21.15.102:0/2062968201 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f68440f1a60 0x7f68441a9ee0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 rx=0x7f68440f42f0 tx=0x7f683c009d40).ready entity=mon.1 client_cookie=a9183e7548faa775 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:40.663 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:40.673 7f68425c5700 1 -- 172.21.15.102:0/2062968201 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f68440f20e0 con 0x7f68440f1a60 2024-04-17T22:31:40.663 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:40.673 7f68425c5700 1 -- 172.21.15.102:0/2062968201 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f683c00b7c0 con 0x7f68440f1a60 2024-04-17T22:31:40.663 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:40.673 7f68425c5700 1 -- 172.21.15.102:0/2062968201 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f683c00b9a0 con 0x7f68440f1a60 2024-04-17T22:31:40.663 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:40.673 7f6849ffd700 1 -- 172.21.15.102:0/2062968201 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f68440f1a60 msgr2=0x7f68441a9ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:40.663 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:40.673 7f6849ffd700 1 --2- 172.21.15.102:0/2062968201 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f68440f1a60 0x7f68441a9ee0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 rx=0x7f68440f42f0 tx=0x7f683c009d40).stop 2024-04-17T22:31:40.663 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:40.673 7f6849ffd700 1 -- 172.21.15.102:0/2062968201 shutdown_connections 2024-04-17T22:31:40.663 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:40.673 7f6849ffd700 1 --2- 172.21.15.102:0/2062968201 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f68440f1a60 0x7f68441a9ee0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:40.663 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:40.673 7f6849ffd700 1 --2- 172.21.15.102:0/2062968201 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f68440f56f0 0x7f684410e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:40.663 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:40.673 7f6849ffd700 1 --2- 172.21.15.102:0/2062968201 >> v2:172.21.15.102:3300/0 conn(0x7f684410e980 0x7f68440f1430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:40.664 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:40.673 7f6849ffd700 1 -- 172.21.15.102:0/2062968201 >> 172.21.15.102:0/2062968201 conn(0x7f68441135a0 msgr2=0x7f68440f4c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:40.664 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:40.674 7f6849ffd700 1 -- 172.21.15.102:0/2062968201 shutdown_connections 2024-04-17T22:31:40.664 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:40.674 7f6849ffd700 1 -- 172.21.15.102:0/2062968201 wait complete. 2024-04-17T22:31:40.682 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:40.696 7f6849ffd700 1 Processor -- start 2024-04-17T22:31:40.682 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:40.696 7f6849ffd700 1 -- start start 2024-04-17T22:31:40.684 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:40.696 7f6849ffd700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f68441ad570 0x7f68441ad9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:40.684 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:40.696 7f6849ffd700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f68441adff0 0x7f68441b3fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:40.684 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:40.696 7f6849ffd700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f68441b45e0 0x7f68441b4a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:40.685 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:40.697 7f6849ffd700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68440cc690 con 0x7f68441ad570 2024-04-17T22:31:40.685 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:40.697 7f6849ffd700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68440d5bc0 con 0x7f68441adff0 2024-04-17T22:31:40.685 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:40.697 7f6849ffd700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f68440dc3c0 con 0x7f68441b45e0 2024-04-17T22:31:40.685 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:40.697 7f6842dc6700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f68441ad570 0x7f68441ad9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:40.685 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:40.697 7f68435c7700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f68441b45e0 0x7f68441b4a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:40.685 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:40.697 7f683bfff700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f68441adff0 0x7f68441b3fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:40.685 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:40.697 7f6842dc6700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f68441ad570 0x7f68441ad9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:57608/0 (socket says 172.21.15.102:57608) 2024-04-17T22:31:40.685 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:40.697 7f68435c7700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f68441b45e0 0x7f68441b4a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:48344/0 (socket says 172.21.15.102:48344) 2024-04-17T22:31:40.685 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:40.697 7f6842dc6700 1 -- 172.21.15.102:0/2403719709 learned_addr learned my addr 172.21.15.102:0/2403719709 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:31:40.685 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:40.697 7f683bfff700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f68441adff0 0x7f68441b3fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:40382/0 (socket says 172.21.15.102:40382) 2024-04-17T22:31:40.685 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:40.697 7f683bfff700 1 -- 172.21.15.102:0/2403719709 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f68441b45e0 msgr2=0x7f68441b4a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:40.685 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:40.697 7f683bfff700 1 --2- 172.21.15.102:0/2403719709 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f68441b45e0 0x7f68441b4a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:40.685 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:40.697 7f683bfff700 1 -- 172.21.15.102:0/2403719709 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f68441ad570 msgr2=0x7f68441ad9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:40.685 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:40.697 7f683bfff700 1 --2- 172.21.15.102:0/2403719709 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f68441ad570 0x7f68441ad9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:40.686 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:40.697 7f683bfff700 1 -- 172.21.15.102:0/2403719709 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68441bb720 con 0x7f68441adff0 2024-04-17T22:31:40.686 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:40.697 7f683bfff700 1 --2- 172.21.15.102:0/2403719709 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f68441adff0 0x7f68441b3fb0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 rx=0x7f682c00c000 tx=0x7f682c00e9e0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:40.686 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:40.697 7f6840dc2700 1 -- 172.21.15.102:0/2403719709 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f68441bb720 con 0x7f68441adff0 2024-04-17T22:31:40.686 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:40.697 7f6840dc2700 1 -- 172.21.15.102:0/2403719709 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f682c012600 con 0x7f68441adff0 2024-04-17T22:31:40.686 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:40.697 7f6849ffd700 1 -- 172.21.15.102:0/2403719709 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68441bb920 con 0x7f68441adff0 2024-04-17T22:31:40.686 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:40.697 7f6840dc2700 1 -- 172.21.15.102:0/2403719709 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f682c00f040 con 0x7f68441adff0 2024-04-17T22:31:40.686 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:40.697 7f6849ffd700 1 -- 172.21.15.102:0/2403719709 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68441bbd00 con 0x7f68441adff0 2024-04-17T22:31:40.686 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:40.698 7f6840dc2700 1 -- 172.21.15.102:0/2403719709 <== mon.1 v2:172.21.15.102:3301/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7f682c011070 con 0x7f68441adff0 2024-04-17T22:31:40.686 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:40.698 7f6840dc2700 1 -- 172.21.15.102:0/2403719709 <== mon.1 v2:172.21.15.102:3301/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 3237+0+0 (secure 0 0 0) 0x7f682c010040 con 0x7f68441adff0 2024-04-17T22:31:40.686 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:40.698 7f6849ffd700 1 -- 172.21.15.102:0/2403719709 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68440f0500 con 0x7f68441adff0 2024-04-17T22:31:40.687 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:40.701 7f6840dc2700 1 -- 172.21.15.102:0/2403719709 <== mon.1 v2:172.21.15.102:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+110455 (secure 0 0 0) 0x7f682c0123f0 con 0x7f68441adff0 2024-04-17T22:31:40.792 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:40.806 7f6849ffd700 1 -- 172.21.15.102:0/2403719709 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f6844103180 con 0x7f68441adff0 2024-04-17T22:31:41.518 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:41.532 7f6840dc2700 1 -- 172.21.15.102:0/2403719709 <== mon.1 v2:172.21.15.102:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v27) v1 ==== 181+0+0 (secure 0 0 0) 0x7f682c000c30 con 0x7f68441adff0 2024-04-17T22:31:41.526 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:41.536 7f6849ffd700 1 -- 172.21.15.102:0/2403719709 >> v1:172.21.15.102:6816/42360 conn(0x7f68280145a0 legacy=0x7f6828016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:41.526 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:41.536 7f6849ffd700 1 -- 172.21.15.102:0/2403719709 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f68441adff0 msgr2=0x7f68441b3fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:41.526 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:41.536 7f6849ffd700 1 --2- 172.21.15.102:0/2403719709 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f68441adff0 0x7f68441b3fb0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 rx=0x7f682c00c000 tx=0x7f682c00e9e0).stop 2024-04-17T22:31:41.526 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:41.537 7f6849ffd700 1 -- 172.21.15.102:0/2403719709 shutdown_connections 2024-04-17T22:31:41.526 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:41.537 7f6849ffd700 1 --2- 172.21.15.102:0/2403719709 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f68441b45e0 0x7f68441b4a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:41.526 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:41.537 7f6849ffd700 1 --2- 172.21.15.102:0/2403719709 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f68441ad570 0x7f68441ad9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:41.526 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:41.537 7f6849ffd700 1 --2- 172.21.15.102:0/2403719709 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f68441adff0 0x7f68441b3fb0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:41.526 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:41.537 7f6849ffd700 1 -- 172.21.15.102:0/2403719709 >> 172.21.15.102:0/2403719709 conn(0x7f68440c7af0 msgr2=0x7f68441a7f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:41.526 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:41.537 7f6849ffd700 1 -- 172.21.15.102:0/2403719709 shutdown_connections 2024-04-17T22:31:41.527 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:41.537 7f6849ffd700 1 -- 172.21.15.102:0/2403719709 wait complete. 2024-04-17T22:31:41.527 INFO:teuthology.orchestra.run.smithi102.stderr:pool 'cephfs_metadata' removed 2024-04-17T22:31:41.543 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-17T22:31:41.684 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:41.693 7f6e947f6700 1 Processor -- start 2024-04-17T22:31:41.684 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:41.693 7f6e947f6700 1 -- start start 2024-04-17T22:31:41.684 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:41.693 7f6e947f6700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f6e8c0f56f0 0x7f6e8c10e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:41.684 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:41.693 7f6e947f6700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f6e8c10e980 0x7f6e8c0f1430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:41.684 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:41.693 7f6e947f6700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6e8c0f1a60 0x7f6e8c1a9ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:41.684 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:41.693 7f6e947f6700 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e8c10edd0 con 0x7f6e8c0f56f0 2024-04-17T22:31:41.684 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:41.693 7f6e947f6700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e8c110960 con 0x7f6e8c0f1a60 2024-04-17T22:31:41.684 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:41.693 7f6e947f6700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6e8c0f1eb0 con 0x7f6e8c10e980 2024-04-17T22:31:41.684 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:41.694 7f6e90df0700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f6e8c10e980 0x7f6e8c0f1430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:41.684 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:41.694 7f6e91df2700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6e8c0f1a60 0x7f6e8c1a9ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:41.684 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:41.694 7f6e915f1700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f6e8c0f56f0 0x7f6e8c10e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:41.684 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:41.694 7f6e90df0700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f6e8c10e980 0x7f6e8c0f1430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:48358/0 (socket says 172.21.15.102:48358) 2024-04-17T22:31:41.685 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:41.694 7f6e91df2700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6e8c0f1a60 0x7f6e8c1a9ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:40390/0 (socket says 172.21.15.102:40390) 2024-04-17T22:31:41.685 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:41.694 7f6e90df0700 1 -- 172.21.15.102:0/3141044479 learned_addr learned my addr 172.21.15.102:0/3141044479 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:31:41.685 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:41.694 7f6e915f1700 1 -- 172.21.15.102:0/3141044479 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f6e8c10e980 msgr2=0x7f6e8c0f1430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:31:41.685 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:41.694 7f6e915f1700 1 --2- 172.21.15.102:0/3141044479 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f6e8c10e980 0x7f6e8c0f1430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:41.685 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:41.694 7f6e915f1700 1 -- 172.21.15.102:0/3141044479 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6e8c0f1a60 msgr2=0x7f6e8c1a9ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:31:41.685 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:41.694 7f6e915f1700 1 --2- 172.21.15.102:0/3141044479 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6e8c0f1a60 0x7f6e8c1a9ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:41.685 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:41.694 7f6e915f1700 1 -- 172.21.15.102:0/3141044479 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e8c0f20e0 con 0x7f6e8c0f56f0 2024-04-17T22:31:41.685 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:41.694 7f6e90df0700 1 --2- 172.21.15.102:0/3141044479 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f6e8c10e980 0x7f6e8c0f1430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:41.685 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:41.694 7f6e91df2700 1 --2- 172.21.15.102:0/3141044479 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6e8c0f1a60 0x7f6e8c1a9ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:41.685 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:41.694 7f6e915f1700 1 --2- 172.21.15.102:0/3141044479 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6e8c0f56f0 0x7f6e8c10e350 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 rx=0x7f6e7c002f70 tx=0x7f6e7c00b9f0).ready entity=mon.0 client_cookie=1af4bf180888e23d server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:41.685 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:41.694 7f6e7bfff700 1 -- 172.21.15.102:0/3141044479 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f6e8c0f20e0 con 0x7f6e8c0f56f0 2024-04-17T22:31:41.685 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:41.694 7f6e7bfff700 1 -- 172.21.15.102:0/3141044479 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e7c00d570 con 0x7f6e8c0f56f0 2024-04-17T22:31:41.685 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:41.694 7f6e7bfff700 1 -- 172.21.15.102:0/3141044479 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f6e7c00d940 con 0x7f6e8c0f56f0 2024-04-17T22:31:41.685 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:41.694 7f6e947f6700 1 -- 172.21.15.102:0/3141044479 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6e8c0f56f0 msgr2=0x7f6e8c10e350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:41.686 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:41.694 7f6e947f6700 1 --2- 172.21.15.102:0/3141044479 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6e8c0f56f0 0x7f6e8c10e350 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 rx=0x7f6e7c002f70 tx=0x7f6e7c00b9f0).stop 2024-04-17T22:31:41.686 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:41.695 7f6e947f6700 1 -- 172.21.15.102:0/3141044479 shutdown_connections 2024-04-17T22:31:41.686 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:41.695 7f6e947f6700 1 --2- 172.21.15.102:0/3141044479 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6e8c0f1a60 0x7f6e8c1a9ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:41.686 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:41.695 7f6e947f6700 1 --2- 172.21.15.102:0/3141044479 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6e8c0f56f0 0x7f6e8c10e350 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:41.686 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:41.695 7f6e947f6700 1 --2- 172.21.15.102:0/3141044479 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f6e8c10e980 0x7f6e8c0f1430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:41.686 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:41.695 7f6e947f6700 1 -- 172.21.15.102:0/3141044479 >> 172.21.15.102:0/3141044479 conn(0x7f6e8c1135a0 msgr2=0x7f6e8c0f4c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:41.686 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:41.695 7f6e947f6700 1 -- 172.21.15.102:0/3141044479 shutdown_connections 2024-04-17T22:31:41.686 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:41.695 7f6e947f6700 1 -- 172.21.15.102:0/3141044479 wait complete. 2024-04-17T22:31:41.706 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:41.717 7f6e947f6700 1 Processor -- start 2024-04-17T22:31:41.706 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:41.717 7f6e947f6700 1 -- start start 2024-04-17T22:31:41.706 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:41.718 7f6e947f6700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6e8c1b0850 0x7f6e8c1b0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:41.706 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:41.718 7f6e947f6700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f6e8c1b12d0 0x7f6e8c1b3c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:41.707 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:41.718 7f6e947f6700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6e8c1b4260 0x7f6e8c1b46b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:41.707 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:41.718 7f6e947f6700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e8c0c55b0 con 0x7f6e8c1b4260 2024-04-17T22:31:41.707 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:41.718 7f6e947f6700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e8c0d6040 con 0x7f6e8c1b0850 2024-04-17T22:31:41.707 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:41.718 7f6e947f6700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6e8c0d5cb0 con 0x7f6e8c1b12d0 2024-04-17T22:31:41.707 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:41.718 7f6e915f1700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6e8c1b0850 0x7f6e8c1b0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:41.707 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:41.718 7f6e90df0700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f6e8c1b12d0 0x7f6e8c1b3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:41.708 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:41.718 7f6e91df2700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6e8c1b4260 0x7f6e8c1b46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:41.708 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:41.718 7f6e915f1700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6e8c1b0850 0x7f6e8c1b0ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:40392/0 (socket says 172.21.15.102:40392) 2024-04-17T22:31:41.708 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:41.718 7f6e915f1700 1 -- 172.21.15.102:0/3154595410 learned_addr learned my addr 172.21.15.102:0/3154595410 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:31:41.708 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:41.718 7f6e90df0700 1 -- 172.21.15.102:0/3154595410 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6e8c1b0850 msgr2=0x7f6e8c1b0ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:41.708 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:41.718 7f6e90df0700 1 --2- 172.21.15.102:0/3154595410 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6e8c1b0850 0x7f6e8c1b0ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:41.708 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:41.718 7f6e90df0700 1 -- 172.21.15.102:0/3154595410 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6e8c1b4260 msgr2=0x7f6e8c1b46b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:41.708 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:41.718 7f6e90df0700 1 --2- 172.21.15.102:0/3154595410 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6e8c1b4260 0x7f6e8c1b46b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:41.709 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:41.718 7f6e90df0700 1 -- 172.21.15.102:0/3154595410 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e8c1bb110 con 0x7f6e8c1b12d0 2024-04-17T22:31:41.709 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:41.718 7f6e915f1700 1 --2- 172.21.15.102:0/3154595410 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6e8c1b0850 0x7f6e8c1b0ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:41.709 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:41.718 7f6e91df2700 1 --2- 172.21.15.102:0/3154595410 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6e8c1b4260 0x7f6e8c1b46b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:41.709 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:41.718 7f6e90df0700 1 --2- 172.21.15.102:0/3154595410 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f6e8c1b12d0 0x7f6e8c1b3c30 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 rx=0x7f6e8800c000 tx=0x7f6e8800e930).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:41.709 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:41.718 7f6e7a7fc700 1 -- 172.21.15.102:0/3154595410 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f6e8c1bb110 con 0x7f6e8c1b12d0 2024-04-17T22:31:41.709 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:41.719 7f6e7a7fc700 1 -- 172.21.15.102:0/3154595410 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e88012600 con 0x7f6e8c1b12d0 2024-04-17T22:31:41.709 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:41.719 7f6e7a7fc700 1 -- 172.21.15.102:0/3154595410 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f6e8800f040 con 0x7f6e8c1b12d0 2024-04-17T22:31:41.710 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:41.719 7f6e947f6700 1 -- 172.21.15.102:0/3154595410 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e8c1bb310 con 0x7f6e8c1b12d0 2024-04-17T22:31:41.710 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:41.719 7f6e947f6700 1 -- 172.21.15.102:0/3154595410 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e8c1bb510 con 0x7f6e8c1b12d0 2024-04-17T22:31:41.710 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:41.719 7f6e7a7fc700 1 -- 172.21.15.102:0/3154595410 <== mon.2 v2:172.21.15.102:3302/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7f6e88022060 con 0x7f6e8c1b12d0 2024-04-17T22:31:41.710 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:41.719 7f6e7a7fc700 1 -- 172.21.15.102:0/3154595410 <== mon.2 v2:172.21.15.102:3302/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 2799+0+0 (secure 0 0 0) 0x7f6e88010040 con 0x7f6e8c1b12d0 2024-04-17T22:31:41.710 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:41.720 7f6e947f6700 1 -- 172.21.15.102:0/3154595410 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e8c0f0500 con 0x7f6e8c1b12d0 2024-04-17T22:31:41.711 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:41.723 7f6e7a7fc700 1 -- 172.21.15.102:0/3154595410 <== mon.2 v2:172.21.15.102:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+110455 (secure 0 0 0) 0x7f6e880123f0 con 0x7f6e8c1b12d0 2024-04-17T22:31:41.814 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:41.828 7f6e947f6700 1 -- 172.21.15.102:0/3154595410 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f6e8c103180 con 0x7f6e8c1b12d0 2024-04-17T22:31:42.526 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:42.540 7f6e7a7fc700 1 -- 172.21.15.102:0/3154595410 <== mon.2 v2:172.21.15.102:3302/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v28) v1 ==== 169+0+0 (secure 0 0 0) 0x7f6e88000c30 con 0x7f6e8c1b12d0 2024-04-17T22:31:42.530 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:42.543 7f6e947f6700 1 -- 172.21.15.102:0/3154595410 >> v1:172.21.15.102:6816/42360 conn(0x7f6e6c0145a0 legacy=0x7f6e6c016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:42.530 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:42.543 7f6e947f6700 1 -- 172.21.15.102:0/3154595410 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f6e8c1b12d0 msgr2=0x7f6e8c1b3c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:42.530 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:42.543 7f6e947f6700 1 --2- 172.21.15.102:0/3154595410 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f6e8c1b12d0 0x7f6e8c1b3c30 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 rx=0x7f6e8800c000 tx=0x7f6e8800e930).stop 2024-04-17T22:31:42.530 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:42.543 7f6e947f6700 1 -- 172.21.15.102:0/3154595410 shutdown_connections 2024-04-17T22:31:42.530 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:42.543 7f6e947f6700 1 --2- 172.21.15.102:0/3154595410 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6e8c1b4260 0x7f6e8c1b46b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:42.530 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:42.543 7f6e947f6700 1 --2- 172.21.15.102:0/3154595410 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6e8c1b0850 0x7f6e8c1b0ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:42.530 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:42.543 7f6e947f6700 1 --2- 172.21.15.102:0/3154595410 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f6e8c1b12d0 0x7f6e8c1b3c30 secure :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 rx=0x7f6e8800c000 tx=0x7f6e8800e930).stop 2024-04-17T22:31:42.530 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:42.543 7f6e947f6700 1 -- 172.21.15.102:0/3154595410 >> 172.21.15.102:0/3154595410 conn(0x7f6e8c0c6790 msgr2=0x7f6e8c0c79c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:42.530 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:42.544 7f6e947f6700 1 -- 172.21.15.102:0/3154595410 shutdown_connections 2024-04-17T22:31:42.531 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:42.544 7f6e947f6700 1 -- 172.21.15.102:0/3154595410 wait complete. 2024-04-17T22:31:42.531 INFO:teuthology.orchestra.run.smithi102.stderr:pool 'cephfs_data' removed 2024-04-17T22:31:42.545 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-17T22:31:42.677 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:42.689 7f58838d9700 1 Processor -- start 2024-04-17T22:31:42.677 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:42.689 7f58838d9700 1 -- start start 2024-04-17T22:31:42.677 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:42.690 7f58838d9700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f587c10e240 0x7f587c10e690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:42.677 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:42.690 7f58838d9700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f587c0f1f30 0x7f587c1ab4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:42.677 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:42.690 7f58838d9700 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f587c0f2380 con 0x7f587c10ecc0 2024-04-17T22:31:42.678 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:42.690 7f58838d9700 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f587c0f9f10 con 0x7f587c0f1f30 2024-04-17T22:31:42.678 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:42.690 7f58838d9700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f587c0f9030 con 0x7f587c10e240 2024-04-17T22:31:42.678 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:42.690 7f587bfff700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f587c10e240 0x7f587c10e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:42.678 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:42.690 7f587bfff700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f587c10e240 0x7f587c10e690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:40402/0 (socket says 172.21.15.102:40402) 2024-04-17T22:31:42.678 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:42.690 7f587bfff700 1 -- 172.21.15.102:0/3032114473 learned_addr learned my addr 172.21.15.102:0/3032114473 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:31:42.678 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:42.690 7f5880ed5700 1 --2- 172.21.15.102:0/3032114473 >> v2:172.21.15.102:3300/0 conn(0x7f587c0f1f30 0x7f587c1ab4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:42.678 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:42.690 7f587bfff700 1 -- 172.21.15.102:0/3032114473 >> v2:172.21.15.102:3300/0 conn(0x7f587c0f1f30 msgr2=0x7f587c1ab4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:31:42.678 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:42.690 7f587bfff700 1 --2- 172.21.15.102:0/3032114473 >> v2:172.21.15.102:3300/0 conn(0x7f587c0f1f30 0x7f587c1ab4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:42.678 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:42.690 7f587bfff700 1 -- 172.21.15.102:0/3032114473 >> v1:172.21.15.102:6789/0 conn(0x7f587c10ecc0 legacy=0x7f587c0f1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:42.678 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:42.690 7f587bfff700 1 -- 172.21.15.102:0/3032114473 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f587c0f25a0 con 0x7f587c10e240 2024-04-17T22:31:42.678 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:42.690 7f587bfff700 1 --2- 172.21.15.102:0/3032114473 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f587c10e240 0x7f587c10e690 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 rx=0x7f586c002f70 tx=0x7f586c0099d0).ready entity=mon.1 client_cookie=e3b6bb16d49253b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:42.678 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:42.690 7f587b7fe700 1 -- 172.21.15.102:0/3032114473 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f587c0f25a0 con 0x7f587c10e240 2024-04-17T22:31:42.678 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:42.690 7f587b7fe700 1 -- 172.21.15.102:0/3032114473 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f586c00f510 con 0x7f587c10e240 2024-04-17T22:31:42.679 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:42.690 7f587b7fe700 1 -- 172.21.15.102:0/3032114473 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f586c00f6f0 con 0x7f587c10e240 2024-04-17T22:31:42.679 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:42.690 7f58838d9700 1 -- 172.21.15.102:0/3032114473 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f587c10e240 msgr2=0x7f587c10e690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:42.679 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:42.690 7f58838d9700 1 --2- 172.21.15.102:0/3032114473 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f587c10e240 0x7f587c10e690 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 rx=0x7f586c002f70 tx=0x7f586c0099d0).stop 2024-04-17T22:31:42.679 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:42.691 7f58838d9700 1 -- 172.21.15.102:0/3032114473 shutdown_connections 2024-04-17T22:31:42.679 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:42.691 7f58838d9700 1 --2- 172.21.15.102:0/3032114473 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f587c10e240 0x7f587c10e690 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:42.679 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:42.691 7f58838d9700 1 --2- 172.21.15.102:0/3032114473 >> v2:172.21.15.102:3300/0 conn(0x7f587c0f1f30 0x7f587c1ab4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:42.679 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:42.691 7f58838d9700 1 -- 172.21.15.102:0/3032114473 >> 172.21.15.102:0/3032114473 conn(0x7f587c1138a0 msgr2=0x7f587c0f4f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:42.679 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:42.691 7f58838d9700 1 -- 172.21.15.102:0/3032114473 shutdown_connections 2024-04-17T22:31:42.679 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:42.691 7f58838d9700 1 -- 172.21.15.102:0/3032114473 wait complete. 2024-04-17T22:31:42.701 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:42.714 7f58838d9700 1 Processor -- start 2024-04-17T22:31:42.702 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:42.714 7f58838d9700 1 -- start start 2024-04-17T22:31:42.702 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:42.714 7f58838d9700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f587c1a6240 0x7f587c1addb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:42.702 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:42.714 7f58838d9700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f587c1ae3e0 0x7f587c1b46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:42.702 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:42.714 7f58838d9700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f587c1ae830 0x7f587c1b4cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:42.702 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:42.714 7f58838d9700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f587c0e2c80 con 0x7f587c1ae830 2024-04-17T22:31:42.702 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:42.714 7f58838d9700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f587c0fa1b0 con 0x7f587c1a6240 2024-04-17T22:31:42.702 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:42.714 7f58838d9700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f587c1a5860 con 0x7f587c1ae3e0 2024-04-17T22:31:42.702 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:42.714 7f587bfff700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f587c1a6240 0x7f587c1addb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:42.702 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:42.714 7f5880ed5700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f587c1ae830 0x7f587c1b4cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:42.702 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:42.714 7f587bfff700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f587c1a6240 0x7f587c1addb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:40418/0 (socket says 172.21.15.102:40418) 2024-04-17T22:31:42.702 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:42.714 7f5880ed5700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f587c1ae830 0x7f587c1b4cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:57642/0 (socket says 172.21.15.102:57642) 2024-04-17T22:31:42.702 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:42.714 7f587bfff700 1 -- 172.21.15.102:0/3752373678 learned_addr learned my addr 172.21.15.102:0/3752373678 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:31:42.703 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:42.714 7f5873fff700 1 --2- 172.21.15.102:0/3752373678 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f587c1ae3e0 0x7f587c1b46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:42.703 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:42.715 7f5880ed5700 1 -- 172.21.15.102:0/3752373678 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f587c1ae3e0 msgr2=0x7f587c1b46a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:42.703 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:42.715 7f5880ed5700 1 --2- 172.21.15.102:0/3752373678 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f587c1ae3e0 0x7f587c1b46a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:42.703 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:42.715 7f5880ed5700 1 -- 172.21.15.102:0/3752373678 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f587c1a6240 msgr2=0x7f587c1addb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:42.703 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:42.715 7f5880ed5700 1 --2- 172.21.15.102:0/3752373678 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f587c1a6240 0x7f587c1addb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:42.705 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:42.715 7f5880ed5700 1 -- 172.21.15.102:0/3752373678 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f587c1b5300 con 0x7f587c1ae830 2024-04-17T22:31:42.705 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:42.715 7f587bfff700 1 --2- 172.21.15.102:0/3752373678 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f587c1a6240 0x7f587c1addb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:42.705 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:42.715 7f5873fff700 1 --2- 172.21.15.102:0/3752373678 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f587c1ae3e0 0x7f587c1b46a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:42.705 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:42.715 7f5880ed5700 1 --2- 172.21.15.102:0/3752373678 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f587c1ae830 0x7f587c1b4cd0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 rx=0x7f587c1a8760 tx=0x7f587400ee30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:42.705 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:42.715 7f5879ffb700 1 -- 172.21.15.102:0/3752373678 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f587c1b5300 con 0x7f587c1ae830 2024-04-17T22:31:42.706 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:42.715 7f5879ffb700 1 -- 172.21.15.102:0/3752373678 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f587400c850 con 0x7f587c1ae830 2024-04-17T22:31:42.706 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:42.715 7f58838d9700 1 -- 172.21.15.102:0/3752373678 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f587c1b5620 con 0x7f587c1ae830 2024-04-17T22:31:42.706 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:42.715 7f5879ffb700 1 -- 172.21.15.102:0/3752373678 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f587400f040 con 0x7f587c1ae830 2024-04-17T22:31:42.706 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:42.715 7f58838d9700 1 -- 172.21.15.102:0/3752373678 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f587c1bc080 con 0x7f587c1ae830 2024-04-17T22:31:42.706 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:42.715 7f5879ffb700 1 -- 172.21.15.102:0/3752373678 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7f5874022060 con 0x7f587c1ae830 2024-04-17T22:31:42.706 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:42.715 7f5879ffb700 1 -- 172.21.15.102:0/3752373678 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 2417+0+0 (secure 0 0 0) 0x7f5874010040 con 0x7f587c1ae830 2024-04-17T22:31:42.706 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:42.716 7f58838d9700 1 -- 172.21.15.102:0/3752373678 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f587c0f0800 con 0x7f587c1ae830 2024-04-17T22:31:42.707 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:42.719 7f5879ffb700 1 -- 172.21.15.102:0/3752373678 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+110455 (secure 0 0 0) 0x7f587400c640 con 0x7f587c1ae830 2024-04-17T22:31:42.810 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:42.824 7f58838d9700 1 -- 172.21.15.102:0/3752373678 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f587c0efed0 con 0x7f587c1ae830 2024-04-17T22:31:42.813 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:42.827 7f5879ffb700 1 -- 172.21.15.102:0/3752373678 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v3) v1 ==== 104+0+0 (secure 0 0 0) 0x7f587c114420 con 0x7f587c1ae830 2024-04-17T22:31:42.815 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:42.828 7f58838d9700 1 -- 172.21.15.102:0/3752373678 >> v1:172.21.15.102:6816/42360 conn(0x7f58600145a0 legacy=0x7f5860016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:42.815 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:42.828 7f58838d9700 1 -- 172.21.15.102:0/3752373678 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f587c1ae830 msgr2=0x7f587c1b4cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:42.815 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:42.828 7f58838d9700 1 --2- 172.21.15.102:0/3752373678 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f587c1ae830 0x7f587c1b4cd0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 rx=0x7f587c1a8760 tx=0x7f587400ee30).stop 2024-04-17T22:31:42.815 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:42.828 7f58838d9700 1 -- 172.21.15.102:0/3752373678 shutdown_connections 2024-04-17T22:31:42.815 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:42.828 7f58838d9700 1 --2- 172.21.15.102:0/3752373678 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f587c1ae830 0x7f587c1b4cd0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:42.815 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:42.828 7f58838d9700 1 --2- 172.21.15.102:0/3752373678 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f587c1a6240 0x7f587c1addb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:42.815 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:42.828 7f58838d9700 1 --2- 172.21.15.102:0/3752373678 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f587c1ae3e0 0x7f587c1b46a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:42.815 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:42.828 7f58838d9700 1 -- 172.21.15.102:0/3752373678 >> 172.21.15.102:0/3752373678 conn(0x7f587c1a7c50 msgr2=0x7f587c0c6600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:42.815 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:42.828 7f58838d9700 1 -- 172.21.15.102:0/3752373678 shutdown_connections 2024-04-17T22:31:42.815 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:42.828 7f58838d9700 1 -- 172.21.15.102:0/3752373678 wait complete. 2024-04-17T22:31:42.825 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_newops.TestNewOps.test_newops_getvxattr' 2024-04-17T22:31:42.952 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:42.963 7fd897c12700 1 Processor -- start 2024-04-17T22:31:42.952 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:42.963 7fd897c12700 1 -- start start 2024-04-17T22:31:42.952 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:42.964 7fd897c12700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd89010e260 0x7fd89010e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:42.952 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:42.964 7fd897c12700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fd89010ece0 0x7fd8900f1750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:42.952 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:42.964 7fd897c12700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd8900f1d80 0x7fd8901aa4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:42.953 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:42.964 7fd897c12700 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8900f9f30 con 0x7fd89010ece0 2024-04-17T22:31:42.953 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:42.964 7fd897c12700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8900f9050 con 0x7fd89010e260 2024-04-17T22:31:42.953 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:42.964 7fd897c12700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd8900fa1d0 con 0x7fd8900f1d80 2024-04-17T22:31:42.953 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:42.964 7fd89520e700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd8900f1d80 0x7fd8901aa4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:42.953 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:42.964 7fd894a0d700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd89010e260 0x7fd89010e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:42.953 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:42.964 7fd88ffff700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fd89010ece0 0x7fd8900f1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:42.953 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:42.964 7fd89520e700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd8900f1d80 0x7fd8901aa4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:48386/0 (socket says 172.21.15.102:48386) 2024-04-17T22:31:42.953 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:42.964 7fd89520e700 1 -- 172.21.15.102:0/3055139397 learned_addr learned my addr 172.21.15.102:0/3055139397 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:31:42.953 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:42.964 7fd894a0d700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd89010e260 0x7fd89010e6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:40420/0 (socket says 172.21.15.102:40420) 2024-04-17T22:31:42.953 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:42.964 7fd89520e700 1 -- 172.21.15.102:0/3055139397 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd89010e260 msgr2=0x7fd89010e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:31:42.953 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:42.964 7fd89520e700 1 --2- 172.21.15.102:0/3055139397 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd89010e260 0x7fd89010e6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:42.953 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:42.964 7fd89520e700 1 -- 172.21.15.102:0/3055139397 >> v2:172.21.15.102:3300/0 conn(0x7fd89010ece0 msgr2=0x7fd8900f1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:31:42.953 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:42.964 7fd89520e700 1 --2- 172.21.15.102:0/3055139397 >> v2:172.21.15.102:3300/0 conn(0x7fd89010ece0 0x7fd8900f1750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:42.954 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:42.964 7fd89520e700 1 -- 172.21.15.102:0/3055139397 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8900f21d0 con 0x7fd8900f1d80 2024-04-17T22:31:42.954 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:42.964 7fd894a0d700 1 --2- 172.21.15.102:0/3055139397 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd89010e260 0x7fd89010e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:42.954 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:42.964 7fd88ffff700 1 --2- 172.21.15.102:0/3055139397 >> v2:172.21.15.102:3300/0 conn(0x7fd89010ece0 0x7fd8900f1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:42.954 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:42.964 7fd89520e700 1 --2- 172.21.15.102:0/3055139397 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd8900f1d80 0x7fd8901aa4f0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 rx=0x7fd8900f4490 tx=0x7fd888009dc0).ready entity=mon.2 client_cookie=8479adb20f109ee2 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:42.954 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:42.965 7fd88f7fe700 1 -- 172.21.15.102:0/3055139397 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd8900f21d0 con 0x7fd8900f1d80 2024-04-17T22:31:42.954 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:42.965 7fd88f7fe700 1 -- 172.21.15.102:0/3055139397 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd88800b8d0 con 0x7fd8900f1d80 2024-04-17T22:31:42.954 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:42.965 7fd88f7fe700 1 -- 172.21.15.102:0/3055139397 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd88800bab0 con 0x7fd8900f1d80 2024-04-17T22:31:42.954 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:42.965 7fd897c12700 1 -- 172.21.15.102:0/3055139397 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd8900f1d80 msgr2=0x7fd8901aa4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:42.954 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:42.965 7fd897c12700 1 --2- 172.21.15.102:0/3055139397 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd8900f1d80 0x7fd8901aa4f0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 rx=0x7fd8900f4490 tx=0x7fd888009dc0).stop 2024-04-17T22:31:42.954 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:42.965 7fd897c12700 1 -- 172.21.15.102:0/3055139397 shutdown_connections 2024-04-17T22:31:42.954 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:42.965 7fd897c12700 1 --2- 172.21.15.102:0/3055139397 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd8900f1d80 0x7fd8901aa4f0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:42.954 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:42.965 7fd897c12700 1 --2- 172.21.15.102:0/3055139397 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd89010e260 0x7fd89010e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:42.954 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:42.965 7fd897c12700 1 --2- 172.21.15.102:0/3055139397 >> v2:172.21.15.102:3300/0 conn(0x7fd89010ece0 0x7fd8900f1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:42.954 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:42.965 7fd897c12700 1 -- 172.21.15.102:0/3055139397 >> 172.21.15.102:0/3055139397 conn(0x7fd8901138c0 msgr2=0x7fd8900f4f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:42.955 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:42.965 7fd897c12700 1 -- 172.21.15.102:0/3055139397 shutdown_connections 2024-04-17T22:31:42.955 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:42.965 7fd897c12700 1 -- 172.21.15.102:0/3055139397 wait complete. 2024-04-17T22:31:42.974 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:42.988 7fd897c12700 1 Processor -- start 2024-04-17T22:31:42.974 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:42.988 7fd897c12700 1 -- start start 2024-04-17T22:31:42.977 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:42.989 7fd897c12700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd8900dfc40 0x7fd8901ad0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:42.977 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:42.989 7fd897c12700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd8901ad6d0 0x7fd8901b3a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:42.977 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:42.989 7fd897c12700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd8901adb20 0x7fd8901b40a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:42.977 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:42.989 7fd897c12700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd89008f2e0 con 0x7fd8901ad6d0 2024-04-17T22:31:42.977 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:42.989 7fd897c12700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8900de1f0 con 0x7fd8900dfc40 2024-04-17T22:31:42.977 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:42.989 7fd897c12700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd8900e0090 con 0x7fd8901adb20 2024-04-17T22:31:42.977 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:42.989 7fd88ffff700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd8901ad6d0 0x7fd8901b3a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:42.978 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:42.989 7fd894a0d700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd8900dfc40 0x7fd8901ad0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:42.978 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:42.989 7fd89520e700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd8901adb20 0x7fd8901b40a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:42.978 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:42.989 7fd88ffff700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd8901ad6d0 0x7fd8901b3a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:57668/0 (socket says 172.21.15.102:57668) 2024-04-17T22:31:42.978 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:42.989 7fd88ffff700 1 -- 172.21.15.102:0/2478429269 learned_addr learned my addr 172.21.15.102:0/2478429269 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:31:42.978 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:42.989 7fd894a0d700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd8900dfc40 0x7fd8901ad0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:40432/0 (socket says 172.21.15.102:40432) 2024-04-17T22:31:42.978 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:42.989 7fd89520e700 1 -- 172.21.15.102:0/2478429269 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd8900dfc40 msgr2=0x7fd8901ad0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:42.978 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:42.989 7fd89520e700 1 --2- 172.21.15.102:0/2478429269 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd8900dfc40 0x7fd8901ad0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:42.978 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:42.989 7fd89520e700 1 -- 172.21.15.102:0/2478429269 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd8901ad6d0 msgr2=0x7fd8901b3a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:42.978 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:42.989 7fd89520e700 1 --2- 172.21.15.102:0/2478429269 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd8901ad6d0 0x7fd8901b3a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:42.978 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:42.989 7fd89520e700 1 -- 172.21.15.102:0/2478429269 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8901b46d0 con 0x7fd8901adb20 2024-04-17T22:31:42.978 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:42.989 7fd88ffff700 1 --2- 172.21.15.102:0/2478429269 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd8901ad6d0 0x7fd8901b3a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:42.978 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:42.989 7fd89520e700 1 --2- 172.21.15.102:0/2478429269 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd8901adb20 0x7fd8901b40a0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 rx=0x7fd8900c54f0 tx=0x7fd888006c40).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:42.979 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:42.989 7fd88dffb700 1 -- 172.21.15.102:0/2478429269 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd8901b46d0 con 0x7fd8901adb20 2024-04-17T22:31:42.979 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:42.989 7fd88dffb700 1 -- 172.21.15.102:0/2478429269 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd888011980 con 0x7fd8901adb20 2024-04-17T22:31:42.979 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:42.989 7fd897c12700 1 -- 172.21.15.102:0/2478429269 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8901b4990 con 0x7fd8901adb20 2024-04-17T22:31:42.979 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:42.989 7fd88dffb700 1 -- 172.21.15.102:0/2478429269 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd888011d50 con 0x7fd8901adb20 2024-04-17T22:31:42.979 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:42.989 7fd897c12700 1 -- 172.21.15.102:0/2478429269 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8901bb3c0 con 0x7fd8901adb20 2024-04-17T22:31:42.979 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:42.990 7fd88dffb700 1 -- 172.21.15.102:0/2478429269 <== mon.2 v2:172.21.15.102:3302/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7fd88801c3e0 con 0x7fd8901adb20 2024-04-17T22:31:42.979 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:42.990 7fd88dffb700 1 -- 172.21.15.102:0/2478429269 <== mon.2 v2:172.21.15.102:3302/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 2417+0+0 (secure 0 0 0) 0x7fd888021d50 con 0x7fd8901adb20 2024-04-17T22:31:42.979 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:42.990 7fd897c12700 1 -- 172.21.15.102:0/2478429269 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8900f0820 con 0x7fd8901adb20 2024-04-17T22:31:42.979 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:42.994 7fd88dffb700 1 -- 172.21.15.102:0/2478429269 <== mon.2 v2:172.21.15.102:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+110455 (secure 0 0 0) 0x7fd888011770 con 0x7fd8901adb20 2024-04-17T22:31:43.118 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:43.132 7fd897c12700 1 -- 172.21.15.102:0/2478429269 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_newops.TestNewOps.test_newops_getvxattr"]} v 0) v1 -- 0x7fd8901ae1b0 con 0x7fd8901adb20 2024-04-17T22:31:43.537 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:43.551 7fd88dffb700 1 -- 172.21.15.102:0/2478429269 <== mon.2 v2:172.21.15.102:3302/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_newops.TestNewOps.test_newops_getvxattr"]}]=0 v95) v1 ==== 136+0+0 (secure 0 0 0) 0x7fd88800b6c0 con 0x7fd8901adb20 2024-04-17T22:31:43.539 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:43.553 7fd897c12700 1 -- 172.21.15.102:0/2478429269 >> v1:172.21.15.102:6816/42360 conn(0x7fd8780145a0 legacy=0x7fd878016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:43.539 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:43.553 7fd897c12700 1 -- 172.21.15.102:0/2478429269 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd8901adb20 msgr2=0x7fd8901b40a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:43.539 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:43.553 7fd897c12700 1 --2- 172.21.15.102:0/2478429269 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd8901adb20 0x7fd8901b40a0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 rx=0x7fd8900c54f0 tx=0x7fd888006c40).stop 2024-04-17T22:31:43.539 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:43.553 7fd897c12700 1 -- 172.21.15.102:0/2478429269 shutdown_connections 2024-04-17T22:31:43.539 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:43.553 7fd897c12700 1 --2- 172.21.15.102:0/2478429269 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd8901adb20 0x7fd8901b40a0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:43.539 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:43.553 7fd897c12700 1 --2- 172.21.15.102:0/2478429269 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd8900dfc40 0x7fd8901ad0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:43.539 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:43.553 7fd897c12700 1 --2- 172.21.15.102:0/2478429269 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd8901ad6d0 0x7fd8901b3a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:43.539 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:43.553 7fd897c12700 1 -- 172.21.15.102:0/2478429269 >> 172.21.15.102:0/2478429269 conn(0x7fd8900c5af0 msgr2=0x7fd8900d6980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:43.540 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:43.553 7fd897c12700 1 -- 172.21.15.102:0/2478429269 shutdown_connections 2024-04-17T22:31:43.540 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:43.553 7fd897c12700 1 -- 172.21.15.102:0/2478429269 wait complete. 2024-04-17T22:31:43.550 INFO:tasks.cephfs_test_runner:For nautilus it will crash the MDSs when receive unknown OPs, as a workaround ... ok 2024-04-17T22:31:43.551 INFO:tasks.cephfs_test_runner: 2024-04-17T22:31:43.551 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-04-17T22:31:43.551 INFO:tasks.cephfs_test_runner:Ran 1 test in 35.598s 2024-04-17T22:31:43.551 INFO:tasks.cephfs_test_runner: 2024-04-17T22:31:43.551 INFO:tasks.cephfs_test_runner:OK 2024-04-17T22:31:43.551 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2024-04-17T22:31:43.563 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-04-17T22:31:43.601 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-04-17T22:31:43.602 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-04-17T22:31:43.602 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T22:31:43.625 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16251168k,nr_inodes=4062792,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=45,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4000 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3253768k,mode=700,uid=1000,gid=1285 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 2024-04-17T22:31:43.625 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T22:31:43.625 DEBUG:teuthology.orchestra.run.smithi190:> ip netns list 2024-04-17T22:31:43.686 INFO:teuthology.orchestra.run.smithi190.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-17T22:31:43.686 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T22:31:43.686 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-17T22:31:43.768 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T22:31:43.769 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip link delete ceph-brx 2024-04-17T22:31:43.869 DEBUG:teuthology.run_tasks:Unwinding manager install.upgrade 2024-04-17T22:31:43.879 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-04-17T22:31:43.911 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-04-17T22:31:43.911 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T22:31:43.911 DEBUG:teuthology.orchestra.run.smithi190:> ip netns list 2024-04-17T22:31:43.931 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T22:31:43.932 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip link delete ceph-brx 2024-04-17T22:31:44.003 INFO:teuthology.orchestra.run.smithi190.stderr:Cannot find device "ceph-brx" 2024-04-17T22:31:44.004 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-17T22:31:44.005 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-04-17T22:31:44.015 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-04-17T22:31:44.015 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-04-17T22:31:44.015 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T22:31:44.064 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16251168k,nr_inodes=4062792,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=45,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4000 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3253768k,mode=700,uid=1000,gid=1285 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 2024-04-17T22:31:44.064 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T22:31:44.064 DEBUG:teuthology.orchestra.run.smithi190:> ip netns list 2024-04-17T22:31:44.122 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T22:31:44.122 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip link delete ceph-brx 2024-04-17T22:31:44.190 INFO:teuthology.orchestra.run.smithi190.stderr:Cannot find device "ceph-brx" 2024-04-17T22:31:44.191 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-17T22:31:44.191 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-04-17T22:31:44.203 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-17T22:31:44.203 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-17T22:31:44.343 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.355 7f1dbb05d700 1 Processor -- start 2024-04-17T22:31:44.343 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.355 7f1dbb05d700 1 -- start start 2024-04-17T22:31:44.343 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.355 7f1dbb05d700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f1db410e260 0x7f1db410e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:44.343 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.355 7f1dbb05d700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f1db410ece0 0x7f1db40f1750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:44.343 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.355 7f1dbb05d700 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1db40f21d0 con 0x7f1db40f1d80 2024-04-17T22:31:44.343 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.355 7f1dbb05d700 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1db40f9f30 con 0x7f1db410ece0 2024-04-17T22:31:44.343 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.355 7f1dbb05d700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1db40f9050 con 0x7f1db410e260 2024-04-17T22:31:44.344 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.355 7f1db37fe700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f1db410e260 0x7f1db410e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:44.344 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.355 7f1db2ffd700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f1db410ece0 0x7f1db40f1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:44.344 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.355 7f1db37fe700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f1db410e260 0x7f1db410e6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:34448/0 (socket says 172.21.15.102:34448) 2024-04-17T22:31:44.344 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.355 7f1db2ffd700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f1db410ece0 0x7f1db40f1750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:34700/0 (socket says 172.21.15.102:34700) 2024-04-17T22:31:44.344 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.355 7f1db3fff700 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f1db40f1d80 0x7f1db41aa4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:47848/0 (socket says 172.21.15.102:47848) 2024-04-17T22:31:44.344 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.355 7f1db37fe700 1 -- 172.21.15.102:0/3327809944 learned_addr learned my addr 172.21.15.102:0/3327809944 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:31:44.344 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.356 7f1db37fe700 1 -- 172.21.15.102:0/3327809944 >> v2:172.21.15.102:3300/0 conn(0x7f1db410ece0 msgr2=0x7f1db40f1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:31:44.344 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.356 7f1db37fe700 1 --2- 172.21.15.102:0/3327809944 >> v2:172.21.15.102:3300/0 conn(0x7f1db410ece0 0x7f1db40f1750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:44.344 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.356 7f1db37fe700 1 -- 172.21.15.102:0/3327809944 >> v1:172.21.15.102:6789/0 conn(0x7f1db40f1d80 legacy=0x7f1db41aa4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:44.344 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.356 7f1db27fc700 1 -- 172.21.15.102:0/3327809944 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1168164328 0 0) 0x7f1dac001490 con 0x7f1db40f1d80 2024-04-17T22:31:44.344 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.356 7f1db37fe700 1 -- 172.21.15.102:0/3327809944 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1db40f23f0 con 0x7f1db410e260 2024-04-17T22:31:44.344 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.356 7f1db2ffd700 1 --2- 172.21.15.102:0/3327809944 >> v2:172.21.15.102:3300/0 conn(0x7f1db410ece0 0x7f1db40f1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:44.344 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.356 7f1db37fe700 1 --2- 172.21.15.102:0/3327809944 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f1db410e260 0x7f1db410e6b0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 rx=0x7f1da8002f70 tx=0x7f1da80099d0).ready entity=mon.2 client_cookie=b70320d080b4f6af server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:44.344 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.356 7f1db27fc700 1 -- 172.21.15.102:0/3327809944 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f1db40f23f0 con 0x7f1db410e260 2024-04-17T22:31:44.345 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.356 7f1db27fc700 1 -- 172.21.15.102:0/3327809944 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1da800f510 con 0x7f1db410e260 2024-04-17T22:31:44.345 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.356 7f1db27fc700 1 -- 172.21.15.102:0/3327809944 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f1da800f6f0 con 0x7f1db410e260 2024-04-17T22:31:44.345 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.356 7f1dbb05d700 1 -- 172.21.15.102:0/3327809944 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f1db410e260 msgr2=0x7f1db410e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:44.345 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.356 7f1dbb05d700 1 --2- 172.21.15.102:0/3327809944 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f1db410e260 0x7f1db410e6b0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 rx=0x7f1da8002f70 tx=0x7f1da80099d0).stop 2024-04-17T22:31:44.345 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.357 7f1dbb05d700 1 -- 172.21.15.102:0/3327809944 shutdown_connections 2024-04-17T22:31:44.345 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.357 7f1dbb05d700 1 --2- 172.21.15.102:0/3327809944 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f1db410e260 0x7f1db410e6b0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:44.345 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.357 7f1dbb05d700 1 --2- 172.21.15.102:0/3327809944 >> v2:172.21.15.102:3300/0 conn(0x7f1db410ece0 0x7f1db40f1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:44.345 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.357 7f1dbb05d700 1 -- 172.21.15.102:0/3327809944 >> 172.21.15.102:0/3327809944 conn(0x7f1db41138c0 msgr2=0x7f1db40f4f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:44.345 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.357 7f1dbb05d700 1 -- 172.21.15.102:0/3327809944 shutdown_connections 2024-04-17T22:31:44.345 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.357 7f1dbb05d700 1 -- 172.21.15.102:0/3327809944 wait complete. 2024-04-17T22:31:44.367 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.379 7f1dbb05d700 1 Processor -- start 2024-04-17T22:31:44.367 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.379 7f1dbb05d700 1 -- start start 2024-04-17T22:31:44.367 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.380 7f1dbb05d700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f1db40d5f40 0x7f1db40d6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:44.367 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.380 7f1dbb05d700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f1db40d69c0 0x7f1db41b3fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:44.368 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.380 7f1dbb05d700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1db41b45e0 0x7f1db41b4a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:44.368 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.380 7f1dbb05d700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1db40fa1d0 con 0x7f1db41b45e0 2024-04-17T22:31:44.368 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.380 7f1dbb05d700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1db41b5060 con 0x7f1db40d5f40 2024-04-17T22:31:44.368 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.380 7f1dbb05d700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1db40c44a0 con 0x7f1db40d69c0 2024-04-17T22:31:44.368 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.380 7f1db2ffd700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f1db40d69c0 0x7f1db41b3fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:44.368 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.380 7f1db37fe700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f1db40d5f40 0x7f1db40d6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:44.368 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.380 7f1db2ffd700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f1db40d69c0 0x7f1db41b3fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:34458/0 (socket says 172.21.15.102:34458) 2024-04-17T22:31:44.368 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.380 7f1db2ffd700 1 -- 172.21.15.102:0/3515830056 learned_addr learned my addr 172.21.15.102:0/3515830056 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:31:44.368 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.380 7f1db3fff700 1 --2- 172.21.15.102:0/3515830056 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1db41b45e0 0x7f1db41b4a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:44.368 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.380 7f1db37fe700 1 -- 172.21.15.102:0/3515830056 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f1db40d69c0 msgr2=0x7f1db41b3fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:44.368 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.380 7f1db37fe700 1 --2- 172.21.15.102:0/3515830056 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f1db40d69c0 0x7f1db41b3fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:44.368 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.380 7f1db37fe700 1 -- 172.21.15.102:0/3515830056 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1db41b45e0 msgr2=0x7f1db41b4a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:44.368 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.380 7f1db37fe700 1 --2- 172.21.15.102:0/3515830056 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1db41b45e0 0x7f1db41b4a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:44.368 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.380 7f1db37fe700 1 -- 172.21.15.102:0/3515830056 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1db40dd990 con 0x7f1db40d5f40 2024-04-17T22:31:44.369 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.380 7f1db2ffd700 1 --2- 172.21.15.102:0/3515830056 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f1db40d69c0 0x7f1db41b3fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:44.369 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.380 7f1db37fe700 1 --2- 172.21.15.102:0/3515830056 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f1db40d5f40 0x7f1db40d6390 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 rx=0x7f1db40d7510 tx=0x7f1da80068c0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:44.369 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.380 7f1db0ff9700 1 -- 172.21.15.102:0/3515830056 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f1db40dd990 con 0x7f1db40d5f40 2024-04-17T22:31:44.369 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.380 7f1db0ff9700 1 -- 172.21.15.102:0/3515830056 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1da8011710 con 0x7f1db40d5f40 2024-04-17T22:31:44.369 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.380 7f1dbb05d700 1 -- 172.21.15.102:0/3515830056 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1db40d7ca0 con 0x7f1db40d5f40 2024-04-17T22:31:44.369 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.380 7f1db0ff9700 1 -- 172.21.15.102:0/3515830056 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f1da8011ae0 con 0x7f1db40d5f40 2024-04-17T22:31:44.369 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.380 7f1dbb05d700 1 -- 172.21.15.102:0/3515830056 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1db41bb720 con 0x7f1db40d5f40 2024-04-17T22:31:44.369 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.381 7f1db0ff9700 1 -- 172.21.15.102:0/3515830056 <== mon.1 v2:172.21.15.102:3301/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7f1da8019030 con 0x7f1db40d5f40 2024-04-17T22:31:44.369 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.381 7f1db0ff9700 1 -- 172.21.15.102:0/3515830056 <== mon.1 v2:172.21.15.102:3301/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 2417+0+0 (secure 0 0 0) 0x7f1da8018040 con 0x7f1db40d5f40 2024-04-17T22:31:44.369 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.382 7f1dbb05d700 1 -- 172.21.15.102:0/3515830056 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1db40f0820 con 0x7f1db40d5f40 2024-04-17T22:31:44.370 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.385 7f1db0ff9700 1 -- 172.21.15.102:0/3515830056 <== mon.1 v2:172.21.15.102:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+110455 (secure 0 0 0) 0x7f1da8011500 con 0x7f1db40d5f40 2024-04-17T22:31:44.471 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.485 7f1dbb05d700 1 -- 172.21.15.102:0/3515830056 --> v1:172.21.15.102:6816/42360 -- command(tid 0: {"prefix": "pg dump", "target": ["mgr", ""], "format": "json"}) v1 -- 0x7f1db40ddce0 con 0x7f1d940145a0 2024-04-17T22:31:44.473 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.487 7f1db0ff9700 1 -- 172.21.15.102:0/3515830056 <== mgr.4104 v1:172.21.15.102:6816/42360 1 ==== command_reply(tid 0: 0 dumped all) v1 ==== 18+0+32422 (unknown 2965378022 0 1847179193) 0x7f1da400bb50 con 0x7f1d940145a0 2024-04-17T22:31:44.473 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-17T22:31:44.474 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.488 7f1dbb05d700 1 -- 172.21.15.102:0/3515830056 >> v1:172.21.15.102:6816/42360 conn(0x7f1d940145a0 legacy=0x7f1d94016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:44.474 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.488 7f1dbb05d700 1 -- 172.21.15.102:0/3515830056 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f1db40d5f40 msgr2=0x7f1db40d6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:44.474 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.488 7f1dbb05d700 1 --2- 172.21.15.102:0/3515830056 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f1db40d5f40 0x7f1db40d6390 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 rx=0x7f1db40d7510 tx=0x7f1da80068c0).stop 2024-04-17T22:31:44.474 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.488 7f1dbb05d700 1 -- 172.21.15.102:0/3515830056 shutdown_connections 2024-04-17T22:31:44.475 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.488 7f1dbb05d700 1 --2- 172.21.15.102:0/3515830056 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1db41b45e0 0x7f1db41b4a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:44.475 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.488 7f1dbb05d700 1 --2- 172.21.15.102:0/3515830056 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f1db40d5f40 0x7f1db40d6390 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:44.475 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.488 7f1dbb05d700 1 --2- 172.21.15.102:0/3515830056 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f1db40d69c0 0x7f1db41b3fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:44.475 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.488 7f1dbb05d700 1 -- 172.21.15.102:0/3515830056 >> 172.21.15.102:0/3515830056 conn(0x7f1db40c8bc0 msgr2=0x7f1db40c9410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:44.475 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.488 7f1dbb05d700 1 -- 172.21.15.102:0/3515830056 shutdown_connections 2024-04-17T22:31:44.475 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.488 7f1dbb05d700 1 -- 172.21.15.102:0/3515830056 wait complete. 2024-04-17T22:31:44.475 INFO:teuthology.orchestra.run.smithi102.stderr:dumped all 2024-04-17T22:31:44.485 INFO:teuthology.orchestra.run.smithi102.stdout:{"pg_ready":true,"pg_map":{"version":109,"stamp":"2024-04-17 22:31:43.222500","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":272,"num_osds":4,"num_per_pool_osds":4,"kb":377487360,"kb_used":4195312,"kb_used_data":968,"kb_used_omap":0,"kb_used_meta":4194304,"kb_avail":373292048,"statfs":{"total":386547056640,"available":382251057152,"internally_reserved":4294967296,"allocated":991232,"data_stored":305326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4294967296},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-2286,"num_objects":-21,"num_object_clones":0,"num_object_copies":-42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-44,"num_write_kb":-13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.309956"},"pg_stats":[{"pgid":"1.5","version":"0'0","reported_seq":"23","reported_epoch":"25","state":"active+clean","last_fresh":"2024-04-17 22:31:29.216143","last_change":"2024-04-17 22:28:59.253977","last_active":"2024-04-17 22:31:29.216143","last_peered":"2024-04-17 22:31:29.216143","last_clean":"2024-04-17 22:31:29.216143","last_became_active":"2024-04-17 22:28:59.253641","last_became_peered":"2024-04-17 22:28:59.253641","last_unstale":"2024-04-17 22:31:29.216143","last_undegraded":"2024-04-17 22:31:29.216143","last_fullsized":"2024-04-17 22:31:29.216143","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:28:58.239187","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:28:58.239187","last_clean_scrub_stamp":"2024-04-17 22:28:58.239187","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":"23","reported_epoch":"25","state":"active+clean","last_fresh":"2024-04-17 22:31:29.420646","last_change":"2024-04-17 22:28:59.255733","last_active":"2024-04-17 22:31:29.420646","last_peered":"2024-04-17 22:31:29.420646","last_clean":"2024-04-17 22:31:29.420646","last_became_active":"2024-04-17 22:28:59.255411","last_became_peered":"2024-04-17 22:28:59.255411","last_unstale":"2024-04-17 22:31:29.420646","last_undegraded":"2024-04-17 22:31:29.420646","last_fullsized":"2024-04-17 22:31:29.420646","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:28:58.239187","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:28:58.239187","last_clean_scrub_stamp":"2024-04-17 22:28:58.239187","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":"23","reported_epoch":"25","state":"active+clean","last_fresh":"2024-04-17 22:31:29.215551","last_change":"2024-04-17 22:28:59.252728","last_active":"2024-04-17 22:31:29.215551","last_peered":"2024-04-17 22:31:29.215551","last_clean":"2024-04-17 22:31:29.215551","last_became_active":"2024-04-17 22:28:59.252345","last_became_peered":"2024-04-17 22:28:59.252345","last_unstale":"2024-04-17 22:31:29.215551","last_undegraded":"2024-04-17 22:31:29.215551","last_fullsized":"2024-04-17 22:31:29.215551","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:28:58.239187","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:28:58.239187","last_clean_scrub_stamp":"2024-04-17 22:28:58.239187","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":"23","reported_epoch":"25","state":"active+clean","last_fresh":"2024-04-17 22:31:29.214261","last_change":"2024-04-17 22:28:59.253392","last_active":"2024-04-17 22:31:29.214261","last_peered":"2024-04-17 22:31:29.214261","last_clean":"2024-04-17 22:31:29.214261","last_became_active":"2024-04-17 22:28:59.253049","last_became_peered":"2024-04-17 22:28:59.253049","last_unstale":"2024-04-17 22:31:29.214261","last_undegraded":"2024-04-17 22:31:29.214261","last_fullsized":"2024-04-17 22:31:29.214261","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:28:58.239187","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:28:58.239187","last_clean_scrub_stamp":"2024-04-17 22:28:58.239187","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":"23","reported_epoch":"25","state":"active+clean","last_fresh":"2024-04-17 22:31:29.214314","last_change":"2024-04-17 22:28:59.254907","last_active":"2024-04-17 22:31:29.214314","last_peered":"2024-04-17 22:31:29.214314","last_clean":"2024-04-17 22:31:29.214314","last_became_active":"2024-04-17 22:28:59.254633","last_became_peered":"2024-04-17 22:28:59.254633","last_unstale":"2024-04-17 22:31:29.214314","last_undegraded":"2024-04-17 22:31:29.214314","last_fullsized":"2024-04-17 22:31:29.214314","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:28:58.239187","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:28:58.239187","last_clean_scrub_stamp":"2024-04-17 22:28:58.239187","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":"23","reported_epoch":"25","state":"active+clean","last_fresh":"2024-04-17 22:31:29.420309","last_change":"2024-04-17 22:28:59.254217","last_active":"2024-04-17 22:31:29.420309","last_peered":"2024-04-17 22:31:29.420309","last_clean":"2024-04-17 22:31:29.420309","last_became_active":"2024-04-17 22:28:59.253877","last_became_peered":"2024-04-17 22:28:59.253877","last_unstale":"2024-04-17 22:31:29.420309","last_undegraded":"2024-04-17 22:31:29.420309","last_fullsized":"2024-04-17 22:31:29.420309","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:28:58.239187","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:28:58.239187","last_clean_scrub_stamp":"2024-04-17 22:28:58.239187","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":"23","reported_epoch":"25","state":"active+clean","last_fresh":"2024-04-17 22:31:29.422721","last_change":"2024-04-17 22:28:59.254830","last_active":"2024-04-17 22:31:29.422721","last_peered":"2024-04-17 22:31:29.422721","last_clean":"2024-04-17 22:31:29.422721","last_became_active":"2024-04-17 22:28:59.254505","last_became_peered":"2024-04-17 22:28:59.254505","last_unstale":"2024-04-17 22:31:29.422721","last_undegraded":"2024-04-17 22:31:29.422721","last_fullsized":"2024-04-17 22:31:29.422721","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:28:58.239187","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:28:58.239187","last_clean_scrub_stamp":"2024-04-17 22:28:58.239187","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":"23","reported_epoch":"25","state":"active+clean","last_fresh":"2024-04-17 22:31:29.216381","last_change":"2024-04-17 22:28:59.254055","last_active":"2024-04-17 22:31:29.216381","last_peered":"2024-04-17 22:31:29.216381","last_clean":"2024-04-17 22:31:29.216381","last_became_active":"2024-04-17 22:28:59.253750","last_became_peered":"2024-04-17 22:28:59.253750","last_unstale":"2024-04-17 22:31:29.216381","last_undegraded":"2024-04-17 22:31:29.216381","last_fullsized":"2024-04-17 22:31:29.216381","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:28:58.239187","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:28:58.239187","last_clean_scrub_stamp":"2024-04-17 22:28:58.239187","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705701,"num_pgs":82,"num_osds":1,"num_per_pool_osds":1,"kb":94371840,"kb_used":1048852,"kb_used_data":260,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93322988,"statfs":{"total":96636764160,"available":95562739712,"internally_reserved":1073741824,"allocated":266240,"data_stored":79117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 17 22:30:59 2024","interfaces":[{"interface":"back","average":{"1min":0.308,"5min":0.30399999999999999,"15min":0.30399999999999999},"min":{"1min":0.20399999999999999,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.30199999999999999,"5min":0.314,"15min":0.316},"min":{"1min":0.21099999999999999,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.44600000000000001,"5min":0.52400000000000002,"15min":0.52400000000000002},"last":0.35499999999999998}]},{"osd":1,"last update":"Wed Apr 17 22:30:59 2024","interfaces":[{"interface":"back","average":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"min":{"1min":0.223,"5min":0.19600000000000001,"15min":0.19600000000000001},"max":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"last":0.378},{"interface":"front","average":{"1min":0.33400000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"min":{"1min":0.249,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.438,"5min":0.55300000000000005,"15min":0.55300000000000005},"last":0.39300000000000002}]},{"osd":2,"last update":"Wed Apr 17 22:30:59 2024","interfaces":[{"interface":"back","average":{"1min":0.33800000000000002,"5min":0.34799999999999998,"15min":0.34999999999999998},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.46200000000000002,"5min":0.50600000000000001,"15min":0.50600000000000001},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.32800000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"min":{"1min":0.23200000000000001,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.46899999999999997,"5min":0.47699999999999998,"15min":0.47699999999999998},"last":0.40400000000000003}]}]},{"osd":2,"up_from":9,"seq":38654705701,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"kb":94371840,"kb_used":1048828,"kb_used_data":244,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323012,"statfs":{"total":96636764160,"available":95562764288,"internally_reserved":1073741824,"allocated":249856,"data_stored":78371,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 17 22:30:58 2024","interfaces":[{"interface":"back","average":{"1min":0.30099999999999999,"5min":0.28499999999999998,"15min":0.28299999999999997},"min":{"1min":0.22600000000000001,"5min":0.112,"15min":0.112},"max":{"1min":0.41499999999999998,"5min":0.46400000000000002,"15min":0.46400000000000002},"last":0.23400000000000001},{"interface":"front","average":{"1min":0.33600000000000002,"5min":0.314,"15min":0.31},"min":{"1min":0.27600000000000002,"5min":0.155,"15min":0.155},"max":{"1min":0.40400000000000003,"5min":0.49199999999999999,"15min":0.49199999999999999},"last":0.27100000000000002}]},{"osd":1,"last update":"Wed Apr 17 22:30:58 2024","interfaces":[{"interface":"back","average":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"min":{"1min":0.214,"5min":0.17899999999999999,"15min":0.17899999999999999},"max":{"1min":0.377,"5min":0.52900000000000003,"15min":0.52900000000000003},"last":0.253},{"interface":"front","average":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"min":{"1min":0.245,"5min":0.192,"15min":0.192},"max":{"1min":0.38800000000000001,"5min":0.47199999999999998,"15min":0.47199999999999998},"last":0.25900000000000001}]},{"osd":3,"last update":"Wed Apr 17 22:30:58 2024","interfaces":[{"interface":"back","average":{"1min":0.30399999999999999,"5min":0.29599999999999999,"15min":0.29499999999999998},"min":{"1min":0.24199999999999999,"5min":0.191,"15min":0.191},"max":{"1min":0.42599999999999999,"5min":0.45600000000000002,"15min":0.45600000000000002},"last":0.245},{"interface":"front","average":{"1min":0.31900000000000001,"5min":0.311,"15min":0.31},"min":{"1min":0.26200000000000001,"5min":0.214,"15min":0.214},"max":{"1min":0.41299999999999998,"5min":0.48699999999999999,"15min":0.48699999999999999},"last":0.28199999999999997}]}]},{"osd":0,"up_from":9,"seq":38654705701,"num_pgs":62,"num_osds":1,"num_per_pool_osds":1,"kb":94371840,"kb_used":1048812,"kb_used_data":228,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323028,"statfs":{"total":96636764160,"available":95562780672,"internally_reserved":1073741824,"allocated":233472,"data_stored":73636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Apr 17 22:30:56 2024","interfaces":[{"interface":"back","average":{"1min":0.248,"5min":0.28299999999999997,"15min":0.28899999999999998},"min":{"1min":0.159,"5min":0.159,"15min":0.159},"max":{"1min":0.33200000000000002,"5min":0.40899999999999997,"15min":0.40899999999999997},"last":0.22900000000000001},{"interface":"front","average":{"1min":0.26000000000000001,"5min":0.29199999999999998,"15min":0.29699999999999999},"min":{"1min":0.189,"5min":0.16600000000000001,"15min":0.16600000000000001},"max":{"1min":0.35399999999999998,"5min":0.42999999999999999,"15min":0.42999999999999999},"last":0.25}]},{"osd":2,"last update":"Wed Apr 17 22:30:56 2024","interfaces":[{"interface":"back","average":{"1min":0.28899999999999998,"5min":0.32100000000000001,"15min":0.32700000000000001},"min":{"1min":0.218,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.39500000000000002,"5min":0.48599999999999999,"15min":0.48599999999999999},"last":0.313},{"interface":"front","average":{"1min":0.27800000000000002,"5min":0.317,"15min":0.32300000000000001},"min":{"1min":0.19400000000000001,"5min":0.19400000000000001,"15min":0.19400000000000001},"max":{"1min":0.371,"5min":0.60699999999999998,"15min":0.60699999999999998},"last":0.30599999999999999}]},{"osd":3,"last update":"Wed Apr 17 22:30:56 2024","interfaces":[{"interface":"back","average":{"1min":0.311,"5min":0.33400000000000002,"15min":0.33800000000000002},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.40600000000000003,"5min":0.505,"15min":0.505},"last":0.33000000000000002},{"interface":"front","average":{"1min":0.28599999999999998,"5min":0.32000000000000001,"15min":0.32600000000000001},"min":{"1min":0.20000000000000001,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.38300000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"last":0.26000000000000001}]}]},{"osd":1,"up_from":9,"seq":38654705700,"num_pgs":73,"num_osds":1,"num_per_pool_osds":1,"kb":94371840,"kb_used":1048820,"kb_used_data":236,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323020,"statfs":{"total":96636764160,"available":95562772480,"internally_reserved":1073741824,"allocated":241664,"data_stored":74202,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 17 22:31:00 2024","interfaces":[{"interface":"back","average":{"1min":0.26900000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"min":{"1min":0.128,"5min":0.128,"15min":0.128},"max":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"last":0.17399999999999999},{"interface":"front","average":{"1min":0.28000000000000003,"5min":0.27600000000000002,"15min":0.27600000000000002},"min":{"1min":0.19400000000000001,"5min":0.191,"15min":0.191},"max":{"1min":0.441,"5min":0.441,"15min":0.441},"last":0.189}]},{"osd":2,"last update":"Wed Apr 17 22:31:00 2024","interfaces":[{"interface":"back","average":{"1min":0.28599999999999998,"5min":0.27900000000000003,"15min":0.27800000000000002},"min":{"1min":0.153,"5min":0.153,"15min":0.153},"max":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"last":0.20999999999999999},{"interface":"front","average":{"1min":0.28599999999999998,"5min":0.29099999999999998,"15min":0.29199999999999998},"min":{"1min":0.18099999999999999,"5min":0.18099999999999999,"15min":0.18099999999999999},"max":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"last":0.20000000000000001}]},{"osd":3,"last update":"Wed Apr 17 22:31:00 2024","interfaces":[{"interface":"back","average":{"1min":0.30499999999999999,"5min":0.29699999999999999,"15min":0.29499999999999998},"min":{"1min":0.23300000000000001,"5min":0.218,"15min":0.218},"max":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"last":0.218},{"interface":"front","average":{"1min":0.309,"5min":0.315,"15min":0.316},"min":{"1min":0.245,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"last":0.23100000000000001}]}]}]}} 2024-04-17T22:31:44.486 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-17T22:31:44.617 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.628 7f3139c65700 1 Processor -- start 2024-04-17T22:31:44.617 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.629 7f3139c65700 1 -- start start 2024-04-17T22:31:44.617 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.629 7f3139c65700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f313410e260 0x7f313410e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:44.617 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.629 7f3139c65700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f313410ece0 0x7f31340f1750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:44.617 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.629 7f3139c65700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f31340f1d80 0x7f31341aa4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:44.617 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.629 7f3139c65700 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31340f9f30 con 0x7f31340f1d80 2024-04-17T22:31:44.617 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.629 7f3139c65700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31340f9050 con 0x7f313410ece0 2024-04-17T22:31:44.617 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.629 7f3139c65700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f31340fa1d0 con 0x7f313410e260 2024-04-17T22:31:44.617 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.629 7f313205c700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f313410ece0 0x7f31340f1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:44.617 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.629 7f313285d700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f313410e260 0x7f313410e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:44.617 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.629 7f313205c700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f313410ece0 0x7f31340f1750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:49708/0 (socket says 172.21.15.102:49708) 2024-04-17T22:31:44.618 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.629 7f313305e700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f31340f1d80 0x7f31341aa4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:44.618 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.629 7f313205c700 1 -- 172.21.15.102:0/4260716357 learned_addr learned my addr 172.21.15.102:0/4260716357 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:31:44.618 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.629 7f313205c700 1 -- 172.21.15.102:0/4260716357 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f313410e260 msgr2=0x7f313410e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:31:44.618 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.629 7f313205c700 1 --2- 172.21.15.102:0/4260716357 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f313410e260 0x7f313410e6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:44.618 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.629 7f313205c700 1 -- 172.21.15.102:0/4260716357 >> v2:172.21.15.102:3300/0 conn(0x7f31340f1d80 msgr2=0x7f31341aa4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:31:44.618 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.629 7f313205c700 1 --2- 172.21.15.102:0/4260716357 >> v2:172.21.15.102:3300/0 conn(0x7f31340f1d80 0x7f31341aa4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:44.618 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.630 7f313205c700 1 -- 172.21.15.102:0/4260716357 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31340f21d0 con 0x7f313410ece0 2024-04-17T22:31:44.618 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.630 7f313285d700 1 --2- 172.21.15.102:0/4260716357 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f313410e260 0x7f313410e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:44.618 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.630 7f313305e700 1 --2- 172.21.15.102:0/4260716357 >> v2:172.21.15.102:3300/0 conn(0x7f31340f1d80 0x7f31341aa4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:44.618 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.630 7f313205c700 1 --2- 172.21.15.102:0/4260716357 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f313410ece0 0x7f31340f1750 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 rx=0x7f3134114380 tx=0x7f3124009a80).ready entity=mon.1 client_cookie=6e70becc9e3b8624 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:44.618 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.630 7f313185b700 1 -- 172.21.15.102:0/4260716357 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f31340f21d0 con 0x7f313410ece0 2024-04-17T22:31:44.618 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.630 7f313185b700 1 -- 172.21.15.102:0/4260716357 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f312400f510 con 0x7f313410ece0 2024-04-17T22:31:44.618 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.630 7f313185b700 1 -- 172.21.15.102:0/4260716357 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f312400f6f0 con 0x7f313410ece0 2024-04-17T22:31:44.619 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.630 7f3139c65700 1 -- 172.21.15.102:0/4260716357 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f313410ece0 msgr2=0x7f31340f1750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:44.619 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.630 7f3139c65700 1 --2- 172.21.15.102:0/4260716357 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f313410ece0 0x7f31340f1750 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 rx=0x7f3134114380 tx=0x7f3124009a80).stop 2024-04-17T22:31:44.619 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.630 7f3139c65700 1 -- 172.21.15.102:0/4260716357 shutdown_connections 2024-04-17T22:31:44.619 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.630 7f3139c65700 1 --2- 172.21.15.102:0/4260716357 >> v2:172.21.15.102:3300/0 conn(0x7f31340f1d80 0x7f31341aa4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:44.619 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.630 7f3139c65700 1 --2- 172.21.15.102:0/4260716357 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f313410e260 0x7f313410e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:44.619 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.630 7f3139c65700 1 --2- 172.21.15.102:0/4260716357 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f313410ece0 0x7f31340f1750 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:44.619 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.630 7f3139c65700 1 -- 172.21.15.102:0/4260716357 >> 172.21.15.102:0/4260716357 conn(0x7f31341138c0 msgr2=0x7f31340f4f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:44.619 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.631 7f3139c65700 1 -- 172.21.15.102:0/4260716357 shutdown_connections 2024-04-17T22:31:44.619 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.631 7f3139c65700 1 -- 172.21.15.102:0/4260716357 wait complete. 2024-04-17T22:31:44.641 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.653 7f3139c65700 1 Processor -- start 2024-04-17T22:31:44.642 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.653 7f3139c65700 1 -- start start 2024-04-17T22:31:44.642 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.653 7f3139c65700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f31340cdc40 0x7f31340d5f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:44.642 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.653 7f3139c65700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f31340d6590 0x7f31341b3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:44.642 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.653 7f3139c65700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f31340d69e0 0x7f31341b4510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:44.642 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.653 7f3139c65700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31341a8aa0 con 0x7f31340cdc40 2024-04-17T22:31:44.642 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.653 7f3139c65700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31341b4b40 con 0x7f31340d6590 2024-04-17T22:31:44.642 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.653 7f3139c65700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f31341b4d70 con 0x7f31340d69e0 2024-04-17T22:31:44.642 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.653 7f313285d700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f31340cdc40 0x7f31340d5f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:44.642 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.653 7f313205c700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f31340d6590 0x7f31341b3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:44.643 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.653 7f313305e700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f31340d69e0 0x7f31341b4510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:44.643 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.654 7f313285d700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f31340cdc40 0x7f31340d5f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:34726/0 (socket says 172.21.15.102:34726) 2024-04-17T22:31:44.643 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.654 7f313285d700 1 -- 172.21.15.102:0/2547691036 learned_addr learned my addr 172.21.15.102:0/2547691036 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:31:44.643 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.654 7f313205c700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f31340d6590 0x7f31341b3ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:49720/0 (socket says 172.21.15.102:49720) 2024-04-17T22:31:44.643 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.654 7f313305e700 1 -- 172.21.15.102:0/2547691036 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f31340d6590 msgr2=0x7f31341b3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:44.643 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.654 7f313305e700 1 --2- 172.21.15.102:0/2547691036 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f31340d6590 0x7f31341b3ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:44.643 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.654 7f313305e700 1 -- 172.21.15.102:0/2547691036 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f31340cdc40 msgr2=0x7f31340d5f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:44.643 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.654 7f313305e700 1 --2- 172.21.15.102:0/2547691036 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f31340cdc40 0x7f31340d5f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:44.643 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.654 7f313305e700 1 -- 172.21.15.102:0/2547691036 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31341bb5d0 con 0x7f31340d69e0 2024-04-17T22:31:44.643 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.654 7f313205c700 1 --2- 172.21.15.102:0/2547691036 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f31340d6590 0x7f31341b3ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:44.643 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.654 7f313285d700 1 --2- 172.21.15.102:0/2547691036 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f31340cdc40 0x7f31340d5f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:44.644 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.654 7f313305e700 1 --2- 172.21.15.102:0/2547691036 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f31340d69e0 0x7f31341b4510 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 rx=0x7f312c00c000 tx=0x7f312c00ed20).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:44.644 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.654 7f3123fff700 1 -- 172.21.15.102:0/2547691036 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f31341bb5d0 con 0x7f31340d69e0 2024-04-17T22:31:44.644 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.654 7f3123fff700 1 -- 172.21.15.102:0/2547691036 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f312c00c830 con 0x7f31340d69e0 2024-04-17T22:31:44.644 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.654 7f3123fff700 1 -- 172.21.15.102:0/2547691036 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f312c00f040 con 0x7f31340d69e0 2024-04-17T22:31:44.644 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.654 7f3139c65700 1 -- 172.21.15.102:0/2547691036 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31341bb7d0 con 0x7f31340d69e0 2024-04-17T22:31:44.644 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.654 7f3139c65700 1 -- 172.21.15.102:0/2547691036 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31341bbbb0 con 0x7f31340d69e0 2024-04-17T22:31:44.644 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.655 7f3123fff700 1 -- 172.21.15.102:0/2547691036 <== mon.2 v2:172.21.15.102:3302/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7f312c022060 con 0x7f31340d69e0 2024-04-17T22:31:44.644 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.655 7f3123fff700 1 -- 172.21.15.102:0/2547691036 <== mon.2 v2:172.21.15.102:3302/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 2417+0+0 (secure 0 0 0) 0x7f312c010040 con 0x7f31340d69e0 2024-04-17T22:31:44.644 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.655 7f3139c65700 1 -- 172.21.15.102:0/2547691036 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31340f0820 con 0x7f31340d69e0 2024-04-17T22:31:44.645 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.658 7f3123fff700 1 -- 172.21.15.102:0/2547691036 <== mon.2 v2:172.21.15.102:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+110455 (secure 0 0 0) 0x7f312c00c620 con 0x7f31340d69e0 2024-04-17T22:31:44.745 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.759 7f3139c65700 1 -- 172.21.15.102:0/2547691036 --> v1:172.21.15.102:6816/42360 -- command(tid 0: {"prefix": "pg dump", "target": ["mgr", ""], "format": "json"}) v1 -- 0x7f313408a300 con 0x7f31100145a0 2024-04-17T22:31:44.746 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.760 7f3123fff700 1 -- 172.21.15.102:0/2547691036 <== mgr.4104 v1:172.21.15.102:6816/42360 1 ==== command_reply(tid 0: 0 dumped all) v1 ==== 18+0+32422 (unknown 2965378022 0 1847179193) 0x7f312800bb50 con 0x7f31100145a0 2024-04-17T22:31:44.746 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-17T22:31:44.747 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.761 7f3139c65700 1 -- 172.21.15.102:0/2547691036 >> v1:172.21.15.102:6816/42360 conn(0x7f31100145a0 legacy=0x7f3110016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:44.748 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.761 7f3139c65700 1 -- 172.21.15.102:0/2547691036 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f31340d69e0 msgr2=0x7f31341b4510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:44.748 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.761 7f3139c65700 1 --2- 172.21.15.102:0/2547691036 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f31340d69e0 0x7f31341b4510 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 rx=0x7f312c00c000 tx=0x7f312c00ed20).stop 2024-04-17T22:31:44.748 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.761 7f3139c65700 1 -- 172.21.15.102:0/2547691036 shutdown_connections 2024-04-17T22:31:44.748 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.761 7f3139c65700 1 --2- 172.21.15.102:0/2547691036 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f31340d69e0 0x7f31341b4510 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:44.748 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.761 7f3139c65700 1 --2- 172.21.15.102:0/2547691036 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f31340cdc40 0x7f31340d5f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:44.748 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.761 7f3139c65700 1 --2- 172.21.15.102:0/2547691036 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f31340d6590 0x7f31341b3ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:44.748 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.761 7f3139c65700 1 -- 172.21.15.102:0/2547691036 >> 172.21.15.102:0/2547691036 conn(0x7f31340c3810 msgr2=0x7f31340dade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:44.748 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.761 7f3139c65700 1 -- 172.21.15.102:0/2547691036 shutdown_connections 2024-04-17T22:31:44.748 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.761 7f3139c65700 1 -- 172.21.15.102:0/2547691036 wait complete. 2024-04-17T22:31:44.749 INFO:teuthology.orchestra.run.smithi102.stderr:dumped all 2024-04-17T22:31:44.758 INFO:teuthology.orchestra.run.smithi102.stdout:{"pg_ready":true,"pg_map":{"version":109,"stamp":"2024-04-17 22:31:43.222500","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":272,"num_osds":4,"num_per_pool_osds":4,"kb":377487360,"kb_used":4195312,"kb_used_data":968,"kb_used_omap":0,"kb_used_meta":4194304,"kb_avail":373292048,"statfs":{"total":386547056640,"available":382251057152,"internally_reserved":4294967296,"allocated":991232,"data_stored":305326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4294967296},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-2286,"num_objects":-21,"num_object_clones":0,"num_object_copies":-42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-44,"num_write_kb":-13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.309956"},"pg_stats":[{"pgid":"1.5","version":"0'0","reported_seq":"23","reported_epoch":"25","state":"active+clean","last_fresh":"2024-04-17 22:31:29.216143","last_change":"2024-04-17 22:28:59.253977","last_active":"2024-04-17 22:31:29.216143","last_peered":"2024-04-17 22:31:29.216143","last_clean":"2024-04-17 22:31:29.216143","last_became_active":"2024-04-17 22:28:59.253641","last_became_peered":"2024-04-17 22:28:59.253641","last_unstale":"2024-04-17 22:31:29.216143","last_undegraded":"2024-04-17 22:31:29.216143","last_fullsized":"2024-04-17 22:31:29.216143","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:28:58.239187","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:28:58.239187","last_clean_scrub_stamp":"2024-04-17 22:28:58.239187","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":"23","reported_epoch":"25","state":"active+clean","last_fresh":"2024-04-17 22:31:29.420646","last_change":"2024-04-17 22:28:59.255733","last_active":"2024-04-17 22:31:29.420646","last_peered":"2024-04-17 22:31:29.420646","last_clean":"2024-04-17 22:31:29.420646","last_became_active":"2024-04-17 22:28:59.255411","last_became_peered":"2024-04-17 22:28:59.255411","last_unstale":"2024-04-17 22:31:29.420646","last_undegraded":"2024-04-17 22:31:29.420646","last_fullsized":"2024-04-17 22:31:29.420646","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:28:58.239187","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:28:58.239187","last_clean_scrub_stamp":"2024-04-17 22:28:58.239187","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":"23","reported_epoch":"25","state":"active+clean","last_fresh":"2024-04-17 22:31:29.215551","last_change":"2024-04-17 22:28:59.252728","last_active":"2024-04-17 22:31:29.215551","last_peered":"2024-04-17 22:31:29.215551","last_clean":"2024-04-17 22:31:29.215551","last_became_active":"2024-04-17 22:28:59.252345","last_became_peered":"2024-04-17 22:28:59.252345","last_unstale":"2024-04-17 22:31:29.215551","last_undegraded":"2024-04-17 22:31:29.215551","last_fullsized":"2024-04-17 22:31:29.215551","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:28:58.239187","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:28:58.239187","last_clean_scrub_stamp":"2024-04-17 22:28:58.239187","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":"23","reported_epoch":"25","state":"active+clean","last_fresh":"2024-04-17 22:31:29.214261","last_change":"2024-04-17 22:28:59.253392","last_active":"2024-04-17 22:31:29.214261","last_peered":"2024-04-17 22:31:29.214261","last_clean":"2024-04-17 22:31:29.214261","last_became_active":"2024-04-17 22:28:59.253049","last_became_peered":"2024-04-17 22:28:59.253049","last_unstale":"2024-04-17 22:31:29.214261","last_undegraded":"2024-04-17 22:31:29.214261","last_fullsized":"2024-04-17 22:31:29.214261","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:28:58.239187","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:28:58.239187","last_clean_scrub_stamp":"2024-04-17 22:28:58.239187","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":"23","reported_epoch":"25","state":"active+clean","last_fresh":"2024-04-17 22:31:29.214314","last_change":"2024-04-17 22:28:59.254907","last_active":"2024-04-17 22:31:29.214314","last_peered":"2024-04-17 22:31:29.214314","last_clean":"2024-04-17 22:31:29.214314","last_became_active":"2024-04-17 22:28:59.254633","last_became_peered":"2024-04-17 22:28:59.254633","last_unstale":"2024-04-17 22:31:29.214314","last_undegraded":"2024-04-17 22:31:29.214314","last_fullsized":"2024-04-17 22:31:29.214314","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:28:58.239187","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:28:58.239187","last_clean_scrub_stamp":"2024-04-17 22:28:58.239187","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":"23","reported_epoch":"25","state":"active+clean","last_fresh":"2024-04-17 22:31:29.420309","last_change":"2024-04-17 22:28:59.254217","last_active":"2024-04-17 22:31:29.420309","last_peered":"2024-04-17 22:31:29.420309","last_clean":"2024-04-17 22:31:29.420309","last_became_active":"2024-04-17 22:28:59.253877","last_became_peered":"2024-04-17 22:28:59.253877","last_unstale":"2024-04-17 22:31:29.420309","last_undegraded":"2024-04-17 22:31:29.420309","last_fullsized":"2024-04-17 22:31:29.420309","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:28:58.239187","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:28:58.239187","last_clean_scrub_stamp":"2024-04-17 22:28:58.239187","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":"23","reported_epoch":"25","state":"active+clean","last_fresh":"2024-04-17 22:31:29.422721","last_change":"2024-04-17 22:28:59.254830","last_active":"2024-04-17 22:31:29.422721","last_peered":"2024-04-17 22:31:29.422721","last_clean":"2024-04-17 22:31:29.422721","last_became_active":"2024-04-17 22:28:59.254505","last_became_peered":"2024-04-17 22:28:59.254505","last_unstale":"2024-04-17 22:31:29.422721","last_undegraded":"2024-04-17 22:31:29.422721","last_fullsized":"2024-04-17 22:31:29.422721","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:28:58.239187","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:28:58.239187","last_clean_scrub_stamp":"2024-04-17 22:28:58.239187","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":"23","reported_epoch":"25","state":"active+clean","last_fresh":"2024-04-17 22:31:29.216381","last_change":"2024-04-17 22:28:59.254055","last_active":"2024-04-17 22:31:29.216381","last_peered":"2024-04-17 22:31:29.216381","last_clean":"2024-04-17 22:31:29.216381","last_became_active":"2024-04-17 22:28:59.253750","last_became_peered":"2024-04-17 22:28:59.253750","last_unstale":"2024-04-17 22:31:29.216381","last_undegraded":"2024-04-17 22:31:29.216381","last_fullsized":"2024-04-17 22:31:29.216381","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:28:58.239187","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:28:58.239187","last_clean_scrub_stamp":"2024-04-17 22:28:58.239187","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705701,"num_pgs":82,"num_osds":1,"num_per_pool_osds":1,"kb":94371840,"kb_used":1048852,"kb_used_data":260,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93322988,"statfs":{"total":96636764160,"available":95562739712,"internally_reserved":1073741824,"allocated":266240,"data_stored":79117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 17 22:30:59 2024","interfaces":[{"interface":"back","average":{"1min":0.308,"5min":0.30399999999999999,"15min":0.30399999999999999},"min":{"1min":0.20399999999999999,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.30199999999999999,"5min":0.314,"15min":0.316},"min":{"1min":0.21099999999999999,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.44600000000000001,"5min":0.52400000000000002,"15min":0.52400000000000002},"last":0.35499999999999998}]},{"osd":1,"last update":"Wed Apr 17 22:30:59 2024","interfaces":[{"interface":"back","average":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"min":{"1min":0.223,"5min":0.19600000000000001,"15min":0.19600000000000001},"max":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"last":0.378},{"interface":"front","average":{"1min":0.33400000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"min":{"1min":0.249,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.438,"5min":0.55300000000000005,"15min":0.55300000000000005},"last":0.39300000000000002}]},{"osd":2,"last update":"Wed Apr 17 22:30:59 2024","interfaces":[{"interface":"back","average":{"1min":0.33800000000000002,"5min":0.34799999999999998,"15min":0.34999999999999998},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.46200000000000002,"5min":0.50600000000000001,"15min":0.50600000000000001},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.32800000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"min":{"1min":0.23200000000000001,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.46899999999999997,"5min":0.47699999999999998,"15min":0.47699999999999998},"last":0.40400000000000003}]}]},{"osd":2,"up_from":9,"seq":38654705701,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"kb":94371840,"kb_used":1048828,"kb_used_data":244,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323012,"statfs":{"total":96636764160,"available":95562764288,"internally_reserved":1073741824,"allocated":249856,"data_stored":78371,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 17 22:30:58 2024","interfaces":[{"interface":"back","average":{"1min":0.30099999999999999,"5min":0.28499999999999998,"15min":0.28299999999999997},"min":{"1min":0.22600000000000001,"5min":0.112,"15min":0.112},"max":{"1min":0.41499999999999998,"5min":0.46400000000000002,"15min":0.46400000000000002},"last":0.23400000000000001},{"interface":"front","average":{"1min":0.33600000000000002,"5min":0.314,"15min":0.31},"min":{"1min":0.27600000000000002,"5min":0.155,"15min":0.155},"max":{"1min":0.40400000000000003,"5min":0.49199999999999999,"15min":0.49199999999999999},"last":0.27100000000000002}]},{"osd":1,"last update":"Wed Apr 17 22:30:58 2024","interfaces":[{"interface":"back","average":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"min":{"1min":0.214,"5min":0.17899999999999999,"15min":0.17899999999999999},"max":{"1min":0.377,"5min":0.52900000000000003,"15min":0.52900000000000003},"last":0.253},{"interface":"front","average":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"min":{"1min":0.245,"5min":0.192,"15min":0.192},"max":{"1min":0.38800000000000001,"5min":0.47199999999999998,"15min":0.47199999999999998},"last":0.25900000000000001}]},{"osd":3,"last update":"Wed Apr 17 22:30:58 2024","interfaces":[{"interface":"back","average":{"1min":0.30399999999999999,"5min":0.29599999999999999,"15min":0.29499999999999998},"min":{"1min":0.24199999999999999,"5min":0.191,"15min":0.191},"max":{"1min":0.42599999999999999,"5min":0.45600000000000002,"15min":0.45600000000000002},"last":0.245},{"interface":"front","average":{"1min":0.31900000000000001,"5min":0.311,"15min":0.31},"min":{"1min":0.26200000000000001,"5min":0.214,"15min":0.214},"max":{"1min":0.41299999999999998,"5min":0.48699999999999999,"15min":0.48699999999999999},"last":0.28199999999999997}]}]},{"osd":0,"up_from":9,"seq":38654705701,"num_pgs":62,"num_osds":1,"num_per_pool_osds":1,"kb":94371840,"kb_used":1048812,"kb_used_data":228,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323028,"statfs":{"total":96636764160,"available":95562780672,"internally_reserved":1073741824,"allocated":233472,"data_stored":73636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Apr 17 22:30:56 2024","interfaces":[{"interface":"back","average":{"1min":0.248,"5min":0.28299999999999997,"15min":0.28899999999999998},"min":{"1min":0.159,"5min":0.159,"15min":0.159},"max":{"1min":0.33200000000000002,"5min":0.40899999999999997,"15min":0.40899999999999997},"last":0.22900000000000001},{"interface":"front","average":{"1min":0.26000000000000001,"5min":0.29199999999999998,"15min":0.29699999999999999},"min":{"1min":0.189,"5min":0.16600000000000001,"15min":0.16600000000000001},"max":{"1min":0.35399999999999998,"5min":0.42999999999999999,"15min":0.42999999999999999},"last":0.25}]},{"osd":2,"last update":"Wed Apr 17 22:30:56 2024","interfaces":[{"interface":"back","average":{"1min":0.28899999999999998,"5min":0.32100000000000001,"15min":0.32700000000000001},"min":{"1min":0.218,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.39500000000000002,"5min":0.48599999999999999,"15min":0.48599999999999999},"last":0.313},{"interface":"front","average":{"1min":0.27800000000000002,"5min":0.317,"15min":0.32300000000000001},"min":{"1min":0.19400000000000001,"5min":0.19400000000000001,"15min":0.19400000000000001},"max":{"1min":0.371,"5min":0.60699999999999998,"15min":0.60699999999999998},"last":0.30599999999999999}]},{"osd":3,"last update":"Wed Apr 17 22:30:56 2024","interfaces":[{"interface":"back","average":{"1min":0.311,"5min":0.33400000000000002,"15min":0.33800000000000002},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.40600000000000003,"5min":0.505,"15min":0.505},"last":0.33000000000000002},{"interface":"front","average":{"1min":0.28599999999999998,"5min":0.32000000000000001,"15min":0.32600000000000001},"min":{"1min":0.20000000000000001,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.38300000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"last":0.26000000000000001}]}]},{"osd":1,"up_from":9,"seq":38654705700,"num_pgs":73,"num_osds":1,"num_per_pool_osds":1,"kb":94371840,"kb_used":1048820,"kb_used_data":236,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323020,"statfs":{"total":96636764160,"available":95562772480,"internally_reserved":1073741824,"allocated":241664,"data_stored":74202,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 17 22:31:00 2024","interfaces":[{"interface":"back","average":{"1min":0.26900000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"min":{"1min":0.128,"5min":0.128,"15min":0.128},"max":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"last":0.17399999999999999},{"interface":"front","average":{"1min":0.28000000000000003,"5min":0.27600000000000002,"15min":0.27600000000000002},"min":{"1min":0.19400000000000001,"5min":0.191,"15min":0.191},"max":{"1min":0.441,"5min":0.441,"15min":0.441},"last":0.189}]},{"osd":2,"last update":"Wed Apr 17 22:31:00 2024","interfaces":[{"interface":"back","average":{"1min":0.28599999999999998,"5min":0.27900000000000003,"15min":0.27800000000000002},"min":{"1min":0.153,"5min":0.153,"15min":0.153},"max":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"last":0.20999999999999999},{"interface":"front","average":{"1min":0.28599999999999998,"5min":0.29099999999999998,"15min":0.29199999999999998},"min":{"1min":0.18099999999999999,"5min":0.18099999999999999,"15min":0.18099999999999999},"max":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"last":0.20000000000000001}]},{"osd":3,"last update":"Wed Apr 17 22:31:00 2024","interfaces":[{"interface":"back","average":{"1min":0.30499999999999999,"5min":0.29699999999999999,"15min":0.29499999999999998},"min":{"1min":0.23300000000000001,"5min":0.218,"15min":0.218},"max":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"last":0.218},{"interface":"front","average":{"1min":0.309,"5min":0.315,"15min":0.316},"min":{"1min":0.245,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"last":0.23100000000000001}]}]}]}} 2024-04-17T22:31:44.759 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-17T22:31:44.761 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-17T22:31:44.887 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.899 7f40d7bd5700 1 Processor -- start 2024-04-17T22:31:44.888 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.899 7f40d7bd5700 1 -- start start 2024-04-17T22:31:44.888 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.899 7f40d7bd5700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f40d010e260 0x7f40d010e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:44.888 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.899 7f40d7bd5700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f40d010ece0 0x7f40d00f1750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:44.888 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.899 7f40d7bd5700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f40d00f1d80 0x7f40d01aa4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:44.888 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.899 7f40d7bd5700 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40d00f9f30 con 0x7f40d010ece0 2024-04-17T22:31:44.888 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.900 7f40d7bd5700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40d00f9050 con 0x7f40d00f1d80 2024-04-17T22:31:44.888 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.900 7f40d7bd5700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f40d00fa1d0 con 0x7f40d010e260 2024-04-17T22:31:44.888 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.900 7f40d49d0700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f40d010e260 0x7f40d010e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:44.888 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.900 7f40cffff700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f40d010ece0 0x7f40d00f1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:44.888 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.900 7f40d51d1700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f40d00f1d80 0x7f40d01aa4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:44.888 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.900 7f40d49d0700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f40d010e260 0x7f40d010e6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:34476/0 (socket says 172.21.15.102:34476) 2024-04-17T22:31:44.889 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.900 7f40d49d0700 1 -- 172.21.15.102:0/1312469581 learned_addr learned my addr 172.21.15.102:0/1312469581 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:31:44.889 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.900 7f40cffff700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f40d010ece0 0x7f40d00f1750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:34734/0 (socket says 172.21.15.102:34734) 2024-04-17T22:31:44.889 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.900 7f40d49d0700 1 -- 172.21.15.102:0/1312469581 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f40d00f1d80 msgr2=0x7f40d01aa4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:31:44.889 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.900 7f40d49d0700 1 --2- 172.21.15.102:0/1312469581 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f40d00f1d80 0x7f40d01aa4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:44.889 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.900 7f40d49d0700 1 -- 172.21.15.102:0/1312469581 >> v2:172.21.15.102:3300/0 conn(0x7f40d010ece0 msgr2=0x7f40d00f1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:31:44.889 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.900 7f40d49d0700 1 --2- 172.21.15.102:0/1312469581 >> v2:172.21.15.102:3300/0 conn(0x7f40d010ece0 0x7f40d00f1750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:44.889 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.900 7f40d49d0700 1 -- 172.21.15.102:0/1312469581 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40d00f21d0 con 0x7f40d010e260 2024-04-17T22:31:44.889 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.900 7f40cffff700 1 --2- 172.21.15.102:0/1312469581 >> v2:172.21.15.102:3300/0 conn(0x7f40d010ece0 0x7f40d00f1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:44.889 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.900 7f40d51d1700 1 --2- 172.21.15.102:0/1312469581 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f40d00f1d80 0x7f40d01aa4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:44.889 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.900 7f40d49d0700 1 --2- 172.21.15.102:0/1312469581 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f40d010e260 0x7f40d010e6b0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 rx=0x7f40d00f4490 tx=0x7f40c4009a30).ready entity=mon.2 client_cookie=1ecbf2c1442d91ad server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:44.889 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.900 7f40cf7fe700 1 -- 172.21.15.102:0/1312469581 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f40d00f21d0 con 0x7f40d010e260 2024-04-17T22:31:44.889 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.900 7f40cf7fe700 1 -- 172.21.15.102:0/1312469581 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40c400f510 con 0x7f40d010e260 2024-04-17T22:31:44.889 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.900 7f40cf7fe700 1 -- 172.21.15.102:0/1312469581 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f40c400f6f0 con 0x7f40d010e260 2024-04-17T22:31:44.889 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.900 7f40d7bd5700 1 -- 172.21.15.102:0/1312469581 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f40d010e260 msgr2=0x7f40d010e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:44.890 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.900 7f40d7bd5700 1 --2- 172.21.15.102:0/1312469581 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f40d010e260 0x7f40d010e6b0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 rx=0x7f40d00f4490 tx=0x7f40c4009a30).stop 2024-04-17T22:31:44.890 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.901 7f40d7bd5700 1 -- 172.21.15.102:0/1312469581 shutdown_connections 2024-04-17T22:31:44.890 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.901 7f40d7bd5700 1 --2- 172.21.15.102:0/1312469581 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f40d00f1d80 0x7f40d01aa4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:44.890 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.901 7f40d7bd5700 1 --2- 172.21.15.102:0/1312469581 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f40d010e260 0x7f40d010e6b0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:44.890 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.901 7f40d7bd5700 1 --2- 172.21.15.102:0/1312469581 >> v2:172.21.15.102:3300/0 conn(0x7f40d010ece0 0x7f40d00f1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:44.890 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.901 7f40d7bd5700 1 -- 172.21.15.102:0/1312469581 >> 172.21.15.102:0/1312469581 conn(0x7f40d01138c0 msgr2=0x7f40d00f4f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:44.890 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.901 7f40d7bd5700 1 -- 172.21.15.102:0/1312469581 shutdown_connections 2024-04-17T22:31:44.890 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.901 7f40d7bd5700 1 -- 172.21.15.102:0/1312469581 wait complete. 2024-04-17T22:31:44.909 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.924 7f40d7bd5700 1 Processor -- start 2024-04-17T22:31:44.910 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.924 7f40d7bd5700 1 -- start start 2024-04-17T22:31:44.912 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.924 7f40d7bd5700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f40d00ca770 0x7f40d00cb950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:44.912 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.924 7f40d7bd5700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f40d00cbf80 0x7f40d00d5000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:44.912 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.924 7f40d7bd5700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f40d00cc3d0 0x7f40d00d5630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:44.912 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.924 7f40d7bd5700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40d008f2e0 con 0x7f40d00ca770 2024-04-17T22:31:44.913 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.924 7f40d7bd5700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40d008ed60 con 0x7f40d00cbf80 2024-04-17T22:31:44.913 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.924 7f40d7bd5700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f40d01a8960 con 0x7f40d00cc3d0 2024-04-17T22:31:44.913 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.924 7f40d49d0700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f40d00ca770 0x7f40d00cb950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:44.913 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.924 7f40cffff700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f40d00cbf80 0x7f40d00d5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:44.913 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.924 7f40d49d0700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f40d00ca770 0x7f40d00cb950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:34750/0 (socket says 172.21.15.102:34750) 2024-04-17T22:31:44.913 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.924 7f40d49d0700 1 -- 172.21.15.102:0/4146582996 learned_addr learned my addr 172.21.15.102:0/4146582996 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:31:44.913 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.924 7f40d51d1700 1 --2- 172.21.15.102:0/4146582996 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f40d00cc3d0 0x7f40d00d5630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:44.913 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.925 7f40d51d1700 1 -- 172.21.15.102:0/4146582996 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f40d00cbf80 msgr2=0x7f40d00d5000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:44.913 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.925 7f40d51d1700 1 --2- 172.21.15.102:0/4146582996 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f40d00cbf80 0x7f40d00d5000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:44.913 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.925 7f40d51d1700 1 -- 172.21.15.102:0/4146582996 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f40d00ca770 msgr2=0x7f40d00cb950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:44.913 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.925 7f40d51d1700 1 --2- 172.21.15.102:0/4146582996 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f40d00ca770 0x7f40d00cb950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:44.913 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.925 7f40d51d1700 1 -- 172.21.15.102:0/4146582996 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40d00caf90 con 0x7f40d00cc3d0 2024-04-17T22:31:44.913 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.925 7f40cffff700 1 --2- 172.21.15.102:0/4146582996 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f40d00cbf80 0x7f40d00d5000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:44.913 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.925 7f40d49d0700 1 --2- 172.21.15.102:0/4146582996 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f40d00ca770 0x7f40d00cb950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:44.914 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.925 7f40d51d1700 1 --2- 172.21.15.102:0/4146582996 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f40d00cc3d0 0x7f40d00d5630 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 rx=0x7f40d00d7620 tx=0x7f40c800ed50).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:44.914 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.925 7f40cdffb700 1 -- 172.21.15.102:0/4146582996 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f40d00caf90 con 0x7f40d00cc3d0 2024-04-17T22:31:44.914 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.925 7f40cdffb700 1 -- 172.21.15.102:0/4146582996 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40c800c830 con 0x7f40d00cc3d0 2024-04-17T22:31:44.914 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.925 7f40d7bd5700 1 -- 172.21.15.102:0/4146582996 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40d00d5c60 con 0x7f40d00cc3d0 2024-04-17T22:31:44.914 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.925 7f40cdffb700 1 -- 172.21.15.102:0/4146582996 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f40c800f040 con 0x7f40d00cc3d0 2024-04-17T22:31:44.914 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.925 7f40d7bd5700 1 -- 172.21.15.102:0/4146582996 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40d01bb820 con 0x7f40d00cc3d0 2024-04-17T22:31:44.914 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.925 7f40cdffb700 1 -- 172.21.15.102:0/4146582996 <== mon.2 v2:172.21.15.102:3302/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7f40c8022060 con 0x7f40d00cc3d0 2024-04-17T22:31:44.914 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.925 7f40cdffb700 1 -- 172.21.15.102:0/4146582996 <== mon.2 v2:172.21.15.102:3302/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 2417+0+0 (secure 0 0 0) 0x7f40c8010040 con 0x7f40d00cc3d0 2024-04-17T22:31:44.914 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.926 7f40d7bd5700 1 -- 172.21.15.102:0/4146582996 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40d00f0820 con 0x7f40d00cc3d0 2024-04-17T22:31:44.915 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:44.929 7f40cdffb700 1 -- 172.21.15.102:0/4146582996 <== mon.2 v2:172.21.15.102:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+110455 (secure 0 0 0) 0x7f40c800c620 con 0x7f40d00cc3d0 2024-04-17T22:31:45.016 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:45.030 7f40d7bd5700 1 -- 172.21.15.102:0/4146582996 --> v1:172.21.15.102:6816/42360 -- command(tid 0: {"prefix": "pg dump", "target": ["mgr", ""], "format": "json"}) v1 -- 0x7f40d008a300 con 0x7f40b80145a0 2024-04-17T22:31:45.017 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:45.031 7f40cdffb700 1 -- 172.21.15.102:0/4146582996 <== mgr.4104 v1:172.21.15.102:6816/42360 1 ==== command_reply(tid 0: 0 dumped all) v1 ==== 18+0+32422 (unknown 2965378022 0 1847179193) 0x7f40c4006c50 con 0x7f40b80145a0 2024-04-17T22:31:45.018 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-17T22:31:45.019 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:45.033 7f40d7bd5700 1 -- 172.21.15.102:0/4146582996 >> v1:172.21.15.102:6816/42360 conn(0x7f40b80145a0 legacy=0x7f40b8016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:45.019 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:45.033 7f40d7bd5700 1 -- 172.21.15.102:0/4146582996 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f40d00cc3d0 msgr2=0x7f40d00d5630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:45.019 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:45.033 7f40d7bd5700 1 --2- 172.21.15.102:0/4146582996 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f40d00cc3d0 0x7f40d00d5630 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 rx=0x7f40d00d7620 tx=0x7f40c800ed50).stop 2024-04-17T22:31:45.019 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:45.033 7f40d7bd5700 1 -- 172.21.15.102:0/4146582996 shutdown_connections 2024-04-17T22:31:45.019 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:45.033 7f40d7bd5700 1 --2- 172.21.15.102:0/4146582996 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f40d00cc3d0 0x7f40d00d5630 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:45.019 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:45.033 7f40d7bd5700 1 --2- 172.21.15.102:0/4146582996 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f40d00ca770 0x7f40d00cb950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:45.020 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:45.033 7f40d7bd5700 1 --2- 172.21.15.102:0/4146582996 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f40d00cbf80 0x7f40d00d5000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:45.020 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:45.033 7f40d7bd5700 1 -- 172.21.15.102:0/4146582996 >> 172.21.15.102:0/4146582996 conn(0x7f40d00c5480 msgr2=0x7f40d00d6030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:45.020 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:45.033 7f40d7bd5700 1 -- 172.21.15.102:0/4146582996 shutdown_connections 2024-04-17T22:31:45.020 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:45.033 7f40d7bd5700 1 -- 172.21.15.102:0/4146582996 wait complete. 2024-04-17T22:31:45.020 INFO:teuthology.orchestra.run.smithi102.stderr:dumped all 2024-04-17T22:31:45.029 INFO:teuthology.orchestra.run.smithi102.stdout:{"pg_ready":true,"pg_map":{"version":109,"stamp":"2024-04-17 22:31:43.222500","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":272,"num_osds":4,"num_per_pool_osds":4,"kb":377487360,"kb_used":4195312,"kb_used_data":968,"kb_used_omap":0,"kb_used_meta":4194304,"kb_avail":373292048,"statfs":{"total":386547056640,"available":382251057152,"internally_reserved":4294967296,"allocated":991232,"data_stored":305326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4294967296},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-2286,"num_objects":-21,"num_object_clones":0,"num_object_copies":-42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-44,"num_write_kb":-13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.309956"},"pg_stats":[{"pgid":"1.5","version":"0'0","reported_seq":"23","reported_epoch":"25","state":"active+clean","last_fresh":"2024-04-17 22:31:29.216143","last_change":"2024-04-17 22:28:59.253977","last_active":"2024-04-17 22:31:29.216143","last_peered":"2024-04-17 22:31:29.216143","last_clean":"2024-04-17 22:31:29.216143","last_became_active":"2024-04-17 22:28:59.253641","last_became_peered":"2024-04-17 22:28:59.253641","last_unstale":"2024-04-17 22:31:29.216143","last_undegraded":"2024-04-17 22:31:29.216143","last_fullsized":"2024-04-17 22:31:29.216143","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:28:58.239187","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:28:58.239187","last_clean_scrub_stamp":"2024-04-17 22:28:58.239187","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":"23","reported_epoch":"25","state":"active+clean","last_fresh":"2024-04-17 22:31:29.420646","last_change":"2024-04-17 22:28:59.255733","last_active":"2024-04-17 22:31:29.420646","last_peered":"2024-04-17 22:31:29.420646","last_clean":"2024-04-17 22:31:29.420646","last_became_active":"2024-04-17 22:28:59.255411","last_became_peered":"2024-04-17 22:28:59.255411","last_unstale":"2024-04-17 22:31:29.420646","last_undegraded":"2024-04-17 22:31:29.420646","last_fullsized":"2024-04-17 22:31:29.420646","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:28:58.239187","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:28:58.239187","last_clean_scrub_stamp":"2024-04-17 22:28:58.239187","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":"23","reported_epoch":"25","state":"active+clean","last_fresh":"2024-04-17 22:31:29.215551","last_change":"2024-04-17 22:28:59.252728","last_active":"2024-04-17 22:31:29.215551","last_peered":"2024-04-17 22:31:29.215551","last_clean":"2024-04-17 22:31:29.215551","last_became_active":"2024-04-17 22:28:59.252345","last_became_peered":"2024-04-17 22:28:59.252345","last_unstale":"2024-04-17 22:31:29.215551","last_undegraded":"2024-04-17 22:31:29.215551","last_fullsized":"2024-04-17 22:31:29.215551","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:28:58.239187","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:28:58.239187","last_clean_scrub_stamp":"2024-04-17 22:28:58.239187","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":"23","reported_epoch":"25","state":"active+clean","last_fresh":"2024-04-17 22:31:29.214261","last_change":"2024-04-17 22:28:59.253392","last_active":"2024-04-17 22:31:29.214261","last_peered":"2024-04-17 22:31:29.214261","last_clean":"2024-04-17 22:31:29.214261","last_became_active":"2024-04-17 22:28:59.253049","last_became_peered":"2024-04-17 22:28:59.253049","last_unstale":"2024-04-17 22:31:29.214261","last_undegraded":"2024-04-17 22:31:29.214261","last_fullsized":"2024-04-17 22:31:29.214261","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:28:58.239187","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:28:58.239187","last_clean_scrub_stamp":"2024-04-17 22:28:58.239187","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":"23","reported_epoch":"25","state":"active+clean","last_fresh":"2024-04-17 22:31:29.214314","last_change":"2024-04-17 22:28:59.254907","last_active":"2024-04-17 22:31:29.214314","last_peered":"2024-04-17 22:31:29.214314","last_clean":"2024-04-17 22:31:29.214314","last_became_active":"2024-04-17 22:28:59.254633","last_became_peered":"2024-04-17 22:28:59.254633","last_unstale":"2024-04-17 22:31:29.214314","last_undegraded":"2024-04-17 22:31:29.214314","last_fullsized":"2024-04-17 22:31:29.214314","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:28:58.239187","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:28:58.239187","last_clean_scrub_stamp":"2024-04-17 22:28:58.239187","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":"23","reported_epoch":"25","state":"active+clean","last_fresh":"2024-04-17 22:31:29.420309","last_change":"2024-04-17 22:28:59.254217","last_active":"2024-04-17 22:31:29.420309","last_peered":"2024-04-17 22:31:29.420309","last_clean":"2024-04-17 22:31:29.420309","last_became_active":"2024-04-17 22:28:59.253877","last_became_peered":"2024-04-17 22:28:59.253877","last_unstale":"2024-04-17 22:31:29.420309","last_undegraded":"2024-04-17 22:31:29.420309","last_fullsized":"2024-04-17 22:31:29.420309","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:28:58.239187","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:28:58.239187","last_clean_scrub_stamp":"2024-04-17 22:28:58.239187","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":"23","reported_epoch":"25","state":"active+clean","last_fresh":"2024-04-17 22:31:29.422721","last_change":"2024-04-17 22:28:59.254830","last_active":"2024-04-17 22:31:29.422721","last_peered":"2024-04-17 22:31:29.422721","last_clean":"2024-04-17 22:31:29.422721","last_became_active":"2024-04-17 22:28:59.254505","last_became_peered":"2024-04-17 22:28:59.254505","last_unstale":"2024-04-17 22:31:29.422721","last_undegraded":"2024-04-17 22:31:29.422721","last_fullsized":"2024-04-17 22:31:29.422721","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:28:58.239187","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:28:58.239187","last_clean_scrub_stamp":"2024-04-17 22:28:58.239187","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":"23","reported_epoch":"25","state":"active+clean","last_fresh":"2024-04-17 22:31:29.216381","last_change":"2024-04-17 22:28:59.254055","last_active":"2024-04-17 22:31:29.216381","last_peered":"2024-04-17 22:31:29.216381","last_clean":"2024-04-17 22:31:29.216381","last_became_active":"2024-04-17 22:28:59.253750","last_became_peered":"2024-04-17 22:28:59.253750","last_unstale":"2024-04-17 22:31:29.216381","last_undegraded":"2024-04-17 22:31:29.216381","last_fullsized":"2024-04-17 22:31:29.216381","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:28:58.239187","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:28:58.239187","last_clean_scrub_stamp":"2024-04-17 22:28:58.239187","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705701,"num_pgs":82,"num_osds":1,"num_per_pool_osds":1,"kb":94371840,"kb_used":1048852,"kb_used_data":260,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93322988,"statfs":{"total":96636764160,"available":95562739712,"internally_reserved":1073741824,"allocated":266240,"data_stored":79117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 17 22:30:59 2024","interfaces":[{"interface":"back","average":{"1min":0.308,"5min":0.30399999999999999,"15min":0.30399999999999999},"min":{"1min":0.20399999999999999,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.30199999999999999,"5min":0.314,"15min":0.316},"min":{"1min":0.21099999999999999,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.44600000000000001,"5min":0.52400000000000002,"15min":0.52400000000000002},"last":0.35499999999999998}]},{"osd":1,"last update":"Wed Apr 17 22:30:59 2024","interfaces":[{"interface":"back","average":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"min":{"1min":0.223,"5min":0.19600000000000001,"15min":0.19600000000000001},"max":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"last":0.378},{"interface":"front","average":{"1min":0.33400000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"min":{"1min":0.249,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.438,"5min":0.55300000000000005,"15min":0.55300000000000005},"last":0.39300000000000002}]},{"osd":2,"last update":"Wed Apr 17 22:30:59 2024","interfaces":[{"interface":"back","average":{"1min":0.33800000000000002,"5min":0.34799999999999998,"15min":0.34999999999999998},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.46200000000000002,"5min":0.50600000000000001,"15min":0.50600000000000001},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.32800000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"min":{"1min":0.23200000000000001,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.46899999999999997,"5min":0.47699999999999998,"15min":0.47699999999999998},"last":0.40400000000000003}]}]},{"osd":2,"up_from":9,"seq":38654705701,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"kb":94371840,"kb_used":1048828,"kb_used_data":244,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323012,"statfs":{"total":96636764160,"available":95562764288,"internally_reserved":1073741824,"allocated":249856,"data_stored":78371,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 17 22:30:58 2024","interfaces":[{"interface":"back","average":{"1min":0.30099999999999999,"5min":0.28499999999999998,"15min":0.28299999999999997},"min":{"1min":0.22600000000000001,"5min":0.112,"15min":0.112},"max":{"1min":0.41499999999999998,"5min":0.46400000000000002,"15min":0.46400000000000002},"last":0.23400000000000001},{"interface":"front","average":{"1min":0.33600000000000002,"5min":0.314,"15min":0.31},"min":{"1min":0.27600000000000002,"5min":0.155,"15min":0.155},"max":{"1min":0.40400000000000003,"5min":0.49199999999999999,"15min":0.49199999999999999},"last":0.27100000000000002}]},{"osd":1,"last update":"Wed Apr 17 22:30:58 2024","interfaces":[{"interface":"back","average":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"min":{"1min":0.214,"5min":0.17899999999999999,"15min":0.17899999999999999},"max":{"1min":0.377,"5min":0.52900000000000003,"15min":0.52900000000000003},"last":0.253},{"interface":"front","average":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"min":{"1min":0.245,"5min":0.192,"15min":0.192},"max":{"1min":0.38800000000000001,"5min":0.47199999999999998,"15min":0.47199999999999998},"last":0.25900000000000001}]},{"osd":3,"last update":"Wed Apr 17 22:30:58 2024","interfaces":[{"interface":"back","average":{"1min":0.30399999999999999,"5min":0.29599999999999999,"15min":0.29499999999999998},"min":{"1min":0.24199999999999999,"5min":0.191,"15min":0.191},"max":{"1min":0.42599999999999999,"5min":0.45600000000000002,"15min":0.45600000000000002},"last":0.245},{"interface":"front","average":{"1min":0.31900000000000001,"5min":0.311,"15min":0.31},"min":{"1min":0.26200000000000001,"5min":0.214,"15min":0.214},"max":{"1min":0.41299999999999998,"5min":0.48699999999999999,"15min":0.48699999999999999},"last":0.28199999999999997}]}]},{"osd":0,"up_from":9,"seq":38654705701,"num_pgs":62,"num_osds":1,"num_per_pool_osds":1,"kb":94371840,"kb_used":1048812,"kb_used_data":228,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323028,"statfs":{"total":96636764160,"available":95562780672,"internally_reserved":1073741824,"allocated":233472,"data_stored":73636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Apr 17 22:30:56 2024","interfaces":[{"interface":"back","average":{"1min":0.248,"5min":0.28299999999999997,"15min":0.28899999999999998},"min":{"1min":0.159,"5min":0.159,"15min":0.159},"max":{"1min":0.33200000000000002,"5min":0.40899999999999997,"15min":0.40899999999999997},"last":0.22900000000000001},{"interface":"front","average":{"1min":0.26000000000000001,"5min":0.29199999999999998,"15min":0.29699999999999999},"min":{"1min":0.189,"5min":0.16600000000000001,"15min":0.16600000000000001},"max":{"1min":0.35399999999999998,"5min":0.42999999999999999,"15min":0.42999999999999999},"last":0.25}]},{"osd":2,"last update":"Wed Apr 17 22:30:56 2024","interfaces":[{"interface":"back","average":{"1min":0.28899999999999998,"5min":0.32100000000000001,"15min":0.32700000000000001},"min":{"1min":0.218,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.39500000000000002,"5min":0.48599999999999999,"15min":0.48599999999999999},"last":0.313},{"interface":"front","average":{"1min":0.27800000000000002,"5min":0.317,"15min":0.32300000000000001},"min":{"1min":0.19400000000000001,"5min":0.19400000000000001,"15min":0.19400000000000001},"max":{"1min":0.371,"5min":0.60699999999999998,"15min":0.60699999999999998},"last":0.30599999999999999}]},{"osd":3,"last update":"Wed Apr 17 22:30:56 2024","interfaces":[{"interface":"back","average":{"1min":0.311,"5min":0.33400000000000002,"15min":0.33800000000000002},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.40600000000000003,"5min":0.505,"15min":0.505},"last":0.33000000000000002},{"interface":"front","average":{"1min":0.28599999999999998,"5min":0.32000000000000001,"15min":0.32600000000000001},"min":{"1min":0.20000000000000001,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.38300000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"last":0.26000000000000001}]}]},{"osd":1,"up_from":9,"seq":38654705700,"num_pgs":73,"num_osds":1,"num_per_pool_osds":1,"kb":94371840,"kb_used":1048820,"kb_used_data":236,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323020,"statfs":{"total":96636764160,"available":95562772480,"internally_reserved":1073741824,"allocated":241664,"data_stored":74202,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 17 22:31:00 2024","interfaces":[{"interface":"back","average":{"1min":0.26900000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"min":{"1min":0.128,"5min":0.128,"15min":0.128},"max":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"last":0.17399999999999999},{"interface":"front","average":{"1min":0.28000000000000003,"5min":0.27600000000000002,"15min":0.27600000000000002},"min":{"1min":0.19400000000000001,"5min":0.191,"15min":0.191},"max":{"1min":0.441,"5min":0.441,"15min":0.441},"last":0.189}]},{"osd":2,"last update":"Wed Apr 17 22:31:00 2024","interfaces":[{"interface":"back","average":{"1min":0.28599999999999998,"5min":0.27900000000000003,"15min":0.27800000000000002},"min":{"1min":0.153,"5min":0.153,"15min":0.153},"max":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"last":0.20999999999999999},{"interface":"front","average":{"1min":0.28599999999999998,"5min":0.29099999999999998,"15min":0.29199999999999998},"min":{"1min":0.18099999999999999,"5min":0.18099999999999999,"15min":0.18099999999999999},"max":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"last":0.20000000000000001}]},{"osd":3,"last update":"Wed Apr 17 22:31:00 2024","interfaces":[{"interface":"back","average":{"1min":0.30499999999999999,"5min":0.29699999999999999,"15min":0.29499999999999998},"min":{"1min":0.23300000000000001,"5min":0.218,"15min":0.218},"max":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"last":0.218},{"interface":"front","average":{"1min":0.309,"5min":0.315,"15min":0.316},"min":{"1min":0.245,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"last":0.23100000000000001}]}]}]}} 2024-04-17T22:31:45.030 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T22:31:45.155 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:45.169 7f6fe9ed9700 1 Processor -- start 2024-04-17T22:31:45.155 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:45.169 7f6fe9ed9700 1 -- start start 2024-04-17T22:31:45.157 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:45.169 7f6fe9ed9700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f6fe40f5860 0x7f6fe410b030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:45.157 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:45.169 7f6fe9ed9700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f6fe410b660 0x7f6fe419fb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:45.157 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:45.169 7f6fe9ed9700 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fe4103b60 con 0x7f6fe410bab0 2024-04-17T22:31:45.157 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:45.169 7f6fe9ed9700 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6fe410bf00 con 0x7f6fe40f5860 2024-04-17T22:31:45.157 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:45.169 7f6fe9ed9700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6fe40fa390 con 0x7f6fe410b660 2024-04-17T22:31:45.157 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:45.169 7f6fe2cc5700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f6fe40f5860 0x7f6fe410b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:45.157 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:45.169 7f6fe24c4700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f6fe410b660 0x7f6fe419fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:45.157 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:45.169 7f6fe34c6700 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f6fe410bab0 0x7f6fe41a0140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:47862/0 (socket says 172.21.15.102:47862) 2024-04-17T22:31:45.158 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:45.169 7f6fe34c6700 1 -- 172.21.15.102:0/4120117224 learned_addr learned my addr 172.21.15.102:0/4120117224 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-17T22:31:45.158 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:45.169 7f6fe2cc5700 1 --2- 172.21.15.102:0/4120117224 >> v2:172.21.15.102:3300/0 conn(0x7f6fe40f5860 0x7f6fe410b030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:34766/0 (socket says 172.21.15.102:34766) 2024-04-17T22:31:45.158 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:45.170 7f6fe24c4700 1 -- 172.21.15.102:0/4120117224 >> v2:172.21.15.102:3300/0 conn(0x7f6fe40f5860 msgr2=0x7f6fe410b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:31:45.158 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:45.170 7f6fe24c4700 1 --2- 172.21.15.102:0/4120117224 >> v2:172.21.15.102:3300/0 conn(0x7f6fe40f5860 0x7f6fe410b030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:45.158 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:45.170 7f6fe24c4700 1 -- 172.21.15.102:0/4120117224 >> v1:172.21.15.102:6789/0 conn(0x7f6fe410bab0 legacy=0x7f6fe41a0140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:45.158 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:45.170 7f6fe24c4700 1 -- 172.21.15.102:0/4120117224 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fe41a0940 con 0x7f6fe410b660 2024-04-17T22:31:45.158 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:45.170 7f6fe2cc5700 1 --2- 172.21.15.102:0/4120117224 >> v2:172.21.15.102:3300/0 conn(0x7f6fe40f5860 0x7f6fe410b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:45.158 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:45.170 7f6fe24c4700 1 --2- 172.21.15.102:0/4120117224 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f6fe410b660 0x7f6fe419fb10 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 rx=0x7f6fd4002f70 tx=0x7f6fd400b970).ready entity=mon.2 client_cookie=7ca2383389a86179 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:45.158 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:45.170 7f6fe1cc3700 1 -- 172.21.15.102:0/4120117224 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f6fe41a0940 con 0x7f6fe410b660 2024-04-17T22:31:45.158 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:45.170 7f6fe1cc3700 1 -- 172.21.15.102:0/4120117224 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fd400d5c0 con 0x7f6fe410b660 2024-04-17T22:31:45.158 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:45.170 7f6fe1cc3700 1 -- 172.21.15.102:0/4120117224 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f6fd400d990 con 0x7f6fe410b660 2024-04-17T22:31:45.158 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:45.170 7f6fe9ed9700 1 -- 172.21.15.102:0/4120117224 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f6fe410b660 msgr2=0x7f6fe419fb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:45.158 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:45.170 7f6fe9ed9700 1 --2- 172.21.15.102:0/4120117224 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f6fe410b660 0x7f6fe419fb10 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 rx=0x7f6fd4002f70 tx=0x7f6fd400b970).stop 2024-04-17T22:31:45.159 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:45.171 7f6fe9ed9700 1 -- 172.21.15.102:0/4120117224 shutdown_connections 2024-04-17T22:31:45.159 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:45.171 7f6fe9ed9700 1 --2- 172.21.15.102:0/4120117224 >> v2:172.21.15.102:3300/0 conn(0x7f6fe40f5860 0x7f6fe410b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:45.159 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:45.171 7f6fe9ed9700 1 --2- 172.21.15.102:0/4120117224 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f6fe410b660 0x7f6fe419fb10 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:45.159 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:45.171 7f6fe9ed9700 1 -- 172.21.15.102:0/4120117224 >> 172.21.15.102:0/4120117224 conn(0x7f6fe41138c0 msgr2=0x7f6fe40f4f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:45.159 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:45.171 7f6fe9ed9700 1 -- 172.21.15.102:0/4120117224 shutdown_connections 2024-04-17T22:31:45.159 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:45.171 7f6fe9ed9700 1 -- 172.21.15.102:0/4120117224 wait complete. 2024-04-17T22:31:45.181 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:45.193 7f6fe9ed9700 1 Processor -- start 2024-04-17T22:31:45.181 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:45.193 7f6fe9ed9700 1 -- start start 2024-04-17T22:31:45.181 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:45.193 7f6fe9ed9700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6fe40f1920 0x7f6fe40d8fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:45.181 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:45.193 7f6fe9ed9700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f6fe40d95f0 0x7f6fe41ab520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:45.181 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:45.193 7f6fe9ed9700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6fe40d9a40 0x7f6fe41abb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:45.181 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:45.193 7f6fe9ed9700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fe4087b80 con 0x7f6fe40d9a40 2024-04-17T22:31:45.181 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:45.193 7f6fe9ed9700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fe40f23a0 con 0x7f6fe40f1920 2024-04-17T22:31:45.182 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:45.193 7f6fe9ed9700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6fe40f1f80 con 0x7f6fe40d95f0 2024-04-17T22:31:45.182 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:45.194 7f6fe2cc5700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6fe40f1920 0x7f6fe40d8fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:45.182 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:45.194 7f6fe34c6700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6fe40d9a40 0x7f6fe41abb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:45.182 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:45.194 7f6fe24c4700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f6fe40d95f0 0x7f6fe41ab520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:45.182 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:45.194 7f6fe2cc5700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6fe40f1920 0x7f6fe40d8fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:49750/0 (socket says 172.21.15.102:49750) 2024-04-17T22:31:45.182 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:45.194 7f6fe2cc5700 1 -- 172.21.15.102:0/2242581079 learned_addr learned my addr 172.21.15.102:0/2242581079 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:31:45.182 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:45.194 7f6fe34c6700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6fe40d9a40 0x7f6fe41abb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:34776/0 (socket says 172.21.15.102:34776) 2024-04-17T22:31:45.182 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:45.194 7f6fe24c4700 1 --2- 172.21.15.102:0/2242581079 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f6fe40d95f0 0x7f6fe41ab520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:34500/0 (socket says 172.21.15.102:34500) 2024-04-17T22:31:45.182 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:45.194 7f6fe34c6700 1 -- 172.21.15.102:0/2242581079 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f6fe40d95f0 msgr2=0x7f6fe41ab520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:45.182 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:45.194 7f6fe34c6700 1 --2- 172.21.15.102:0/2242581079 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f6fe40d95f0 0x7f6fe41ab520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:45.182 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:45.194 7f6fe34c6700 1 -- 172.21.15.102:0/2242581079 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6fe40f1920 msgr2=0x7f6fe40d8fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:45.182 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:45.194 7f6fe34c6700 1 --2- 172.21.15.102:0/2242581079 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6fe40f1920 0x7f6fe40d8fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:45.182 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:45.194 7f6fe34c6700 1 -- 172.21.15.102:0/2242581079 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fe41ac180 con 0x7f6fe40d9a40 2024-04-17T22:31:45.182 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:45.194 7f6fe24c4700 1 --2- 172.21.15.102:0/2242581079 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f6fe40d95f0 0x7f6fe41ab520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:45.183 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:45.194 7f6fe34c6700 1 --2- 172.21.15.102:0/2242581079 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6fe40d9a40 0x7f6fe41abb50 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 rx=0x7f6fdc000f40 tx=0x7f6fdc00c910).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:45.183 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:45.194 7f6fd3fff700 1 -- 172.21.15.102:0/2242581079 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f6fe41ac180 con 0x7f6fe40d9a40 2024-04-17T22:31:45.183 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:45.194 7f6fd3fff700 1 -- 172.21.15.102:0/2242581079 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fdc005bb0 con 0x7f6fe40d9a40 2024-04-17T22:31:45.183 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:45.194 7f6fd3fff700 1 -- 172.21.15.102:0/2242581079 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f6fdc0070d0 con 0x7f6fe40d9a40 2024-04-17T22:31:45.183 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:45.194 7f6fe9ed9700 1 -- 172.21.15.102:0/2242581079 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fe41b96a0 con 0x7f6fe40d9a40 2024-04-17T22:31:45.183 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:45.194 7f6fe9ed9700 1 -- 172.21.15.102:0/2242581079 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fe41b9a80 con 0x7f6fe40d9a40 2024-04-17T22:31:45.183 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:45.195 7f6fd3fff700 1 -- 172.21.15.102:0/2242581079 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7f6fdc01b070 con 0x7f6fe40d9a40 2024-04-17T22:31:45.183 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:45.195 7f6fd3fff700 1 -- 172.21.15.102:0/2242581079 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 2417+0+0 (secure 0 0 0) 0x7f6fdc013450 con 0x7f6fe40d9a40 2024-04-17T22:31:45.183 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:45.195 7f6fe9ed9700 1 -- 172.21.15.102:0/2242581079 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fe40f0820 con 0x7f6fe40d9a40 2024-04-17T22:31:45.184 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:45.198 7f6fd3fff700 1 -- 172.21.15.102:0/2242581079 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+110455 (secure 0 0 0) 0x7f6fdc0059a0 con 0x7f6fe40d9a40 2024-04-17T22:31:45.290 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:45.305 7f6fe9ed9700 1 -- 172.21.15.102:0/2242581079 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6fe41ac690 con 0x7f6fe40d9a40 2024-04-17T22:31:45.291 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:45.305 7f6fd3fff700 1 -- 172.21.15.102:0/2242581079 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+5966 (secure 0 0 0) 0x7f6fdc000fb0 con 0x7f6fe40d9a40 2024-04-17T22:31:45.291 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-17T22:31:45.291 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":28,"fsid":"cb63e789-091d-49b1-b120-55f9db526871","created":"2024-04-17 22:28:48.209635","modified":"2024-04-17 22:31:42.528819","last_up_change":"2024-04-17 22:28:54.236215","last_in_change":"2024-04-17 22:28:54.236215","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":4,"require_min_compat_client":"nautilus","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-04-17 22:28:57.478545","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"3351ef85-ade4-4b90-a8f4-213f4e00ba01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6812","nonce":42955}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6813","nonce":42955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6815","nonce":42955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6814","nonce":42955}]},"public_addr":"172.21.15.102:6812/42955","cluster_addr":"172.21.15.102:6813/42955","heartbeat_back_addr":"172.21.15.102:6815/42955","heartbeat_front_addr":"172.21.15.102:6814/42955","state":["exists","up"]},{"osd":1,"uuid":"04f67eac-f36b-48f5-982d-4cfc776596d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6808","nonce":42958}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6809","nonce":42958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6811","nonce":42958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6810","nonce":42958}]},"public_addr":"172.21.15.102:6808/42958","cluster_addr":"172.21.15.102:6809/42958","heartbeat_back_addr":"172.21.15.102:6811/42958","heartbeat_front_addr":"172.21.15.102:6810/42958","state":["exists","up"]},{"osd":2,"uuid":"d524204b-70b2-40a8-8f73-f9e7d44318af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6804","nonce":42954}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6805","nonce":42954}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6807","nonce":42954}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6806","nonce":42954}]},"public_addr":"172.21.15.102:6804/42954","cluster_addr":"172.21.15.102:6805/42954","heartbeat_back_addr":"172.21.15.102:6807/42954","heartbeat_front_addr":"172.21.15.102:6806/42954","state":["exists","up"]},{"osd":3,"uuid":"4fc9d20c-1ad0-484f-aafa-2c9a7574245b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6800","nonce":42953}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6801","nonce":42953}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6803","nonce":42953}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6802","nonce":42953}]},"public_addr":"172.21.15.102:6800/42953","cluster_addr":"172.21.15.102:6801/42953","heartbeat_back_addr":"172.21.15.102:6803/42953","heartbeat_front_addr":"172.21.15.102:6802/42953","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854035861503,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{"172.21.15.102:6817/479367134":"2024-04-18 22:31:39.202411"},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2024-04-17T22:31:45.293 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:45.306 7f6fe9ed9700 1 -- 172.21.15.102:0/2242581079 >> v1:172.21.15.102:6816/42360 conn(0x7f6fc00145a0 legacy=0x7f6fc0016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:45.293 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:45.306 7f6fe9ed9700 1 -- 172.21.15.102:0/2242581079 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6fe40d9a40 msgr2=0x7f6fe41abb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:45.293 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:45.306 7f6fe9ed9700 1 --2- 172.21.15.102:0/2242581079 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6fe40d9a40 0x7f6fe41abb50 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 rx=0x7f6fdc000f40 tx=0x7f6fdc00c910).stop 2024-04-17T22:31:45.293 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:45.306 7f6fe9ed9700 1 -- 172.21.15.102:0/2242581079 shutdown_connections 2024-04-17T22:31:45.293 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:45.306 7f6fe9ed9700 1 --2- 172.21.15.102:0/2242581079 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6fe40d9a40 0x7f6fe41abb50 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:45.293 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:45.306 7f6fe9ed9700 1 --2- 172.21.15.102:0/2242581079 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6fe40f1920 0x7f6fe40d8fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:45.293 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:45.306 7f6fe9ed9700 1 --2- 172.21.15.102:0/2242581079 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f6fe40d95f0 0x7f6fe41ab520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:45.294 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:45.306 7f6fe9ed9700 1 -- 172.21.15.102:0/2242581079 >> 172.21.15.102:0/2242581079 conn(0x7f6fe40c6a60 msgr2=0x7f6fe40d6220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:45.294 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:45.306 7f6fe9ed9700 1 -- 172.21.15.102:0/2242581079 shutdown_connections 2024-04-17T22:31:45.294 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:45.306 7f6fe9ed9700 1 -- 172.21.15.102:0/2242581079 wait complete. 2024-04-17T22:31:46.304 INFO:tasks.ceph:Scrubbing osd.0 2024-04-17T22:31:46.304 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-04-17T22:31:46.447 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.459 7f877bf77700 1 Processor -- start 2024-04-17T22:31:46.448 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.459 7f877bf77700 1 -- start start 2024-04-17T22:31:46.448 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.459 7f877bf77700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f87740f5450 0x7f877410be50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:46.448 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.459 7f877bf77700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f877410c480 0x7f87740f1430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:46.448 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.460 7f877bf77700 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87740f8d60 con 0x7f877410c8d0 2024-04-17T22:31:46.448 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.460 7f877bf77700 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87740f9000 con 0x7f87740f5450 2024-04-17T22:31:46.448 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.460 7f877bf77700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87740f2260 con 0x7f877410c480 2024-04-17T22:31:46.448 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.460 7f8773fff700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f877410c480 0x7f87740f1430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:46.448 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.460 7f8778d72700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f87740f5450 0x7f877410be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:46.448 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.460 7f8773fff700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f877410c480 0x7f87740f1430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:49764/0 (socket says 172.21.15.102:49764) 2024-04-17T22:31:46.448 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.460 7f8773fff700 1 -- 172.21.15.102:0/4152365914 learned_addr learned my addr 172.21.15.102:0/4152365914 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:31:46.448 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.460 7f8779573700 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f877410c8d0 0x7f87740f1a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:47872/0 (socket says 172.21.15.102:47872) 2024-04-17T22:31:46.449 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.460 7f8773fff700 1 -- 172.21.15.102:0/4152365914 >> v2:172.21.15.102:3300/0 conn(0x7f87740f5450 msgr2=0x7f877410be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:31:46.449 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.460 7f8773fff700 1 --2- 172.21.15.102:0/4152365914 >> v2:172.21.15.102:3300/0 conn(0x7f87740f5450 0x7f877410be50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:46.449 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.460 7f8773fff700 1 -- 172.21.15.102:0/4152365914 >> v1:172.21.15.102:6789/0 conn(0x7f877410c8d0 legacy=0x7f87740f1a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:46.449 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.460 7f8773fff700 1 -- 172.21.15.102:0/4152365914 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87741aa030 con 0x7f877410c480 2024-04-17T22:31:46.449 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.460 7f87737fe700 1 -- 172.21.15.102:0/4152365914 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2589046485 0 0) 0x7f87740f8d60 con 0x7f877410c8d0 2024-04-17T22:31:46.449 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.460 7f8778d72700 1 --2- 172.21.15.102:0/4152365914 >> v2:172.21.15.102:3300/0 conn(0x7f87740f5450 0x7f877410be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:46.449 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.460 7f8773fff700 1 --2- 172.21.15.102:0/4152365914 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f877410c480 0x7f87740f1430 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 rx=0x7f87740f91b0 tx=0x7f875c009a00).ready entity=mon.1 client_cookie=3c97c1df2c142a8f server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:46.449 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.460 7f87737fe700 1 -- 172.21.15.102:0/4152365914 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f87741aa030 con 0x7f877410c480 2024-04-17T22:31:46.449 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.460 7f87737fe700 1 -- 172.21.15.102:0/4152365914 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f875c00f510 con 0x7f877410c480 2024-04-17T22:31:46.449 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.460 7f87737fe700 1 -- 172.21.15.102:0/4152365914 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f875c00f6f0 con 0x7f877410c480 2024-04-17T22:31:46.449 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.460 7f877bf77700 1 -- 172.21.15.102:0/4152365914 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f877410c480 msgr2=0x7f87740f1430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:46.449 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.460 7f877bf77700 1 --2- 172.21.15.102:0/4152365914 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f877410c480 0x7f87740f1430 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 rx=0x7f87740f91b0 tx=0x7f875c009a00).stop 2024-04-17T22:31:46.449 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.461 7f877bf77700 1 -- 172.21.15.102:0/4152365914 shutdown_connections 2024-04-17T22:31:46.449 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.461 7f877bf77700 1 --2- 172.21.15.102:0/4152365914 >> v2:172.21.15.102:3300/0 conn(0x7f87740f5450 0x7f877410be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:46.450 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.461 7f877bf77700 1 --2- 172.21.15.102:0/4152365914 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f877410c480 0x7f87740f1430 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:46.450 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.461 7f877bf77700 1 -- 172.21.15.102:0/4152365914 >> 172.21.15.102:0/4152365914 conn(0x7f87741135a0 msgr2=0x7f87740f4c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:46.450 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.461 7f877bf77700 1 -- 172.21.15.102:0/4152365914 shutdown_connections 2024-04-17T22:31:46.450 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.461 7f877bf77700 1 -- 172.21.15.102:0/4152365914 wait complete. 2024-04-17T22:31:46.469 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.483 7f877bf77700 1 Processor -- start 2024-04-17T22:31:46.469 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.483 7f877bf77700 1 -- start start 2024-04-17T22:31:46.472 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.484 7f877bf77700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f87740ddf90 0x7f87741ad160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:46.472 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.484 7f877bf77700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f87741ad790 0x7f87741b3be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:46.472 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.484 7f877bf77700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f87741adbe0 0x7f87741b4210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:46.472 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.484 7f877bf77700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87740a8720 con 0x7f87741adbe0 2024-04-17T22:31:46.472 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.484 7f877bf77700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87740dea60 con 0x7f87740ddf90 2024-04-17T22:31:46.472 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.484 7f877bf77700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f87741b4840 con 0x7f87741ad790 2024-04-17T22:31:46.472 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.484 7f8778d72700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f87740ddf90 0x7f87741ad160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:46.472 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.484 7f8779573700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f87741adbe0 0x7f87741b4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:46.472 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.484 7f8778d72700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f87740ddf90 0x7f87741ad160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:49772/0 (socket says 172.21.15.102:49772) 2024-04-17T22:31:46.473 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.484 7f8778d72700 1 -- 172.21.15.102:0/17147861 learned_addr learned my addr 172.21.15.102:0/17147861 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:31:46.473 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.484 7f8773fff700 1 --2- 172.21.15.102:0/17147861 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f87741ad790 0x7f87741b3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:46.473 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.484 7f8778d72700 1 -- 172.21.15.102:0/17147861 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f87741ad790 msgr2=0x7f87741b3be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:46.473 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.484 7f8778d72700 1 --2- 172.21.15.102:0/17147861 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f87741ad790 0x7f87741b3be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:46.473 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.484 7f8778d72700 1 -- 172.21.15.102:0/17147861 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f87741adbe0 msgr2=0x7f87741b4210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:46.473 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.484 7f8778d72700 1 --2- 172.21.15.102:0/17147861 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f87741adbe0 0x7f87741b4210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:46.473 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.484 7f8778d72700 1 -- 172.21.15.102:0/17147861 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87741b4a70 con 0x7f87740ddf90 2024-04-17T22:31:46.473 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.484 7f8779573700 1 --2- 172.21.15.102:0/17147861 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f87741adbe0 0x7f87741b4210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:46.473 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.484 7f8778d72700 1 --2- 172.21.15.102:0/17147861 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f87740ddf90 0x7f87741ad160 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 rx=0x7f87740e63a0 tx=0x7f876400bf70).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:46.473 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.484 7f8771ffb700 1 -- 172.21.15.102:0/17147861 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f87741b4a70 con 0x7f87740ddf90 2024-04-17T22:31:46.473 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.484 7f8771ffb700 1 -- 172.21.15.102:0/17147861 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f876400c9a0 con 0x7f87740ddf90 2024-04-17T22:31:46.473 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.484 7f877bf77700 1 -- 172.21.15.102:0/17147861 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87741bb350 con 0x7f87740ddf90 2024-04-17T22:31:46.473 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.484 7f8771ffb700 1 -- 172.21.15.102:0/17147861 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f876400f040 con 0x7f87740ddf90 2024-04-17T22:31:46.473 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.485 7f877bf77700 1 -- 172.21.15.102:0/17147861 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87741bb730 con 0x7f87740ddf90 2024-04-17T22:31:46.473 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.485 7f8771ffb700 1 -- 172.21.15.102:0/17147861 <== mon.1 v2:172.21.15.102:3301/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7f8764011070 con 0x7f87740ddf90 2024-04-17T22:31:46.474 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.485 7f8771ffb700 1 -- 172.21.15.102:0/17147861 <== mon.1 v2:172.21.15.102:3301/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 2417+0+0 (secure 0 0 0) 0x7f8764010040 con 0x7f87740ddf90 2024-04-17T22:31:46.474 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.485 7f877bf77700 1 -- 172.21.15.102:0/17147861 --> v1:172.21.15.102:6812/42955 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f87740c4ed0 con 0x7f87740e5560 2024-04-17T22:31:46.474 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.486 7f8771ffb700 1 -- 172.21.15.102:0/17147861 <== osd.0 v1:172.21.15.102:6812/42955 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+5151 (unknown 0 0 4057655604) 0x7f876c001b80 con 0x7f87740e5560 2024-04-17T22:31:46.478 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.492 7f877bf77700 1 -- 172.21.15.102:0/17147861 --> v1:172.21.15.102:6812/42955 -- command(tid 2: {"prefix": "config set", "key": "osd_debug_deep_scrub_sleep", "value": "0"}) v1 -- 0x7f87740878c0 con 0x7f87740e5560 2024-04-17T22:31:46.478 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.492 7f8771ffb700 1 -- 172.21.15.102:0/17147861 <== osd.0 v1:172.21.15.102:6812/42955 2 ==== command_reply(tid 2: 0 Set osd_debug_deep_scrub_sleep to 0) v1 ==== 43+0+0 (unknown 1994953735 0 0) 0x7f876c001d60 con 0x7f87740e5560 2024-04-17T22:31:46.478 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.492 7f877bf77700 1 -- 172.21.15.102:0/17147861 >> v1:172.21.15.102:6812/42955 conn(0x7f87740e5560 legacy=0x7f87740a7bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:46.478 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.492 7f877bf77700 1 -- 172.21.15.102:0/17147861 >> v1:172.21.15.102:6816/42360 conn(0x7f87580145a0 legacy=0x7f8758016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:46.478 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.492 7f877bf77700 1 -- 172.21.15.102:0/17147861 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f87740ddf90 msgr2=0x7f87741ad160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:46.479 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.492 7f877bf77700 1 --2- 172.21.15.102:0/17147861 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f87740ddf90 0x7f87741ad160 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 rx=0x7f87740e63a0 tx=0x7f876400bf70).stop 2024-04-17T22:31:46.479 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.492 7f8779573700 1 -- 172.21.15.102:0/17147861 reap_dead start 2024-04-17T22:31:46.479 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.492 7f877bf77700 1 -- 172.21.15.102:0/17147861 shutdown_connections 2024-04-17T22:31:46.479 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.492 7f877bf77700 1 -- 172.21.15.102:0/17147861 >> 172.21.15.102:0/17147861 conn(0x7f87740cbd50 msgr2=0x7f87740c39a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:46.479 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.492 7f877bf77700 1 -- 172.21.15.102:0/17147861 shutdown_connections 2024-04-17T22:31:46.479 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.492 7f877bf77700 1 -- 172.21.15.102:0/17147861 wait complete. 2024-04-17T22:31:46.479 INFO:teuthology.orchestra.run.smithi102.stderr:Set osd_debug_deep_scrub_sleep to 0 2024-04-17T22:31:46.489 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-04-17T22:31:46.616 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.628 7f60a2c0f700 1 Processor -- start 2024-04-17T22:31:46.617 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.628 7f60a2c0f700 1 -- start start 2024-04-17T22:31:46.617 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.628 7f60a2c0f700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f609c0f5840 0x7f609c10b010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:46.617 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.628 7f60a2c0f700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f609c10b640 0x7f609c19faf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:46.617 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.629 7f60a2c0f700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f609c10ba90 0x7f609c1a0120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:46.617 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.629 7f60a2c0f700 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f609c10bee0 con 0x7f609c10ba90 2024-04-17T22:31:46.617 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.629 7f60a2c0f700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f609c0fa370 con 0x7f609c0f5840 2024-04-17T22:31:46.617 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.629 7f60a2c0f700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f609c103b40 con 0x7f609c10b640 2024-04-17T22:31:46.617 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.629 7f609affd700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f609c10b640 0x7f609c19faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:46.617 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.629 7f609bfff700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f609c10ba90 0x7f609c1a0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:46.617 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.629 7f609b7fe700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f609c0f5840 0x7f609c10b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:46.617 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.629 7f609affd700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f609c10b640 0x7f609c19faf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:34514/0 (socket says 172.21.15.102:34514) 2024-04-17T22:31:46.617 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.629 7f609bfff700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f609c10ba90 0x7f609c1a0120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:34788/0 (socket says 172.21.15.102:34788) 2024-04-17T22:31:46.617 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.629 7f609affd700 1 -- 172.21.15.102:0/2840724369 learned_addr learned my addr 172.21.15.102:0/2840724369 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:31:46.618 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.629 7f609affd700 1 -- 172.21.15.102:0/2840724369 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f609c0f5840 msgr2=0x7f609c10b010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:31:46.618 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.629 7f609affd700 1 --2- 172.21.15.102:0/2840724369 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f609c0f5840 0x7f609c10b010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:46.618 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.629 7f609affd700 1 -- 172.21.15.102:0/2840724369 >> v2:172.21.15.102:3300/0 conn(0x7f609c10ba90 msgr2=0x7f609c1a0120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:31:46.618 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.629 7f609affd700 1 --2- 172.21.15.102:0/2840724369 >> v2:172.21.15.102:3300/0 conn(0x7f609c10ba90 0x7f609c1a0120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:46.618 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.629 7f609affd700 1 -- 172.21.15.102:0/2840724369 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f609c1a0750 con 0x7f609c10b640 2024-04-17T22:31:46.618 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.629 7f609bfff700 1 --2- 172.21.15.102:0/2840724369 >> v2:172.21.15.102:3300/0 conn(0x7f609c10ba90 0x7f609c1a0120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:46.618 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.629 7f609b7fe700 1 --2- 172.21.15.102:0/2840724369 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f609c0f5840 0x7f609c10b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:46.618 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.629 7f609affd700 1 --2- 172.21.15.102:0/2840724369 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f609c10b640 0x7f609c19faf0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 rx=0x7f609c0fa520 tx=0x7f608c009a00).ready entity=mon.2 client_cookie=ce1686a0996a6fc7 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:46.618 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.629 7f609a7fc700 1 -- 172.21.15.102:0/2840724369 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f609c1a0750 con 0x7f609c10b640 2024-04-17T22:31:46.618 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.629 7f609a7fc700 1 -- 172.21.15.102:0/2840724369 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f608c00f510 con 0x7f609c10b640 2024-04-17T22:31:46.618 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.629 7f609a7fc700 1 -- 172.21.15.102:0/2840724369 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f608c00f6f0 con 0x7f609c10b640 2024-04-17T22:31:46.618 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.629 7f60a2c0f700 1 -- 172.21.15.102:0/2840724369 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f609c10b640 msgr2=0x7f609c19faf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:46.618 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.629 7f60a2c0f700 1 --2- 172.21.15.102:0/2840724369 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f609c10b640 0x7f609c19faf0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 rx=0x7f609c0fa520 tx=0x7f608c009a00).stop 2024-04-17T22:31:46.619 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.630 7f60a2c0f700 1 -- 172.21.15.102:0/2840724369 shutdown_connections 2024-04-17T22:31:46.619 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.630 7f60a2c0f700 1 --2- 172.21.15.102:0/2840724369 >> v2:172.21.15.102:3300/0 conn(0x7f609c10ba90 0x7f609c1a0120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:46.619 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.630 7f60a2c0f700 1 --2- 172.21.15.102:0/2840724369 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f609c0f5840 0x7f609c10b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:46.619 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.630 7f60a2c0f700 1 --2- 172.21.15.102:0/2840724369 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f609c10b640 0x7f609c19faf0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:46.619 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.630 7f60a2c0f700 1 -- 172.21.15.102:0/2840724369 >> 172.21.15.102:0/2840724369 conn(0x7f609c1138a0 msgr2=0x7f609c0f4f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:46.619 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.630 7f60a2c0f700 1 -- 172.21.15.102:0/2840724369 shutdown_connections 2024-04-17T22:31:46.619 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.630 7f60a2c0f700 1 -- 172.21.15.102:0/2840724369 wait complete. 2024-04-17T22:31:46.638 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.652 7f60a2c0f700 1 Processor -- start 2024-04-17T22:31:46.638 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.652 7f60a2c0f700 1 -- start start 2024-04-17T22:31:46.641 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.653 7f60a2c0f700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f609c0dc540 0x7f609c1ab090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:46.641 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.653 7f60a2c0f700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f609c1ab6c0 0x7f609c0cfa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:46.641 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.653 7f60a2c0f700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f609c1abb10 0x7f609c0d0070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:46.641 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.653 7f60a2c0f700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f609c1abf60 con 0x7f609c0dc540 2024-04-17T22:31:46.641 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.653 7f60a2c0f700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f609c0dbee0 con 0x7f609c1ab6c0 2024-04-17T22:31:46.641 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.653 7f60a2c0f700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f609c0d06a0 con 0x7f609c1abb10 2024-04-17T22:31:46.641 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.653 7f609b7fe700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f609c0dc540 0x7f609c1ab090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:46.641 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.653 7f609bfff700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f609c1abb10 0x7f609c0d0070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:46.642 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.653 7f609b7fe700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f609c0dc540 0x7f609c1ab090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:34804/0 (socket says 172.21.15.102:34804) 2024-04-17T22:31:46.642 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.653 7f609bfff700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f609c1abb10 0x7f609c0d0070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:34522/0 (socket says 172.21.15.102:34522) 2024-04-17T22:31:46.642 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.653 7f609b7fe700 1 -- 172.21.15.102:0/2415820706 learned_addr learned my addr 172.21.15.102:0/2415820706 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:31:46.642 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.653 7f609affd700 1 --2- 172.21.15.102:0/2415820706 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f609c1ab6c0 0x7f609c0cfa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:46.642 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.653 7f609b7fe700 1 -- 172.21.15.102:0/2415820706 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f609c1abb10 msgr2=0x7f609c0d0070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:46.642 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.653 7f609b7fe700 1 --2- 172.21.15.102:0/2415820706 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f609c1abb10 0x7f609c0d0070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:46.642 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.653 7f609b7fe700 1 -- 172.21.15.102:0/2415820706 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f609c1ab6c0 msgr2=0x7f609c0cfa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:46.642 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.653 7f609b7fe700 1 --2- 172.21.15.102:0/2415820706 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f609c1ab6c0 0x7f609c0cfa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:46.642 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.653 7f609b7fe700 1 -- 172.21.15.102:0/2415820706 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f609c0d0880 con 0x7f609c0dc540 2024-04-17T22:31:46.642 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.653 7f609bfff700 1 --2- 172.21.15.102:0/2415820706 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f609c1abb10 0x7f609c0d0070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:46.642 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.653 7f609affd700 1 --2- 172.21.15.102:0/2415820706 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f609c1ab6c0 0x7f609c0cfa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:46.642 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.653 7f609b7fe700 1 --2- 172.21.15.102:0/2415820706 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f609c0dc540 0x7f609c1ab090 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 rx=0x7f609000c000 tx=0x7f609000ecd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:46.642 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.653 7f6098ff9700 1 -- 172.21.15.102:0/2415820706 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f609c0d0880 con 0x7f609c0dc540 2024-04-17T22:31:46.642 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.654 7f6098ff9700 1 -- 172.21.15.102:0/2415820706 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f609000c940 con 0x7f609c0dc540 2024-04-17T22:31:46.643 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.654 7f60a2c0f700 1 -- 172.21.15.102:0/2415820706 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f609c1b6d70 con 0x7f609c0dc540 2024-04-17T22:31:46.643 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.654 7f6098ff9700 1 -- 172.21.15.102:0/2415820706 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f609000f040 con 0x7f609c0dc540 2024-04-17T22:31:46.643 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.654 7f60a2c0f700 1 -- 172.21.15.102:0/2415820706 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f609c1b7150 con 0x7f609c0dc540 2024-04-17T22:31:46.643 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.654 7f6098ff9700 1 -- 172.21.15.102:0/2415820706 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7f6090011070 con 0x7f609c0dc540 2024-04-17T22:31:46.643 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.654 7f6098ff9700 1 -- 172.21.15.102:0/2415820706 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 2417+0+0 (secure 0 0 0) 0x7f6090010040 con 0x7f609c0dc540 2024-04-17T22:31:46.643 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.655 7f60a2c0f700 1 -- 172.21.15.102:0/2415820706 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f609c0f0800 con 0x7f609c0dc540 2024-04-17T22:31:46.643 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.658 7f6098ff9700 1 -- 172.21.15.102:0/2415820706 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+110455 (secure 0 0 0) 0x7f609000c540 con 0x7f609c0dc540 2024-04-17T22:31:46.749 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.763 7f60a2c0f700 1 -- 172.21.15.102:0/2415820706 --> v1:172.21.15.102:6816/42360 -- command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mgr", ""]}) v1 -- 0x7f609c08a3b0 con 0x7f607c0145a0 2024-04-17T22:31:46.749 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.763 7f6098ff9700 1 -- 172.21.15.102:0/2415820706 <== mgr.4104 v1:172.21.15.102:6816/42360 1 ==== command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (unknown 2057319476 0 0) 0x7f608c009240 con 0x7f607c0145a0 2024-04-17T22:31:46.751 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.764 7f60a2c0f700 1 -- 172.21.15.102:0/2415820706 >> v1:172.21.15.102:6816/42360 conn(0x7f607c0145a0 legacy=0x7f607c016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:46.752 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.764 7f60a2c0f700 1 -- 172.21.15.102:0/2415820706 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f609c0dc540 msgr2=0x7f609c1ab090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:46.752 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.764 7f60a2c0f700 1 --2- 172.21.15.102:0/2415820706 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f609c0dc540 0x7f609c1ab090 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 rx=0x7f609000c000 tx=0x7f609000ecd0).stop 2024-04-17T22:31:46.752 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.764 7f60a2c0f700 1 -- 172.21.15.102:0/2415820706 shutdown_connections 2024-04-17T22:31:46.752 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.764 7f60a2c0f700 1 --2- 172.21.15.102:0/2415820706 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f609c1abb10 0x7f609c0d0070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:46.752 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.764 7f60a2c0f700 1 --2- 172.21.15.102:0/2415820706 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f609c0dc540 0x7f609c1ab090 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:46.752 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.765 7f60a2c0f700 1 --2- 172.21.15.102:0/2415820706 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f609c1ab6c0 0x7f609c0cfa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:46.753 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.765 7f60a2c0f700 1 -- 172.21.15.102:0/2415820706 >> 172.21.15.102:0/2415820706 conn(0x7f609c0c5970 msgr2=0x7f609c0c46a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:46.753 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.765 7f60a2c0f700 1 -- 172.21.15.102:0/2415820706 shutdown_connections 2024-04-17T22:31:46.753 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.765 7f60a2c0f700 1 -- 172.21.15.102:0/2415820706 wait complete. 2024-04-17T22:31:46.753 INFO:teuthology.orchestra.run.smithi102.stderr:instructed osd(s) 0 to deep-scrub 2024-04-17T22:31:46.762 INFO:tasks.ceph:Scrubbing osd.1 2024-04-17T22:31:46.762 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-04-17T22:31:46.894 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.905 7f9e445f4700 1 Processor -- start 2024-04-17T22:31:46.895 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.905 7f9e445f4700 1 -- start start 2024-04-17T22:31:46.895 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.905 7f9e445f4700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f9e3c0f96d0 0x7f9e3c10e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:46.895 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.905 7f9e445f4700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f9e3c10e7a0 0x7f9e3c0f1790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:46.895 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.905 7f9e445f4700 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e3c0fa0d0 con 0x7f9e3c10ebf0 2024-04-17T22:31:46.895 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.905 7f9e445f4700 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e3c0f25c0 con 0x7f9e3c0f96d0 2024-04-17T22:31:46.895 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.905 7f9e445f4700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e3c1ac4d0 con 0x7f9e3c10e7a0 2024-04-17T22:31:46.895 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.905 7f9e40bee700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f9e3c10e7a0 0x7f9e3c0f1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:46.896 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.905 7f9e413ef700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f9e3c0f96d0 0x7f9e3c10e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:46.896 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.905 7f9e40bee700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f9e3c10e7a0 0x7f9e3c0f1790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:49798/0 (socket says 172.21.15.102:49798) 2024-04-17T22:31:46.896 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.905 7f9e413ef700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f9e3c0f96d0 0x7f9e3c10e170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:34810/0 (socket says 172.21.15.102:34810) 2024-04-17T22:31:46.897 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.905 7f9e40bee700 1 -- 172.21.15.102:0/3682494499 learned_addr learned my addr 172.21.15.102:0/3682494499 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:31:46.897 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.906 7f9e413ef700 1 -- 172.21.15.102:0/3682494499 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f9e3c10e7a0 msgr2=0x7f9e3c0f1790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:31:46.897 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.906 7f9e413ef700 1 --2- 172.21.15.102:0/3682494499 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f9e3c10e7a0 0x7f9e3c0f1790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:46.897 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.906 7f9e413ef700 1 -- 172.21.15.102:0/3682494499 >> v1:172.21.15.102:6789/0 conn(0x7f9e3c10ebf0 legacy=0x7f9e3c0f1dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:46.897 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.906 7f9e413ef700 1 -- 172.21.15.102:0/3682494499 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e3c1ac680 con 0x7f9e3c0f96d0 2024-04-17T22:31:46.897 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.906 7f9e3bfff700 1 -- 172.21.15.102:0/3682494499 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1456801471 0 0) 0x7f9e2c001110 con 0x7f9e3c10ebf0 2024-04-17T22:31:46.898 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.906 7f9e40bee700 1 --2- 172.21.15.102:0/3682494499 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f9e3c10e7a0 0x7f9e3c0f1790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:46.898 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.906 7f9e413ef700 1 --2- 172.21.15.102:0/3682494499 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9e3c0f96d0 0x7f9e3c10e170 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 rx=0x7f9e340032f0 tx=0x7f9e34009d10).ready entity=mon.0 client_cookie=2ac7d884a19112fb server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:46.898 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.906 7f9e3bfff700 1 -- 172.21.15.102:0/3682494499 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9e3c1ac680 con 0x7f9e3c0f96d0 2024-04-17T22:31:46.898 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.906 7f9e3bfff700 1 -- 172.21.15.102:0/3682494499 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e3400b7a0 con 0x7f9e3c0f96d0 2024-04-17T22:31:46.898 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.906 7f9e3bfff700 1 -- 172.21.15.102:0/3682494499 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9e3400b980 con 0x7f9e3c0f96d0 2024-04-17T22:31:46.898 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.906 7f9e445f4700 1 -- 172.21.15.102:0/3682494499 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9e3c0f96d0 msgr2=0x7f9e3c10e170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:46.899 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.906 7f9e445f4700 1 --2- 172.21.15.102:0/3682494499 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9e3c0f96d0 0x7f9e3c10e170 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 rx=0x7f9e340032f0 tx=0x7f9e34009d10).stop 2024-04-17T22:31:46.899 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.906 7f9e445f4700 1 -- 172.21.15.102:0/3682494499 shutdown_connections 2024-04-17T22:31:46.899 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.906 7f9e445f4700 1 --2- 172.21.15.102:0/3682494499 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9e3c0f96d0 0x7f9e3c10e170 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:46.899 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.906 7f9e445f4700 1 --2- 172.21.15.102:0/3682494499 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f9e3c10e7a0 0x7f9e3c0f1790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:46.899 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.906 7f9e445f4700 1 -- 172.21.15.102:0/3682494499 >> 172.21.15.102:0/3682494499 conn(0x7f9e3c1138d0 msgr2=0x7f9e3c0f4fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:46.899 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.907 7f9e445f4700 1 -- 172.21.15.102:0/3682494499 shutdown_connections 2024-04-17T22:31:46.899 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.907 7f9e445f4700 1 -- 172.21.15.102:0/3682494499 wait complete. 2024-04-17T22:31:46.918 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.929 7f9e445f4700 1 Processor -- start 2024-04-17T22:31:46.918 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.929 7f9e445f4700 1 -- start start 2024-04-17T22:31:46.918 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.929 7f9e445f4700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f9e3c0c45d0 0x7f9e3c0d2870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:46.918 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.929 7f9e445f4700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f9e3c0d2ea0 0x7f9e3c0d1400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:46.918 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.930 7f9e445f4700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9e3c0d32f0 0x7f9e3c0d1a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:46.919 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.930 7f9e445f4700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e3c0efc30 con 0x7f9e3c0d32f0 2024-04-17T22:31:46.919 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.930 7f9e445f4700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e3c0d54e0 con 0x7f9e3c0d2ea0 2024-04-17T22:31:46.919 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.930 7f9e445f4700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9e3c0c5370 con 0x7f9e3c0c45d0 2024-04-17T22:31:46.919 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.930 7f9e40bee700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f9e3c0d2ea0 0x7f9e3c0d1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:46.919 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.930 7f9e40bee700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f9e3c0d2ea0 0x7f9e3c0d1400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:49806/0 (socket says 172.21.15.102:49806) 2024-04-17T22:31:46.919 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.930 7f9e40bee700 1 -- 172.21.15.102:0/2333614479 learned_addr learned my addr 172.21.15.102:0/2333614479 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:31:46.919 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.930 7f9e413ef700 1 --2- 172.21.15.102:0/2333614479 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f9e3c0c45d0 0x7f9e3c0d2870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:46.919 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.930 7f9e41bf0700 1 --2- 172.21.15.102:0/2333614479 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9e3c0d32f0 0x7f9e3c0d1a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:46.919 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.930 7f9e41bf0700 1 -- 172.21.15.102:0/2333614479 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f9e3c0c45d0 msgr2=0x7f9e3c0d2870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:46.919 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.930 7f9e41bf0700 1 --2- 172.21.15.102:0/2333614479 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f9e3c0c45d0 0x7f9e3c0d2870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:46.919 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.930 7f9e41bf0700 1 -- 172.21.15.102:0/2333614479 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f9e3c0d2ea0 msgr2=0x7f9e3c0d1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:46.920 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.930 7f9e41bf0700 1 --2- 172.21.15.102:0/2333614479 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f9e3c0d2ea0 0x7f9e3c0d1400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:46.920 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.930 7f9e41bf0700 1 -- 172.21.15.102:0/2333614479 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e3c0d2060 con 0x7f9e3c0d32f0 2024-04-17T22:31:46.920 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.930 7f9e40bee700 1 --2- 172.21.15.102:0/2333614479 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f9e3c0d2ea0 0x7f9e3c0d1400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:46.920 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.930 7f9e413ef700 1 --2- 172.21.15.102:0/2333614479 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f9e3c0c45d0 0x7f9e3c0d2870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:46.920 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.930 7f9e41bf0700 1 --2- 172.21.15.102:0/2333614479 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9e3c0d32f0 0x7f9e3c0d1a30 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 rx=0x7f9e3c0e0d80 tx=0x7f9e2c00c610).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:46.920 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.930 7f9e3a7fc700 1 -- 172.21.15.102:0/2333614479 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9e3c0d2060 con 0x7f9e3c0d32f0 2024-04-17T22:31:46.920 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.930 7f9e3a7fc700 1 -- 172.21.15.102:0/2333614479 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e2c007030 con 0x7f9e3c0d32f0 2024-04-17T22:31:46.920 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.930 7f9e445f4700 1 -- 172.21.15.102:0/2333614479 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e3c1bb470 con 0x7f9e3c0d32f0 2024-04-17T22:31:46.920 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.930 7f9e3a7fc700 1 -- 172.21.15.102:0/2333614479 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f9e2c006520 con 0x7f9e3c0d32f0 2024-04-17T22:31:46.920 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.930 7f9e445f4700 1 -- 172.21.15.102:0/2333614479 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e3c1bb850 con 0x7f9e3c0d32f0 2024-04-17T22:31:46.920 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.931 7f9e3a7fc700 1 -- 172.21.15.102:0/2333614479 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7f9e2c01a020 con 0x7f9e3c0d32f0 2024-04-17T22:31:46.921 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.931 7f9e3a7fc700 1 -- 172.21.15.102:0/2333614479 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 2417+0+0 (secure 0 0 0) 0x7f9e2c01a760 con 0x7f9e3c0d32f0 2024-04-17T22:31:46.921 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.931 7f9e445f4700 1 -- 172.21.15.102:0/2333614479 --> v1:172.21.15.102:6808/42958 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9e3c0878c0 con 0x7f9e3c0e58c0 2024-04-17T22:31:46.921 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.932 7f9e3a7fc700 1 -- 172.21.15.102:0/2333614479 <== osd.1 v1:172.21.15.102:6808/42958 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+5151 (unknown 0 0 4057655604) 0x7f9e30009b30 con 0x7f9e3c0e58c0 2024-04-17T22:31:46.924 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.938 7f9e445f4700 1 -- 172.21.15.102:0/2333614479 --> v1:172.21.15.102:6808/42958 -- command(tid 2: {"prefix": "config set", "key": "osd_debug_deep_scrub_sleep", "value": "0"}) v1 -- 0x7f9e3c0f0860 con 0x7f9e3c0e58c0 2024-04-17T22:31:46.924 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.938 7f9e3a7fc700 1 -- 172.21.15.102:0/2333614479 <== osd.1 v1:172.21.15.102:6808/42958 2 ==== command_reply(tid 2: 0 Set osd_debug_deep_scrub_sleep to 0) v1 ==== 43+0+0 (unknown 1994953735 0 0) 0x7f9e3c0f0860 con 0x7f9e3c0e58c0 2024-04-17T22:31:46.925 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.938 7f9e445f4700 1 -- 172.21.15.102:0/2333614479 >> v1:172.21.15.102:6808/42958 conn(0x7f9e3c0e58c0 legacy=0x7f9e3c0a7bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:46.925 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.938 7f9e445f4700 1 -- 172.21.15.102:0/2333614479 >> v1:172.21.15.102:6816/42360 conn(0x7f9e1c0145a0 legacy=0x7f9e1c016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:46.925 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.938 7f9e445f4700 1 -- 172.21.15.102:0/2333614479 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9e3c0d32f0 msgr2=0x7f9e3c0d1a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:46.925 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.938 7f9e445f4700 1 --2- 172.21.15.102:0/2333614479 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9e3c0d32f0 0x7f9e3c0d1a30 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 rx=0x7f9e3c0e0d80 tx=0x7f9e2c00c610).stop 2024-04-17T22:31:46.925 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.938 7f9e41bf0700 1 -- 172.21.15.102:0/2333614479 reap_dead start 2024-04-17T22:31:46.925 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.938 7f9e445f4700 1 -- 172.21.15.102:0/2333614479 shutdown_connections 2024-04-17T22:31:46.925 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.938 7f9e445f4700 1 -- 172.21.15.102:0/2333614479 >> 172.21.15.102:0/2333614479 conn(0x7f9e3c0c5790 msgr2=0x7f9e3c0bd6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:46.925 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.938 7f9e445f4700 1 -- 172.21.15.102:0/2333614479 shutdown_connections 2024-04-17T22:31:46.925 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:46.938 7f9e445f4700 1 -- 172.21.15.102:0/2333614479 wait complete. 2024-04-17T22:31:46.926 INFO:teuthology.orchestra.run.smithi102.stderr:Set osd_debug_deep_scrub_sleep to 0 2024-04-17T22:31:46.935 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-04-17T22:31:47.062 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.074 7fb257f57700 1 Processor -- start 2024-04-17T22:31:47.062 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.074 7fb257f57700 1 -- start start 2024-04-17T22:31:47.063 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.075 7fb257f57700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fb2501a6440 0x7fb2501a88a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:47.063 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.075 7fb257f57700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb2501a8ed0 0x7fb2501ab330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:47.063 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.075 7fb257f57700 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb25010ee20 con 0x7fb25010e1d0 2024-04-17T22:31:47.063 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.075 7fb257f57700 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2500f9ea0 con 0x7fb2501a6440 2024-04-17T22:31:47.063 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.075 7fb257f57700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb2500f8fc0 con 0x7fb2501a8ed0 2024-04-17T22:31:47.063 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.075 7fb24ffff700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fb2501a6440 0x7fb2501a88a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:47.063 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.075 7fb255553700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb2501a8ed0 0x7fb2501ab330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:47.063 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.075 7fb255553700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb2501a8ed0 0x7fb2501ab330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:34542/0 (socket says 172.21.15.102:34542) 2024-04-17T22:31:47.063 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.075 7fb24ffff700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fb2501a6440 0x7fb2501a88a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:34820/0 (socket says 172.21.15.102:34820) 2024-04-17T22:31:47.063 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.075 7fb255553700 1 -- 172.21.15.102:0/164360659 learned_addr learned my addr 172.21.15.102:0/164360659 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:31:47.063 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.075 7fb254d52700 1 --1- 172.21.15.102:0/164360659 >> v1:172.21.15.102:6789/0 conn(0x7fb25010e1d0 0x7fb25010e620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:47880/0 (socket says 172.21.15.102:47880) 2024-04-17T22:31:47.063 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.075 7fb255553700 1 -- 172.21.15.102:0/164360659 >> v2:172.21.15.102:3300/0 conn(0x7fb2501a6440 msgr2=0x7fb2501a88a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:31:47.063 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.075 7fb255553700 1 --2- 172.21.15.102:0/164360659 >> v2:172.21.15.102:3300/0 conn(0x7fb2501a6440 0x7fb2501a88a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:47.064 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.075 7fb255553700 1 -- 172.21.15.102:0/164360659 >> v1:172.21.15.102:6789/0 conn(0x7fb25010e1d0 legacy=0x7fb25010e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:47.064 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.075 7fb255553700 1 -- 172.21.15.102:0/164360659 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2501ac970 con 0x7fb2501a8ed0 2024-04-17T22:31:47.064 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.075 7fb24ffff700 1 --2- 172.21.15.102:0/164360659 >> v2:172.21.15.102:3300/0 conn(0x7fb2501a6440 0x7fb2501a88a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:47.064 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.075 7fb255553700 1 --2- 172.21.15.102:0/164360659 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb2501a8ed0 0x7fb2501ab330 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 rx=0x7fb2500f4400 tx=0x7fb248009d90).ready entity=mon.2 client_cookie=9768c0665ebe77fb server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:47.064 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.075 7fb24f7fe700 1 -- 172.21.15.102:0/164360659 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb2501ac970 con 0x7fb2501a8ed0 2024-04-17T22:31:47.064 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.075 7fb24f7fe700 1 -- 172.21.15.102:0/164360659 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb24800b840 con 0x7fb2501a8ed0 2024-04-17T22:31:47.064 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.075 7fb24f7fe700 1 -- 172.21.15.102:0/164360659 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb24800ba20 con 0x7fb2501a8ed0 2024-04-17T22:31:47.064 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.075 7fb257f57700 1 -- 172.21.15.102:0/164360659 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb2501a8ed0 msgr2=0x7fb2501ab330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:47.064 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.075 7fb257f57700 1 --2- 172.21.15.102:0/164360659 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb2501a8ed0 0x7fb2501ab330 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 rx=0x7fb2500f4400 tx=0x7fb248009d90).stop 2024-04-17T22:31:47.064 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.076 7fb257f57700 1 -- 172.21.15.102:0/164360659 shutdown_connections 2024-04-17T22:31:47.064 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.076 7fb257f57700 1 --2- 172.21.15.102:0/164360659 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb2501a8ed0 0x7fb2501ab330 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:47.064 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.076 7fb257f57700 1 --2- 172.21.15.102:0/164360659 >> v2:172.21.15.102:3300/0 conn(0x7fb2501a6440 0x7fb2501a88a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:47.064 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.076 7fb257f57700 1 -- 172.21.15.102:0/164360659 >> 172.21.15.102:0/164360659 conn(0x7fb250113830 msgr2=0x7fb2500f4ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:47.064 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.076 7fb257f57700 1 -- 172.21.15.102:0/164360659 shutdown_connections 2024-04-17T22:31:47.065 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.076 7fb257f57700 1 -- 172.21.15.102:0/164360659 wait complete. 2024-04-17T22:31:47.085 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.099 7fb257f57700 1 Processor -- start 2024-04-17T22:31:47.085 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.099 7fb257f57700 1 -- start start 2024-04-17T22:31:47.087 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.099 7fb257f57700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb2500cc140 0x7fb2500d5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:47.087 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.099 7fb257f57700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb2500d5890 0x7fb2501b3fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:47.087 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.099 7fb257f57700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb2500d5ce0 0x7fb2501b45d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:47.088 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.099 7fb257f57700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb25008a9b0 con 0x7fb2500d5ce0 2024-04-17T22:31:47.088 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.099 7fb257f57700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2500fa140 con 0x7fb2500d5890 2024-04-17T22:31:47.088 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.099 7fb257f57700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb2501b4c00 con 0x7fb2500cc140 2024-04-17T22:31:47.088 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.100 7fb254d52700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb2500cc140 0x7fb2500d5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:47.088 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.100 7fb255553700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb2500d5ce0 0x7fb2501b45d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:47.088 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.100 7fb254d52700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb2500cc140 0x7fb2500d5260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:34558/0 (socket says 172.21.15.102:34558) 2024-04-17T22:31:47.088 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.100 7fb255553700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb2500d5ce0 0x7fb2501b45d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:34832/0 (socket says 172.21.15.102:34832) 2024-04-17T22:31:47.088 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.100 7fb254d52700 1 -- 172.21.15.102:0/260722720 learned_addr learned my addr 172.21.15.102:0/260722720 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:31:47.088 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.100 7fb24ffff700 1 --2- 172.21.15.102:0/260722720 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb2500d5890 0x7fb2501b3fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:47.088 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.100 7fb24ffff700 1 -- 172.21.15.102:0/260722720 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb2500cc140 msgr2=0x7fb2500d5260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:47.088 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.100 7fb24ffff700 1 --2- 172.21.15.102:0/260722720 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb2500cc140 0x7fb2500d5260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:47.088 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.100 7fb24ffff700 1 -- 172.21.15.102:0/260722720 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb2500d5ce0 msgr2=0x7fb2501b45d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:47.088 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.100 7fb24ffff700 1 --2- 172.21.15.102:0/260722720 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb2500d5ce0 0x7fb2501b45d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:47.088 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.100 7fb24ffff700 1 -- 172.21.15.102:0/260722720 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2501b4e30 con 0x7fb2500d5890 2024-04-17T22:31:47.089 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.100 7fb255553700 1 --2- 172.21.15.102:0/260722720 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb2500d5ce0 0x7fb2501b45d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:47.089 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.100 7fb254d52700 1 --2- 172.21.15.102:0/260722720 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb2500cc140 0x7fb2500d5260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:47.089 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.100 7fb24ffff700 1 --2- 172.21.15.102:0/260722720 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb2500d5890 0x7fb2501b3fa0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 rx=0x7fb2500d73a0 tx=0x7fb23800ecd0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:47.089 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.100 7fb24dffb700 1 -- 172.21.15.102:0/260722720 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb2501b4e30 con 0x7fb2500d5890 2024-04-17T22:31:47.089 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.100 7fb24dffb700 1 -- 172.21.15.102:0/260722720 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb23800c790 con 0x7fb2500d5890 2024-04-17T22:31:47.089 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.100 7fb257f57700 1 -- 172.21.15.102:0/260722720 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2501bb710 con 0x7fb2500d5890 2024-04-17T22:31:47.089 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.100 7fb24dffb700 1 -- 172.21.15.102:0/260722720 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb23800f040 con 0x7fb2500d5890 2024-04-17T22:31:47.089 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.100 7fb257f57700 1 -- 172.21.15.102:0/260722720 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2501bbaf0 con 0x7fb2500d5890 2024-04-17T22:31:47.089 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.101 7fb24dffb700 1 -- 172.21.15.102:0/260722720 <== mon.1 v2:172.21.15.102:3301/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7fb238011070 con 0x7fb2500d5890 2024-04-17T22:31:47.089 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.101 7fb24dffb700 1 -- 172.21.15.102:0/260722720 <== mon.1 v2:172.21.15.102:3301/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 2417+0+0 (secure 0 0 0) 0x7fb238010040 con 0x7fb2500d5890 2024-04-17T22:31:47.089 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.101 7fb257f57700 1 -- 172.21.15.102:0/260722720 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2500f0790 con 0x7fb2500d5890 2024-04-17T22:31:47.090 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.104 7fb24dffb700 1 -- 172.21.15.102:0/260722720 <== mon.1 v2:172.21.15.102:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+110455 (secure 0 0 0) 0x7fb23800ee10 con 0x7fb2500d5890 2024-04-17T22:31:47.195 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.209 7fb257f57700 1 -- 172.21.15.102:0/260722720 --> v1:172.21.15.102:6816/42360 -- command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mgr", ""]}) v1 -- 0x7fb2501ae6c0 con 0x7fb2340145a0 2024-04-17T22:31:47.195 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.209 7fb24dffb700 1 -- 172.21.15.102:0/260722720 <== mgr.4104 v1:172.21.15.102:6816/42360 1 ==== command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (unknown 489605429 0 0) 0x7fb2501ae6c0 con 0x7fb2340145a0 2024-04-17T22:31:47.196 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.210 7fb257f57700 1 -- 172.21.15.102:0/260722720 >> v1:172.21.15.102:6816/42360 conn(0x7fb2340145a0 legacy=0x7fb234016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:47.197 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.210 7fb257f57700 1 -- 172.21.15.102:0/260722720 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb2500d5890 msgr2=0x7fb2501b3fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:47.197 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.210 7fb257f57700 1 --2- 172.21.15.102:0/260722720 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb2500d5890 0x7fb2501b3fa0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 rx=0x7fb2500d73a0 tx=0x7fb23800ecd0).stop 2024-04-17T22:31:47.197 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.210 7fb257f57700 1 -- 172.21.15.102:0/260722720 shutdown_connections 2024-04-17T22:31:47.197 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.210 7fb257f57700 1 --2- 172.21.15.102:0/260722720 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb2500d5ce0 0x7fb2501b45d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:47.197 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.210 7fb257f57700 1 --2- 172.21.15.102:0/260722720 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb2500cc140 0x7fb2500d5260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:47.197 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.210 7fb257f57700 1 --2- 172.21.15.102:0/260722720 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb2500d5890 0x7fb2501b3fa0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:47.197 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.210 7fb257f57700 1 -- 172.21.15.102:0/260722720 >> 172.21.15.102:0/260722720 conn(0x7fb2500c6a60 msgr2=0x7fb2500d6340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:47.197 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.210 7fb257f57700 1 -- 172.21.15.102:0/260722720 shutdown_connections 2024-04-17T22:31:47.197 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.210 7fb257f57700 1 -- 172.21.15.102:0/260722720 wait complete. 2024-04-17T22:31:47.198 INFO:teuthology.orchestra.run.smithi102.stderr:instructed osd(s) 1 to deep-scrub 2024-04-17T22:31:47.207 INFO:tasks.ceph:Scrubbing osd.2 2024-04-17T22:31:47.207 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-04-17T22:31:47.338 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.350 7eff2fa88700 1 Processor -- start 2024-04-17T22:31:47.338 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.350 7eff2fa88700 1 -- start start 2024-04-17T22:31:47.338 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.350 7eff2fa88700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7eff280f54c0 0x7eff2810b440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:47.338 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.350 7eff2fa88700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7eff281a4ef0 0x7eff281a7350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:47.338 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.350 7eff2fa88700 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff281a07a0 con 0x7eff281050d0 2024-04-17T22:31:47.338 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.350 7eff2fa88700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff2810ba70 con 0x7eff281a4ef0 2024-04-17T22:31:47.338 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.350 7eff2fa88700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7eff281a8990 con 0x7eff280f54c0 2024-04-17T22:31:47.338 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.350 7eff27fff700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7eff280f54c0 0x7eff2810b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:47.338 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.350 7eff2c883700 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7eff281050d0 0x7eff2810ac40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:47882/0 (socket says 172.21.15.102:47882) 2024-04-17T22:31:47.339 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.350 7eff2c883700 1 -- 172.21.15.102:0/341936317 learned_addr learned my addr 172.21.15.102:0/341936317 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-17T22:31:47.339 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.350 7eff2d084700 1 --2- 172.21.15.102:0/341936317 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7eff281a4ef0 0x7eff281a7350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:47.339 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.351 7eff27fff700 1 -- 172.21.15.102:0/341936317 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7eff281a4ef0 msgr2=0x7eff281a7350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:31:47.339 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.351 7eff27fff700 1 --2- 172.21.15.102:0/341936317 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7eff281a4ef0 0x7eff281a7350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:47.339 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.351 7eff27fff700 1 -- 172.21.15.102:0/341936317 >> v1:172.21.15.102:6789/0 conn(0x7eff281050d0 legacy=0x7eff2810ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:47.339 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.351 7eff277fe700 1 -- 172.21.15.102:0/341936317 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 864055385 0 0) 0x7eff1c001110 con 0x7eff281050d0 2024-04-17T22:31:47.339 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.351 7eff27fff700 1 -- 172.21.15.102:0/341936317 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff281a8b40 con 0x7eff280f54c0 2024-04-17T22:31:47.339 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.351 7eff2d084700 1 --2- 172.21.15.102:0/341936317 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7eff281a4ef0 0x7eff281a7350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:47.339 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.351 7eff27fff700 1 --2- 172.21.15.102:0/341936317 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7eff280f54c0 0x7eff2810b440 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 rx=0x7eff14002f70 tx=0x7eff1400b9a0).ready entity=mon.2 client_cookie=1fc7045786c0abb0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:47.339 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.351 7eff277fe700 1 -- 172.21.15.102:0/341936317 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7eff281a8b40 con 0x7eff280f54c0 2024-04-17T22:31:47.339 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.351 7eff277fe700 1 -- 172.21.15.102:0/341936317 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff1400d570 con 0x7eff280f54c0 2024-04-17T22:31:47.339 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.351 7eff277fe700 1 -- 172.21.15.102:0/341936317 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7eff1400d940 con 0x7eff280f54c0 2024-04-17T22:31:47.339 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.351 7eff2fa88700 1 -- 172.21.15.102:0/341936317 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7eff280f54c0 msgr2=0x7eff2810b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:47.339 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.351 7eff2fa88700 1 --2- 172.21.15.102:0/341936317 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7eff280f54c0 0x7eff2810b440 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 rx=0x7eff14002f70 tx=0x7eff1400b9a0).stop 2024-04-17T22:31:47.340 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.352 7eff2fa88700 1 -- 172.21.15.102:0/341936317 shutdown_connections 2024-04-17T22:31:47.340 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.352 7eff2fa88700 1 --2- 172.21.15.102:0/341936317 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7eff281a4ef0 0x7eff281a7350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:47.340 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.352 7eff2fa88700 1 --2- 172.21.15.102:0/341936317 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7eff280f54c0 0x7eff2810b440 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:47.340 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.352 7eff2fa88700 1 -- 172.21.15.102:0/341936317 >> 172.21.15.102:0/341936317 conn(0x7eff281135a0 msgr2=0x7eff280f4c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:47.340 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.352 7eff2fa88700 1 -- 172.21.15.102:0/341936317 shutdown_connections 2024-04-17T22:31:47.340 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.352 7eff2fa88700 1 -- 172.21.15.102:0/341936317 wait complete. 2024-04-17T22:31:47.360 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.374 7eff2fa88700 1 Processor -- start 2024-04-17T22:31:47.360 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.374 7eff2fa88700 1 -- start start 2024-04-17T22:31:47.363 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.375 7eff2fa88700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7eff280ce290 0x7eff280ce6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:47.363 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.375 7eff2fa88700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7eff280ced10 0x7eff281b17c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:47.363 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.375 7eff2fa88700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7eff281b1df0 0x7eff281b2240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:47.363 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.375 7eff2fa88700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff280e6060 con 0x7eff281b1df0 2024-04-17T22:31:47.363 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.375 7eff2fa88700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff280d77d0 con 0x7eff280ced10 2024-04-17T22:31:47.363 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.375 7eff2fa88700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7eff281b2870 con 0x7eff280ce290 2024-04-17T22:31:47.363 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.375 7eff2c883700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7eff280ce290 0x7eff280ce6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:47.363 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.375 7eff27fff700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7eff280ced10 0x7eff281b17c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:47.364 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.375 7eff2c883700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7eff280ce290 0x7eff280ce6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:34570/0 (socket says 172.21.15.102:34570) 2024-04-17T22:31:47.364 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.375 7eff27fff700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7eff280ced10 0x7eff281b17c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:49840/0 (socket says 172.21.15.102:49840) 2024-04-17T22:31:47.364 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.375 7eff2c883700 1 -- 172.21.15.102:0/3669986509 learned_addr learned my addr 172.21.15.102:0/3669986509 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:31:47.364 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.375 7eff2d084700 1 --2- 172.21.15.102:0/3669986509 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7eff281b1df0 0x7eff281b2240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:47.364 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.375 7eff2c883700 1 -- 172.21.15.102:0/3669986509 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7eff280ced10 msgr2=0x7eff281b17c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:47.364 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.375 7eff2c883700 1 --2- 172.21.15.102:0/3669986509 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7eff280ced10 0x7eff281b17c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:47.364 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.375 7eff2c883700 1 -- 172.21.15.102:0/3669986509 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7eff281b1df0 msgr2=0x7eff281b2240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:47.364 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.375 7eff2c883700 1 --2- 172.21.15.102:0/3669986509 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7eff281b1df0 0x7eff281b2240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:47.364 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.375 7eff2c883700 1 -- 172.21.15.102:0/3669986509 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff280deea0 con 0x7eff280ce290 2024-04-17T22:31:47.364 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.375 7eff2d084700 1 --2- 172.21.15.102:0/3669986509 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7eff281b1df0 0x7eff281b2240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:47.364 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.375 7eff27fff700 1 --2- 172.21.15.102:0/3669986509 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7eff280ced10 0x7eff281b17c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:47.364 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.375 7eff2c883700 1 --2- 172.21.15.102:0/3669986509 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7eff280ce290 0x7eff280ce6e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 rx=0x7eff280beb20 tx=0x7eff1c00c810).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:47.364 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.375 7eff25ffb700 1 -- 172.21.15.102:0/3669986509 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7eff280deea0 con 0x7eff280ce290 2024-04-17T22:31:47.365 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.375 7eff25ffb700 1 -- 172.21.15.102:0/3669986509 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff1c0072e0 con 0x7eff280ce290 2024-04-17T22:31:47.365 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.375 7eff2fa88700 1 -- 172.21.15.102:0/3669986509 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff280df680 con 0x7eff280ce290 2024-04-17T22:31:47.365 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.375 7eff25ffb700 1 -- 172.21.15.102:0/3669986509 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7eff1c005810 con 0x7eff280ce290 2024-04-17T22:31:47.365 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.375 7eff2fa88700 1 -- 172.21.15.102:0/3669986509 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff280bbb80 con 0x7eff280ce290 2024-04-17T22:31:47.365 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.376 7eff25ffb700 1 -- 172.21.15.102:0/3669986509 <== mon.2 v2:172.21.15.102:3302/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7eff1c005a10 con 0x7eff280ce290 2024-04-17T22:31:47.365 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.376 7eff25ffb700 1 -- 172.21.15.102:0/3669986509 <== mon.2 v2:172.21.15.102:3302/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 2417+0+0 (secure 0 0 0) 0x7eff1c0146e0 con 0x7eff280ce290 2024-04-17T22:31:47.365 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.376 7eff2fa88700 1 -- 172.21.15.102:0/3669986509 --> v1:172.21.15.102:6804/42954 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7eff280878c0 con 0x7eff280a7bd0 2024-04-17T22:31:47.365 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.377 7eff25ffb700 1 -- 172.21.15.102:0/3669986509 <== osd.2 v1:172.21.15.102:6804/42954 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+5151 (unknown 0 0 4057655604) 0x7eff20009dd0 con 0x7eff280a7bd0 2024-04-17T22:31:47.368 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.383 7eff2fa88700 1 -- 172.21.15.102:0/3669986509 --> v1:172.21.15.102:6804/42954 -- command(tid 2: {"prefix": "config set", "key": "osd_debug_deep_scrub_sleep", "value": "0"}) v1 -- 0x7eff280f0500 con 0x7eff280a7bd0 2024-04-17T22:31:47.369 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.383 7eff25ffb700 1 -- 172.21.15.102:0/3669986509 <== osd.2 v1:172.21.15.102:6804/42954 2 ==== command_reply(tid 2: 0 Set osd_debug_deep_scrub_sleep to 0) v1 ==== 43+0+0 (unknown 1994953735 0 0) 0x7eff280f0500 con 0x7eff280a7bd0 2024-04-17T22:31:47.370 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.383 7eff2fa88700 1 -- 172.21.15.102:0/3669986509 >> v1:172.21.15.102:6804/42954 conn(0x7eff280a7bd0 legacy=0x7eff280a6f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:47.370 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.383 7eff2fa88700 1 -- 172.21.15.102:0/3669986509 >> v1:172.21.15.102:6816/42360 conn(0x7eff100145a0 legacy=0x7eff10016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:47.370 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.383 7eff2fa88700 1 -- 172.21.15.102:0/3669986509 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7eff280ce290 msgr2=0x7eff280ce6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:47.370 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.383 7eff2fa88700 1 --2- 172.21.15.102:0/3669986509 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7eff280ce290 0x7eff280ce6e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 rx=0x7eff280beb20 tx=0x7eff1c00c810).stop 2024-04-17T22:31:47.370 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.383 7eff2d084700 1 -- 172.21.15.102:0/3669986509 reap_dead start 2024-04-17T22:31:47.370 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.383 7eff2fa88700 1 -- 172.21.15.102:0/3669986509 shutdown_connections 2024-04-17T22:31:47.370 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.383 7eff2fa88700 1 -- 172.21.15.102:0/3669986509 >> 172.21.15.102:0/3669986509 conn(0x7eff280f1560 msgr2=0x7eff280dd920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:47.371 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.383 7eff2fa88700 1 -- 172.21.15.102:0/3669986509 shutdown_connections 2024-04-17T22:31:47.371 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.383 7eff2fa88700 1 -- 172.21.15.102:0/3669986509 wait complete. 2024-04-17T22:31:47.371 INFO:teuthology.orchestra.run.smithi102.stderr:Set osd_debug_deep_scrub_sleep to 0 2024-04-17T22:31:47.380 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-04-17T22:31:47.508 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.519 7faec7938700 1 Processor -- start 2024-04-17T22:31:47.508 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.520 7faec7938700 1 -- start start 2024-04-17T22:31:47.508 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.520 7faec7938700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7faec00f5750 0x7faec010c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:47.508 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.520 7faec7938700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7faec010c720 0x7faec01a0be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:47.508 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.520 7faec7938700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7faec010cb70 0x7faec01a1210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:47.508 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.520 7faec7938700 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faec00f93c0 con 0x7faec010cb70 2024-04-17T22:31:47.508 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.520 7faec7938700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faec01a1840 con 0x7faec010c720 2024-04-17T22:31:47.508 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.520 7faec7938700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7faec01a1a70 con 0x7faec00f5750 2024-04-17T22:31:47.508 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.520 7faebffff700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7faec00f5750 0x7faec010c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:47.508 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.520 7faeb7fff700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7faec010c720 0x7faec01a0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:47.509 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.520 7faec4f34700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7faec010cb70 0x7faec01a1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:47.509 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.520 7faebffff700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7faec00f5750 0x7faec010c0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:34586/0 (socket says 172.21.15.102:34586) 2024-04-17T22:31:47.509 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.520 7faeb7fff700 1 --2- 172.21.15.102:0/3552086751 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7faec010c720 0x7faec01a0be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:49842/0 (socket says 172.21.15.102:49842) 2024-04-17T22:31:47.509 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.520 7faebffff700 1 -- 172.21.15.102:0/3552086751 learned_addr learned my addr 172.21.15.102:0/3552086751 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:31:47.509 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.520 7faec4f34700 1 -- 172.21.15.102:0/3552086751 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7faec00f5750 msgr2=0x7faec010c0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:31:47.509 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.520 7faec4f34700 1 --2- 172.21.15.102:0/3552086751 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7faec00f5750 0x7faec010c0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:47.509 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.520 7faec4f34700 1 -- 172.21.15.102:0/3552086751 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7faec010c720 msgr2=0x7faec01a0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:31:47.509 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.520 7faec4f34700 1 --2- 172.21.15.102:0/3552086751 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7faec010c720 0x7faec01a0be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:47.509 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.520 7faec4f34700 1 -- 172.21.15.102:0/3552086751 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faec01a82f0 con 0x7faec010cb70 2024-04-17T22:31:47.509 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.520 7faeb7fff700 1 --2- 172.21.15.102:0/3552086751 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7faec010c720 0x7faec01a0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:47.509 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.520 7faebffff700 1 --2- 172.21.15.102:0/3552086751 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7faec00f5750 0x7faec010c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:47.509 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.521 7faec4f34700 1 --2- 172.21.15.102:0/3552086751 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faec010cb70 0x7faec01a1210 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 rx=0x7faeb80032f0 tx=0x7faeb8009d10).ready entity=mon.0 client_cookie=9f73577e9d1fb6d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:47.510 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.521 7faebf7fe700 1 -- 172.21.15.102:0/3552086751 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7faec01a82f0 con 0x7faec010cb70 2024-04-17T22:31:47.510 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.521 7faebf7fe700 1 -- 172.21.15.102:0/3552086751 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faeb800b800 con 0x7faec010cb70 2024-04-17T22:31:47.510 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.521 7faebf7fe700 1 -- 172.21.15.102:0/3552086751 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7faeb800b9e0 con 0x7faec010cb70 2024-04-17T22:31:47.510 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.521 7faec7938700 1 -- 172.21.15.102:0/3552086751 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faec010cb70 msgr2=0x7faec01a1210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:47.510 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.521 7faec7938700 1 --2- 172.21.15.102:0/3552086751 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faec010cb70 0x7faec01a1210 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 rx=0x7faeb80032f0 tx=0x7faeb8009d10).stop 2024-04-17T22:31:47.510 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.521 7faec7938700 1 -- 172.21.15.102:0/3552086751 shutdown_connections 2024-04-17T22:31:47.510 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.521 7faec7938700 1 --2- 172.21.15.102:0/3552086751 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faec010cb70 0x7faec01a1210 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:47.510 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.521 7faec7938700 1 --2- 172.21.15.102:0/3552086751 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7faec00f5750 0x7faec010c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:47.510 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.521 7faec7938700 1 --2- 172.21.15.102:0/3552086751 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7faec010c720 0x7faec01a0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:47.510 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.521 7faec7938700 1 -- 172.21.15.102:0/3552086751 >> 172.21.15.102:0/3552086751 conn(0x7faec01138a0 msgr2=0x7faec00f4f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:47.510 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.521 7faec7938700 1 -- 172.21.15.102:0/3552086751 shutdown_connections 2024-04-17T22:31:47.511 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.521 7faec7938700 1 -- 172.21.15.102:0/3552086751 wait complete. 2024-04-17T22:31:47.532 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.544 7faec7938700 1 Processor -- start 2024-04-17T22:31:47.532 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.544 7faec7938700 1 -- start start 2024-04-17T22:31:47.532 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.545 7faec7938700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faec01ab450 0x7faec01ab8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:47.532 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.545 7faec7938700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7faec01b1a00 0x7faec01b1e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:47.533 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.545 7faec7938700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7faec01b2480 0x7faec01b8ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:47.533 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.545 7faec7938700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faec00c40e0 con 0x7faec01ab450 2024-04-17T22:31:47.533 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.545 7faec7938700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faec01abed0 con 0x7faec01b1a00 2024-04-17T22:31:47.533 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.545 7faec7938700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7faec00d34d0 con 0x7faec01b2480 2024-04-17T22:31:47.533 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.545 7faebffff700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faec01ab450 0x7faec01ab8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:47.533 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.545 7faeb7fff700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7faec01b1a00 0x7faec01b1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:47.533 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.545 7faebffff700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faec01ab450 0x7faec01ab8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:34852/0 (socket says 172.21.15.102:34852) 2024-04-17T22:31:47.533 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.545 7faebffff700 1 -- 172.21.15.102:0/806571793 learned_addr learned my addr 172.21.15.102:0/806571793 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:31:47.533 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.545 7faeb7fff700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7faec01b1a00 0x7faec01b1e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:49852/0 (socket says 172.21.15.102:49852) 2024-04-17T22:31:47.533 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.545 7faec4f34700 1 --2- 172.21.15.102:0/806571793 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7faec01b2480 0x7faec01b8ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:47.533 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.545 7faebffff700 1 -- 172.21.15.102:0/806571793 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7faec01b2480 msgr2=0x7faec01b8ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:47.533 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.545 7faebffff700 1 --2- 172.21.15.102:0/806571793 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7faec01b2480 0x7faec01b8ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:47.533 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.545 7faebffff700 1 -- 172.21.15.102:0/806571793 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7faec01b1a00 msgr2=0x7faec01b1e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:47.533 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.545 7faebffff700 1 --2- 172.21.15.102:0/806571793 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7faec01b1a00 0x7faec01b1e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:47.534 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.545 7faebffff700 1 -- 172.21.15.102:0/806571793 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faec01b28d0 con 0x7faec01ab450 2024-04-17T22:31:47.534 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.545 7faeb7fff700 1 --2- 172.21.15.102:0/806571793 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7faec01b1a00 0x7faec01b1e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:47.534 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.545 7faebffff700 1 --2- 172.21.15.102:0/806571793 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faec01ab450 0x7faec01ab8a0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 rx=0x7faec01a21a0 tx=0x7faeb000ed00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:47.534 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.545 7faebdffb700 1 -- 172.21.15.102:0/806571793 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7faec01b28d0 con 0x7faec01ab450 2024-04-17T22:31:47.534 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.545 7faebdffb700 1 -- 172.21.15.102:0/806571793 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faeb000ee40 con 0x7faec01ab450 2024-04-17T22:31:47.534 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.545 7faec7938700 1 -- 172.21.15.102:0/806571793 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faec00de4a0 con 0x7faec01ab450 2024-04-17T22:31:47.534 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.545 7faebdffb700 1 -- 172.21.15.102:0/806571793 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7faeb000f040 con 0x7faec01ab450 2024-04-17T22:31:47.534 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.545 7faec7938700 1 -- 172.21.15.102:0/806571793 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faec01b96f0 con 0x7faec01ab450 2024-04-17T22:31:47.534 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.546 7faebdffb700 1 -- 172.21.15.102:0/806571793 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7faeb0011070 con 0x7faec01ab450 2024-04-17T22:31:47.534 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.546 7faebdffb700 1 -- 172.21.15.102:0/806571793 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 2417+0+0 (secure 0 0 0) 0x7faeb0010040 con 0x7faec01ab450 2024-04-17T22:31:47.534 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.546 7faec7938700 1 -- 172.21.15.102:0/806571793 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faec00f0800 con 0x7faec01ab450 2024-04-17T22:31:47.535 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.550 7faebdffb700 1 -- 172.21.15.102:0/806571793 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+110455 (secure 0 0 0) 0x7faeb000c5a0 con 0x7faec01ab450 2024-04-17T22:31:47.642 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.656 7faec7938700 1 -- 172.21.15.102:0/806571793 --> v1:172.21.15.102:6816/42360 -- command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mgr", ""]}) v1 -- 0x7faec008a3b0 con 0x7faea40145a0 2024-04-17T22:31:47.642 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.656 7faebdffb700 1 -- 172.21.15.102:0/806571793 <== mgr.4104 v1:172.21.15.102:6816/42360 1 ==== command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (unknown 3049115190 0 0) 0x7faea800ba80 con 0x7faea40145a0 2024-04-17T22:31:47.643 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.657 7faec7938700 1 -- 172.21.15.102:0/806571793 >> v1:172.21.15.102:6816/42360 conn(0x7faea40145a0 legacy=0x7faea4016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:47.644 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.657 7faec7938700 1 -- 172.21.15.102:0/806571793 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faec01ab450 msgr2=0x7faec01ab8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:47.644 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.657 7faec7938700 1 --2- 172.21.15.102:0/806571793 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faec01ab450 0x7faec01ab8a0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 rx=0x7faec01a21a0 tx=0x7faeb000ed00).stop 2024-04-17T22:31:47.644 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.657 7faec7938700 1 -- 172.21.15.102:0/806571793 shutdown_connections 2024-04-17T22:31:47.644 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.657 7faec7938700 1 --2- 172.21.15.102:0/806571793 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7faec01b2480 0x7faec01b8ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:47.644 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.657 7faec7938700 1 --2- 172.21.15.102:0/806571793 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faec01ab450 0x7faec01ab8a0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:47.644 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.657 7faec7938700 1 --2- 172.21.15.102:0/806571793 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7faec01b1a00 0x7faec01b1e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:47.644 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.657 7faec7938700 1 -- 172.21.15.102:0/806571793 >> 172.21.15.102:0/806571793 conn(0x7faec01a4e90 msgr2=0x7faec00d6810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:47.644 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.657 7faec7938700 1 -- 172.21.15.102:0/806571793 shutdown_connections 2024-04-17T22:31:47.644 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.657 7faec7938700 1 -- 172.21.15.102:0/806571793 wait complete. 2024-04-17T22:31:47.644 INFO:teuthology.orchestra.run.smithi102.stderr:instructed osd(s) 2 to deep-scrub 2024-04-17T22:31:47.654 INFO:tasks.ceph:Scrubbing osd.3 2024-04-17T22:31:47.654 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-04-17T22:31:47.783 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.797 7fac38826700 1 Processor -- start 2024-04-17T22:31:47.783 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.797 7fac38826700 1 -- start start 2024-04-17T22:31:47.783 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.797 7fac38826700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fac340f56f0 0x7fac3410e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:47.783 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.797 7fac38826700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fac340f1c30 0x7fac341aaef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:47.783 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.797 7fac38826700 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac34103180 con 0x7fac3410e980 2024-04-17T22:31:47.784 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.797 7fac38826700 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac3410edd0 con 0x7fac340f1c30 2024-04-17T22:31:47.784 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.797 7fac38826700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac34110960 con 0x7fac340f56f0 2024-04-17T22:31:47.784 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.797 7fac31dfc700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fac340f1c30 0x7fac341aaef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:47.784 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.797 7fac30dfa700 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7fac3410e980 0x7fac340f1430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:47890/0 (socket says 172.21.15.102:47890) 2024-04-17T22:31:47.784 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.797 7fac315fb700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fac340f56f0 0x7fac3410e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:47.784 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.797 7fac30dfa700 1 -- 172.21.15.102:0/1933046132 learned_addr learned my addr 172.21.15.102:0/1933046132 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-17T22:31:47.786 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.798 7fac315fb700 1 -- 172.21.15.102:0/1933046132 >> v2:172.21.15.102:3300/0 conn(0x7fac340f1c30 msgr2=0x7fac341aaef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:31:47.786 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.798 7fac315fb700 1 --2- 172.21.15.102:0/1933046132 >> v2:172.21.15.102:3300/0 conn(0x7fac340f1c30 0x7fac341aaef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:47.786 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.798 7fac315fb700 1 -- 172.21.15.102:0/1933046132 >> v1:172.21.15.102:6789/0 conn(0x7fac3410e980 legacy=0x7fac340f1430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:47.786 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.798 7fac315fb700 1 -- 172.21.15.102:0/1933046132 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac340f2080 con 0x7fac340f56f0 2024-04-17T22:31:47.786 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.798 7fac2bfff700 1 -- 172.21.15.102:0/1933046132 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4075508920 0 0) 0x7fac24001110 con 0x7fac3410e980 2024-04-17T22:31:47.786 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.798 7fac31dfc700 1 --2- 172.21.15.102:0/1933046132 >> v2:172.21.15.102:3300/0 conn(0x7fac340f1c30 0x7fac341aaef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:47.786 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.798 7fac315fb700 1 --2- 172.21.15.102:0/1933046132 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fac340f56f0 0x7fac3410e350 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 rx=0x7fac1c002f70 tx=0x7fac1c00b970).ready entity=mon.1 client_cookie=46da862e4dd5a292 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:47.786 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.798 7fac2bfff700 1 -- 172.21.15.102:0/1933046132 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fac340f2080 con 0x7fac340f56f0 2024-04-17T22:31:47.786 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.798 7fac2bfff700 1 -- 172.21.15.102:0/1933046132 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac1c00d5c0 con 0x7fac340f56f0 2024-04-17T22:31:47.786 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.798 7fac2bfff700 1 -- 172.21.15.102:0/1933046132 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fac1c00d990 con 0x7fac340f56f0 2024-04-17T22:31:47.786 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.798 7fac38826700 1 -- 172.21.15.102:0/1933046132 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fac340f56f0 msgr2=0x7fac3410e350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:47.786 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.798 7fac38826700 1 --2- 172.21.15.102:0/1933046132 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fac340f56f0 0x7fac3410e350 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 rx=0x7fac1c002f70 tx=0x7fac1c00b970).stop 2024-04-17T22:31:47.786 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.799 7fac38826700 1 -- 172.21.15.102:0/1933046132 shutdown_connections 2024-04-17T22:31:47.787 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.799 7fac38826700 1 --2- 172.21.15.102:0/1933046132 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fac340f56f0 0x7fac3410e350 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:47.787 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.799 7fac38826700 1 --2- 172.21.15.102:0/1933046132 >> v2:172.21.15.102:3300/0 conn(0x7fac340f1c30 0x7fac341aaef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:47.787 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.799 7fac38826700 1 -- 172.21.15.102:0/1933046132 >> 172.21.15.102:0/1933046132 conn(0x7fac341135a0 msgr2=0x7fac340f4c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:47.787 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.799 7fac38826700 1 -- 172.21.15.102:0/1933046132 shutdown_connections 2024-04-17T22:31:47.787 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.799 7fac38826700 1 -- 172.21.15.102:0/1933046132 wait complete. 2024-04-17T22:31:47.810 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.822 7fac38826700 1 Processor -- start 2024-04-17T22:31:47.810 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.822 7fac38826700 1 -- start start 2024-04-17T22:31:47.810 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.822 7fac38826700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fac340d5ed0 0x7fac340d6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:47.810 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.822 7fac38826700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fac340d6950 0x7fac341b3a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:47.810 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.822 7fac38826700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fac341b40c0 0x7fac341b4510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:47.810 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.822 7fac38826700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac340cc0a0 con 0x7fac340d5ed0 2024-04-17T22:31:47.810 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.822 7fac38826700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac341b4b40 con 0x7fac341b40c0 2024-04-17T22:31:47.810 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.822 7fac38826700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fac340ccf00 con 0x7fac340d6950 2024-04-17T22:31:47.810 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.822 7fac315fb700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fac340d5ed0 0x7fac340d6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:47.811 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.822 7fac31dfc700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fac341b40c0 0x7fac341b4510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:47.811 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.822 7fac30dfa700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fac340d6950 0x7fac341b3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:47.811 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.822 7fac315fb700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fac340d5ed0 0x7fac340d6320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:34868/0 (socket says 172.21.15.102:34868) 2024-04-17T22:31:47.811 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.822 7fac315fb700 1 -- 172.21.15.102:0/1785996677 learned_addr learned my addr 172.21.15.102:0/1785996677 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:31:47.811 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.823 7fac315fb700 1 -- 172.21.15.102:0/1785996677 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fac340d6950 msgr2=0x7fac341b3a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:47.811 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.823 7fac315fb700 1 --2- 172.21.15.102:0/1785996677 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fac340d6950 0x7fac341b3a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:47.811 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.823 7fac315fb700 1 -- 172.21.15.102:0/1785996677 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fac341b40c0 msgr2=0x7fac341b4510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:47.811 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.823 7fac315fb700 1 --2- 172.21.15.102:0/1785996677 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fac341b40c0 0x7fac341b4510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:47.811 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.823 7fac315fb700 1 -- 172.21.15.102:0/1785996677 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac340db550 con 0x7fac340d5ed0 2024-04-17T22:31:47.811 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.823 7fac31dfc700 1 --2- 172.21.15.102:0/1785996677 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fac341b40c0 0x7fac341b4510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:47.811 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.823 7fac315fb700 1 --2- 172.21.15.102:0/1785996677 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fac340d5ed0 0x7fac340d6320 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 rx=0x7fac1c00bf10 tx=0x7fac1c0068c0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:47.811 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.823 7fac2a7fc700 1 -- 172.21.15.102:0/1785996677 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fac340db550 con 0x7fac340d5ed0 2024-04-17T22:31:47.811 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.823 7fac2a7fc700 1 -- 172.21.15.102:0/1785996677 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac1c011760 con 0x7fac340d5ed0 2024-04-17T22:31:47.811 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.823 7fac38826700 1 -- 172.21.15.102:0/1785996677 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac341bb200 con 0x7fac340d5ed0 2024-04-17T22:31:47.812 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.823 7fac2a7fc700 1 -- 172.21.15.102:0/1785996677 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fac1c017040 con 0x7fac340d5ed0 2024-04-17T22:31:47.812 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.823 7fac38826700 1 -- 172.21.15.102:0/1785996677 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac341bb5e0 con 0x7fac340d5ed0 2024-04-17T22:31:47.812 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.824 7fac2a7fc700 1 -- 172.21.15.102:0/1785996677 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7fac1c011b30 con 0x7fac340d5ed0 2024-04-17T22:31:47.812 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.824 7fac2a7fc700 1 -- 172.21.15.102:0/1785996677 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 2417+0+0 (secure 0 0 0) 0x7fac1c018070 con 0x7fac340d5ed0 2024-04-17T22:31:47.812 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.824 7fac38826700 1 -- 172.21.15.102:0/1785996677 --> v1:172.21.15.102:6800/42953 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fac340cc8d0 con 0x7fac340e3890 2024-04-17T22:31:47.812 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.825 7fac2a7fc700 1 -- 172.21.15.102:0/1785996677 <== osd.3 v1:172.21.15.102:6800/42953 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+5151 (unknown 0 0 4057655604) 0x7fac2c009d30 con 0x7fac340e3890 2024-04-17T22:31:47.817 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.830 7fac38826700 1 -- 172.21.15.102:0/1785996677 --> v1:172.21.15.102:6800/42953 -- command(tid 2: {"prefix": "config set", "key": "osd_debug_deep_scrub_sleep", "value": "0"}) v1 -- 0x7fac340878c0 con 0x7fac340e3890 2024-04-17T22:31:47.817 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.831 7fac2a7fc700 1 -- 172.21.15.102:0/1785996677 <== osd.3 v1:172.21.15.102:6800/42953 2 ==== command_reply(tid 2: 0 Set osd_debug_deep_scrub_sleep to 0) v1 ==== 43+0+0 (unknown 1994953735 0 0) 0x7fac2c00c150 con 0x7fac340e3890 2024-04-17T22:31:47.817 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.831 7fac38826700 1 -- 172.21.15.102:0/1785996677 >> v1:172.21.15.102:6800/42953 conn(0x7fac340e3890 legacy=0x7fac340a7bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:47.817 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.831 7fac38826700 1 -- 172.21.15.102:0/1785996677 >> v1:172.21.15.102:6816/42360 conn(0x7fac180145a0 legacy=0x7fac18016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:47.817 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.831 7fac38826700 1 -- 172.21.15.102:0/1785996677 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fac340d5ed0 msgr2=0x7fac340d6320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:47.817 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.831 7fac38826700 1 --2- 172.21.15.102:0/1785996677 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fac340d5ed0 0x7fac340d6320 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 rx=0x7fac1c00bf10 tx=0x7fac1c0068c0).stop 2024-04-17T22:31:47.818 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.831 7fac31dfc700 1 -- 172.21.15.102:0/1785996677 reap_dead start 2024-04-17T22:31:47.818 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.831 7fac38826700 1 -- 172.21.15.102:0/1785996677 shutdown_connections 2024-04-17T22:31:47.818 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.831 7fac38826700 1 -- 172.21.15.102:0/1785996677 >> 172.21.15.102:0/1785996677 conn(0x7fac340c7690 msgr2=0x7fac340d6fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:47.818 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.831 7fac38826700 1 -- 172.21.15.102:0/1785996677 shutdown_connections 2024-04-17T22:31:47.818 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.831 7fac38826700 1 -- 172.21.15.102:0/1785996677 wait complete. 2024-04-17T22:31:47.818 INFO:teuthology.orchestra.run.smithi102.stderr:Set osd_debug_deep_scrub_sleep to 0 2024-04-17T22:31:47.827 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-04-17T22:31:47.955 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.967 7fd38c533700 1 Processor -- start 2024-04-17T22:31:47.955 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.967 7fd38c533700 1 -- start start 2024-04-17T22:31:47.955 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.968 7fd38c533700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd38410b810 0x7fd3841a52c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:47.956 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.968 7fd38c533700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd38410bc60 0x7fd3841a7900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:47.956 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.968 7fd38c533700 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd384103b40 con 0x7fd3840f5840 2024-04-17T22:31:47.956 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.968 7fd38c533700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3840fa370 con 0x7fd38410b810 2024-04-17T22:31:47.956 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.968 7fd38c533700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd3841a0b00 con 0x7fd38410bc60 2024-04-17T22:31:47.956 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.968 7fd388b2d700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd38410b810 0x7fd3841a52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:47.956 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.968 7fd38932e700 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7fd3840f5840 0x7fd38410b010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:47902/0 (socket says 172.21.15.102:47902) 2024-04-17T22:31:47.956 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.968 7fd389b2f700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd38410bc60 0x7fd3841a7900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:47.956 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.968 7fd38932e700 1 -- 172.21.15.102:0/428594591 learned_addr learned my addr 172.21.15.102:0/428594591 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-17T22:31:47.956 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.968 7fd377fff700 1 -- 172.21.15.102:0/428594591 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 239264650 0 0) 0x7fd378001110 con 0x7fd3840f5840 2024-04-17T22:31:47.956 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.968 7fd377fff700 1 -- 172.21.15.102:0/428594591 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd36c003270 con 0x7fd3840f5840 2024-04-17T22:31:47.956 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.968 7fd388b2d700 1 -- 172.21.15.102:0/428594591 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd38410bc60 msgr2=0x7fd3841a7900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:31:47.956 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.968 7fd388b2d700 1 --2- 172.21.15.102:0/428594591 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd38410bc60 0x7fd3841a7900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:47.956 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.968 7fd388b2d700 1 -- 172.21.15.102:0/428594591 >> v1:172.21.15.102:6789/0 conn(0x7fd3840f5840 legacy=0x7fd38410b010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:47.957 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.968 7fd388b2d700 1 -- 172.21.15.102:0/428594591 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3841a8f40 con 0x7fd38410b810 2024-04-17T22:31:47.957 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.968 7fd389b2f700 1 --2- 172.21.15.102:0/428594591 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd38410bc60 0x7fd3841a7900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:47.957 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.968 7fd388b2d700 1 --2- 172.21.15.102:0/428594591 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd38410b810 0x7fd3841a52c0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 rx=0x7fd380002f70 tx=0x7fd38000ba20).ready entity=mon.1 client_cookie=c839c694f15503f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:47.957 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.969 7fd377fff700 1 -- 172.21.15.102:0/428594591 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd3841a8f40 con 0x7fd38410b810 2024-04-17T22:31:47.957 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.969 7fd377fff700 1 -- 172.21.15.102:0/428594591 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd38000d570 con 0x7fd38410b810 2024-04-17T22:31:47.957 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.969 7fd377fff700 1 -- 172.21.15.102:0/428594591 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd38000d940 con 0x7fd38410b810 2024-04-17T22:31:47.957 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.969 7fd38c533700 1 -- 172.21.15.102:0/428594591 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd38410b810 msgr2=0x7fd3841a52c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:47.957 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.969 7fd38c533700 1 --2- 172.21.15.102:0/428594591 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd38410b810 0x7fd3841a52c0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 rx=0x7fd380002f70 tx=0x7fd38000ba20).stop 2024-04-17T22:31:47.957 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.969 7fd38c533700 1 -- 172.21.15.102:0/428594591 shutdown_connections 2024-04-17T22:31:47.957 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.969 7fd38c533700 1 --2- 172.21.15.102:0/428594591 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd38410bc60 0x7fd3841a7900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:47.957 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.969 7fd38c533700 1 --2- 172.21.15.102:0/428594591 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd38410b810 0x7fd3841a52c0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:47.957 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.969 7fd38c533700 1 -- 172.21.15.102:0/428594591 >> 172.21.15.102:0/428594591 conn(0x7fd3841138a0 msgr2=0x7fd3840f4f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:47.957 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.969 7fd38c533700 1 -- 172.21.15.102:0/428594591 shutdown_connections 2024-04-17T22:31:47.958 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.969 7fd38c533700 1 -- 172.21.15.102:0/428594591 wait complete. 2024-04-17T22:31:47.980 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.992 7fd38c533700 1 Processor -- start 2024-04-17T22:31:47.980 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.992 7fd38c533700 1 -- start start 2024-04-17T22:31:47.980 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.992 7fd38c533700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd3840d5ec0 0x7fd3840d6310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:47.981 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.992 7fd38c533700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd3840d6940 0x7fd3841b0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:47.981 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.992 7fd38c533700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd3841b1510 0x7fd3841b1960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:47.981 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.992 7fd38c533700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3840c43d0 con 0x7fd3840d6940 2024-04-17T22:31:47.981 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.992 7fd38c533700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3840cd670 con 0x7fd3841b1510 2024-04-17T22:31:47.981 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.992 7fd38c533700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd3840cd250 con 0x7fd3840d5ec0 2024-04-17T22:31:47.981 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.992 7fd38932e700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd3840d5ec0 0x7fd3840d6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:47.981 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.992 7fd388b2d700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd3840d6940 0x7fd3841b0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:47.981 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.992 7fd389b2f700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd3841b1510 0x7fd3841b1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:47.981 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.993 7fd38932e700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd3840d5ec0 0x7fd3840d6310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:34622/0 (socket says 172.21.15.102:34622) 2024-04-17T22:31:47.981 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.993 7fd388b2d700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd3840d6940 0x7fd3841b0ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:34870/0 (socket says 172.21.15.102:34870) 2024-04-17T22:31:47.981 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.993 7fd389b2f700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd3841b1510 0x7fd3841b1960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:49880/0 (socket says 172.21.15.102:49880) 2024-04-17T22:31:47.981 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.993 7fd38932e700 1 -- 172.21.15.102:0/3772672485 learned_addr learned my addr 172.21.15.102:0/3772672485 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:31:47.982 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.993 7fd38932e700 1 -- 172.21.15.102:0/3772672485 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd3841b1510 msgr2=0x7fd3841b1960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:47.982 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.993 7fd38932e700 1 --2- 172.21.15.102:0/3772672485 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd3841b1510 0x7fd3841b1960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:47.982 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.993 7fd38932e700 1 -- 172.21.15.102:0/3772672485 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd3840d6940 msgr2=0x7fd3841b0ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:47.982 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.993 7fd38932e700 1 --2- 172.21.15.102:0/3772672485 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd3840d6940 0x7fd3841b0ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:47.982 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.993 7fd38932e700 1 -- 172.21.15.102:0/3772672485 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3840cd930 con 0x7fd3840d5ec0 2024-04-17T22:31:47.982 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.993 7fd388b2d700 1 --2- 172.21.15.102:0/3772672485 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd3840d6940 0x7fd3841b0ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:47.982 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.993 7fd38932e700 1 --2- 172.21.15.102:0/3772672485 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd3840d5ec0 0x7fd3840d6310 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 rx=0x7fd3840cd5a0 tx=0x7fd37800c7c0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:47.982 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.993 7fd3767fc700 1 -- 172.21.15.102:0/3772672485 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd3840cd930 con 0x7fd3840d5ec0 2024-04-17T22:31:47.982 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.993 7fd3767fc700 1 -- 172.21.15.102:0/3772672485 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3780072e0 con 0x7fd3840d5ec0 2024-04-17T22:31:47.982 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.993 7fd38c533700 1 -- 172.21.15.102:0/3772672485 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3840ce110 con 0x7fd3840d5ec0 2024-04-17T22:31:47.982 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.993 7fd3767fc700 1 -- 172.21.15.102:0/3772672485 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fd378005810 con 0x7fd3840d5ec0 2024-04-17T22:31:47.983 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.993 7fd38c533700 1 -- 172.21.15.102:0/3772672485 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3840f1990 con 0x7fd3840d5ec0 2024-04-17T22:31:47.983 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.994 7fd3767fc700 1 -- 172.21.15.102:0/3772672485 <== mon.2 v2:172.21.15.102:3302/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7fd378005a10 con 0x7fd3840d5ec0 2024-04-17T22:31:47.983 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.994 7fd3767fc700 1 -- 172.21.15.102:0/3772672485 <== mon.2 v2:172.21.15.102:3302/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 2417+0+0 (secure 0 0 0) 0x7fd3780146e0 con 0x7fd3840d5ec0 2024-04-17T22:31:47.983 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.994 7fd38c533700 1 -- 172.21.15.102:0/3772672485 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3840f0800 con 0x7fd3840d5ec0 2024-04-17T22:31:47.983 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:47.998 7fd3767fc700 1 -- 172.21.15.102:0/3772672485 <== mon.2 v2:172.21.15.102:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+110455 (secure 0 0 0) 0x7fd3780070d0 con 0x7fd3840d5ec0 2024-04-17T22:31:48.090 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:48.104 7fd38c533700 1 -- 172.21.15.102:0/3772672485 --> v1:172.21.15.102:6816/42360 -- command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mgr", ""]}) v1 -- 0x7fd38408a3b0 con 0x7fd3640145a0 2024-04-17T22:31:48.090 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:48.104 7fd3767fc700 1 -- 172.21.15.102:0/3772672485 <== mgr.4104 v1:172.21.15.102:6816/42360 1 ==== command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (unknown 3526570807 0 0) 0x7fd380006950 con 0x7fd3640145a0 2024-04-17T22:31:48.091 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:48.105 7fd38c533700 1 -- 172.21.15.102:0/3772672485 >> v1:172.21.15.102:6816/42360 conn(0x7fd3640145a0 legacy=0x7fd364016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:48.092 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:48.105 7fd38c533700 1 -- 172.21.15.102:0/3772672485 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd3840d5ec0 msgr2=0x7fd3840d6310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:48.092 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:48.105 7fd38c533700 1 --2- 172.21.15.102:0/3772672485 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd3840d5ec0 0x7fd3840d6310 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 rx=0x7fd3840cd5a0 tx=0x7fd37800c7c0).stop 2024-04-17T22:31:48.092 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:48.105 7fd38c533700 1 -- 172.21.15.102:0/3772672485 shutdown_connections 2024-04-17T22:31:48.092 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:48.105 7fd38c533700 1 --2- 172.21.15.102:0/3772672485 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd3841b1510 0x7fd3841b1960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:48.092 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:48.105 7fd38c533700 1 --2- 172.21.15.102:0/3772672485 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd3840d5ec0 0x7fd3840d6310 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:48.092 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:48.105 7fd38c533700 1 --2- 172.21.15.102:0/3772672485 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd3840d6940 0x7fd3841b0ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:48.092 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:48.105 7fd38c533700 1 -- 172.21.15.102:0/3772672485 >> 172.21.15.102:0/3772672485 conn(0x7fd3840c37a0 msgr2=0x7fd3840d7640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:48.092 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:48.105 7fd38c533700 1 -- 172.21.15.102:0/3772672485 shutdown_connections 2024-04-17T22:31:48.092 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:48.105 7fd38c533700 1 -- 172.21.15.102:0/3772672485 wait complete. 2024-04-17T22:31:48.093 INFO:teuthology.orchestra.run.smithi102.stderr:instructed osd(s) 3 to deep-scrub 2024-04-17T22:31:48.102 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-17T22:31:48.230 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:48.240 7fb395f3c700 1 Processor -- start 2024-04-17T22:31:48.230 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:48.240 7fb395f3c700 1 -- start start 2024-04-17T22:31:48.230 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:48.241 7fb395f3c700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb39010e260 0x7fb39010e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:48.230 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:48.241 7fb395f3c700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fb39010ece0 0x7fb3900f1750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:48.231 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:48.241 7fb395f3c700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb3900f1d80 0x7fb3901aa4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:48.231 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:48.241 7fb395f3c700 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3900f9f30 con 0x7fb39010ece0 2024-04-17T22:31:48.231 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:48.241 7fb395f3c700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3900f9050 con 0x7fb39010e260 2024-04-17T22:31:48.231 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:48.241 7fb395f3c700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb3900fa1d0 con 0x7fb3900f1d80 2024-04-17T22:31:48.231 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:48.241 7fb38f4ee700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb3900f1d80 0x7fb3901aa4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:48.231 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:48.241 7fb38e4ec700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fb39010ece0 0x7fb3900f1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:48.231 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:48.241 7fb38e4ec700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fb39010ece0 0x7fb3900f1750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:34876/0 (socket says 172.21.15.102:34876) 2024-04-17T22:31:48.231 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:48.241 7fb38f4ee700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb3900f1d80 0x7fb3901aa4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:34634/0 (socket says 172.21.15.102:34634) 2024-04-17T22:31:48.231 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:48.241 7fb38e4ec700 1 -- 172.21.15.102:0/80618693 learned_addr learned my addr 172.21.15.102:0/80618693 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:31:48.231 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:48.241 7fb38eced700 1 --2- 172.21.15.102:0/80618693 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb39010e260 0x7fb39010e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:48.231 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:48.241 7fb38e4ec700 1 -- 172.21.15.102:0/80618693 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb3900f1d80 msgr2=0x7fb3901aa4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:31:48.231 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:48.241 7fb38e4ec700 1 --2- 172.21.15.102:0/80618693 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb3900f1d80 0x7fb3901aa4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:48.232 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:48.241 7fb38e4ec700 1 -- 172.21.15.102:0/80618693 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb39010e260 msgr2=0x7fb39010e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:31:48.232 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:48.241 7fb38e4ec700 1 --2- 172.21.15.102:0/80618693 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb39010e260 0x7fb39010e6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:48.232 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:48.241 7fb38e4ec700 1 -- 172.21.15.102:0/80618693 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3900f21d0 con 0x7fb39010ece0 2024-04-17T22:31:48.232 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:48.241 7fb38f4ee700 1 --2- 172.21.15.102:0/80618693 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb3900f1d80 0x7fb3901aa4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:48.232 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:48.241 7fb38eced700 1 --2- 172.21.15.102:0/80618693 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb39010e260 0x7fb39010e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:48.233 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:48.242 7fb38e4ec700 1 --2- 172.21.15.102:0/80618693 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb39010ece0 0x7fb3900f1750 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 rx=0x7fb3900f46c0 tx=0x7fb380009ab0).ready entity=mon.0 client_cookie=e8b8ec16e647c480 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:48.233 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:48.242 7fb38dceb700 1 -- 172.21.15.102:0/80618693 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb3900f21d0 con 0x7fb39010ece0 2024-04-17T22:31:48.233 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:48.242 7fb38dceb700 1 -- 172.21.15.102:0/80618693 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb38000f510 con 0x7fb39010ece0 2024-04-17T22:31:48.233 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:48.242 7fb38dceb700 1 -- 172.21.15.102:0/80618693 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb38000f6f0 con 0x7fb39010ece0 2024-04-17T22:31:48.233 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:48.242 7fb395f3c700 1 -- 172.21.15.102:0/80618693 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb39010ece0 msgr2=0x7fb3900f1750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:48.233 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:48.242 7fb395f3c700 1 --2- 172.21.15.102:0/80618693 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb39010ece0 0x7fb3900f1750 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 rx=0x7fb3900f46c0 tx=0x7fb380009ab0).stop 2024-04-17T22:31:48.234 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:48.242 7fb395f3c700 1 -- 172.21.15.102:0/80618693 shutdown_connections 2024-04-17T22:31:48.234 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:48.242 7fb395f3c700 1 --2- 172.21.15.102:0/80618693 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb3900f1d80 0x7fb3901aa4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:48.234 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:48.242 7fb395f3c700 1 --2- 172.21.15.102:0/80618693 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb39010e260 0x7fb39010e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:48.234 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:48.242 7fb395f3c700 1 --2- 172.21.15.102:0/80618693 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb39010ece0 0x7fb3900f1750 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:48.234 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:48.242 7fb395f3c700 1 -- 172.21.15.102:0/80618693 >> 172.21.15.102:0/80618693 conn(0x7fb3901138c0 msgr2=0x7fb3900f4f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:48.234 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:48.242 7fb395f3c700 1 -- 172.21.15.102:0/80618693 shutdown_connections 2024-04-17T22:31:48.234 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:48.242 7fb395f3c700 1 -- 172.21.15.102:0/80618693 wait complete. 2024-04-17T22:31:48.258 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:48.265 7fb395f3c700 1 Processor -- start 2024-04-17T22:31:48.258 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:48.265 7fb395f3c700 1 -- start start 2024-04-17T22:31:48.258 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:48.265 7fb395f3c700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb3900de420 0x7fb3900ca2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:48.258 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:48.265 7fb395f3c700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb3900ca910 0x7fb3901b3be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:48.258 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:48.265 7fb395f3c700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb3900cad60 0x7fb3901b4210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:31:48.258 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:48.265 7fb395f3c700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb39008f2e0 con 0x7fb3900ca910 2024-04-17T22:31:48.258 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:48.265 7fb395f3c700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3900d1170 con 0x7fb3900de420 2024-04-17T22:31:48.258 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:48.265 7fb395f3c700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb3900d1500 con 0x7fb3900cad60 2024-04-17T22:31:48.258 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:48.265 7fb38e4ec700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb3900ca910 0x7fb3901b3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:48.259 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:48.265 7fb38f4ee700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb3900cad60 0x7fb3901b4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:48.259 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:48.265 7fb38eced700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb3900de420 0x7fb3900ca2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:31:48.259 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:48.265 7fb38e4ec700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb3900ca910 0x7fb3901b3be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:34878/0 (socket says 172.21.15.102:34878) 2024-04-17T22:31:48.259 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:48.265 7fb38e4ec700 1 -- 172.21.15.102:0/1500574760 learned_addr learned my addr 172.21.15.102:0/1500574760 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:31:48.259 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:48.265 7fb38f4ee700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb3900cad60 0x7fb3901b4210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:34646/0 (socket says 172.21.15.102:34646) 2024-04-17T22:31:48.259 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:48.265 7fb38eced700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb3900de420 0x7fb3900ca2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:49890/0 (socket says 172.21.15.102:49890) 2024-04-17T22:31:48.259 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:48.266 7fb38e4ec700 1 -- 172.21.15.102:0/1500574760 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb3900cad60 msgr2=0x7fb3901b4210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:48.260 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:48.266 7fb38e4ec700 1 --2- 172.21.15.102:0/1500574760 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb3900cad60 0x7fb3901b4210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:48.260 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:48.266 7fb38e4ec700 1 -- 172.21.15.102:0/1500574760 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb3900de420 msgr2=0x7fb3900ca2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:48.260 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:48.266 7fb38e4ec700 1 --2- 172.21.15.102:0/1500574760 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb3900de420 0x7fb3900ca2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:48.260 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:48.266 7fb38e4ec700 1 -- 172.21.15.102:0/1500574760 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3901b4840 con 0x7fb3900ca910 2024-04-17T22:31:48.260 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:48.266 7fb38f4ee700 1 --2- 172.21.15.102:0/1500574760 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb3900cad60 0x7fb3901b4210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:48.260 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:48.266 7fb38eced700 1 --2- 172.21.15.102:0/1500574760 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb3900de420 0x7fb3900ca2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:31:48.260 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:48.266 7fb38e4ec700 1 --2- 172.21.15.102:0/1500574760 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb3900ca910 0x7fb3901b3be0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 rx=0x7fb3900f46c0 tx=0x7fb3800068c0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:31:48.260 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:48.266 7fb37ffff700 1 -- 172.21.15.102:0/1500574760 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb3901b4840 con 0x7fb3900ca910 2024-04-17T22:31:48.260 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:48.266 7fb37ffff700 1 -- 172.21.15.102:0/1500574760 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb38001c720 con 0x7fb3900ca910 2024-04-17T22:31:48.261 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:48.266 7fb395f3c700 1 -- 172.21.15.102:0/1500574760 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3901b4a40 con 0x7fb3900ca910 2024-04-17T22:31:48.261 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:48.266 7fb37ffff700 1 -- 172.21.15.102:0/1500574760 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7fb38001caf0 con 0x7fb3900ca910 2024-04-17T22:31:48.261 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:48.266 7fb395f3c700 1 -- 172.21.15.102:0/1500574760 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3901bb5c0 con 0x7fb3900ca910 2024-04-17T22:31:48.261 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:48.266 7fb37ffff700 1 -- 172.21.15.102:0/1500574760 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7fb380019030 con 0x7fb3900ca910 2024-04-17T22:31:48.261 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:48.266 7fb37ffff700 1 -- 172.21.15.102:0/1500574760 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 2417+0+0 (secure 0 0 0) 0x7fb380018040 con 0x7fb3900ca910 2024-04-17T22:31:48.261 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:48.267 7fb395f3c700 1 -- 172.21.15.102:0/1500574760 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3900f0820 con 0x7fb3900ca910 2024-04-17T22:31:48.261 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:48.270 7fb37ffff700 1 -- 172.21.15.102:0/1500574760 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+110455 (secure 0 0 0) 0x7fb38001c510 con 0x7fb3900ca910 2024-04-17T22:31:48.357 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:48.371 7fb395f3c700 1 -- 172.21.15.102:0/1500574760 --> v1:172.21.15.102:6816/42360 -- command(tid 0: {"prefix": "pg dump", "target": ["mgr", ""], "format": "json"}) v1 -- 0x7fb39008a300 con 0x7fb36c0145a0 2024-04-17T22:31:48.358 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:48.372 7fb37ffff700 1 -- 172.21.15.102:0/1500574760 <== mgr.4104 v1:172.21.15.102:6816/42360 1 ==== command_reply(tid 0: 0 dumped all) v1 ==== 18+0+32484 (unknown 2965378022 0 2948472885) 0x7fb38400ba60 con 0x7fb36c0145a0 2024-04-17T22:31:48.359 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-17T22:31:48.361 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:48.373 7fb395f3c700 1 -- 172.21.15.102:0/1500574760 >> v1:172.21.15.102:6816/42360 conn(0x7fb36c0145a0 legacy=0x7fb36c016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:48.361 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:48.373 7fb395f3c700 1 -- 172.21.15.102:0/1500574760 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb3900ca910 msgr2=0x7fb3901b3be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:31:48.361 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:48.373 7fb395f3c700 1 --2- 172.21.15.102:0/1500574760 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb3900ca910 0x7fb3901b3be0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 rx=0x7fb3900f46c0 tx=0x7fb3800068c0).stop 2024-04-17T22:31:48.361 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:48.374 7fb395f3c700 1 -- 172.21.15.102:0/1500574760 shutdown_connections 2024-04-17T22:31:48.362 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:48.374 7fb395f3c700 1 --2- 172.21.15.102:0/1500574760 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb3900cad60 0x7fb3901b4210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:48.362 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:48.374 7fb395f3c700 1 --2- 172.21.15.102:0/1500574760 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb3900de420 0x7fb3900ca2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:48.362 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:48.374 7fb395f3c700 1 --2- 172.21.15.102:0/1500574760 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb3900ca910 0x7fb3901b3be0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:31:48.362 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:48.374 7fb395f3c700 1 -- 172.21.15.102:0/1500574760 >> 172.21.15.102:0/1500574760 conn(0x7fb3901a7050 msgr2=0x7fb3900d69d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:31:48.363 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:48.374 7fb395f3c700 1 -- 172.21.15.102:0/1500574760 shutdown_connections 2024-04-17T22:31:48.363 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:31:48.374 7fb395f3c700 1 -- 172.21.15.102:0/1500574760 wait complete. 2024-04-17T22:31:48.363 INFO:teuthology.orchestra.run.smithi102.stderr:dumped all 2024-04-17T22:31:48.370 INFO:teuthology.orchestra.run.smithi102.stdout:{"pg_ready":true,"pg_map":{"version":111,"stamp":"2024-04-17 22:31:47.223103","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":55,"num_osds":4,"num_per_pool_osds":4,"kb":377487360,"kb_used":4195352,"kb_used_data":992,"kb_used_omap":0,"kb_used_meta":4194304,"kb_avail":373292008,"statfs":{"total":386547056640,"available":382251016192,"internally_reserved":4294967296,"allocated":1015808,"data_stored":319928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4294967296},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-2286,"num_objects":-21,"num_object_clones":0,"num_object_copies":-42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-44,"num_write_kb":-13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.309956"},"pg_stats":[{"pgid":"1.5","version":"0'0","reported_seq":"25","reported_epoch":"27","state":"active+clean","last_fresh":"2024-04-17 22:31:41.534910","last_change":"2024-04-17 22:28:59.253977","last_active":"2024-04-17 22:31:41.534910","last_peered":"2024-04-17 22:31:41.534910","last_clean":"2024-04-17 22:31:41.534910","last_became_active":"2024-04-17 22:28:59.253641","last_became_peered":"2024-04-17 22:28:59.253641","last_unstale":"2024-04-17 22:31:41.534910","last_undegraded":"2024-04-17 22:31:41.534910","last_fullsized":"2024-04-17 22:31:41.534910","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:28:58.239187","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:28:58.239187","last_clean_scrub_stamp":"2024-04-17 22:28:58.239187","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":"25","reported_epoch":"27","state":"active+clean","last_fresh":"2024-04-17 22:31:41.638378","last_change":"2024-04-17 22:28:59.255733","last_active":"2024-04-17 22:31:41.638378","last_peered":"2024-04-17 22:31:41.638378","last_clean":"2024-04-17 22:31:41.638378","last_became_active":"2024-04-17 22:28:59.255411","last_became_peered":"2024-04-17 22:28:59.255411","last_unstale":"2024-04-17 22:31:41.638378","last_undegraded":"2024-04-17 22:31:41.638378","last_fullsized":"2024-04-17 22:31:41.638378","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:28:58.239187","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:28:58.239187","last_clean_scrub_stamp":"2024-04-17 22:28:58.239187","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":"25","reported_epoch":"27","state":"active+clean","last_fresh":"2024-04-17 22:31:41.541197","last_change":"2024-04-17 22:28:59.252728","last_active":"2024-04-17 22:31:41.541197","last_peered":"2024-04-17 22:31:41.541197","last_clean":"2024-04-17 22:31:41.541197","last_became_active":"2024-04-17 22:28:59.252345","last_became_peered":"2024-04-17 22:28:59.252345","last_unstale":"2024-04-17 22:31:41.541197","last_undegraded":"2024-04-17 22:31:41.541197","last_fullsized":"2024-04-17 22:31:41.541197","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:28:58.239187","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:28:58.239187","last_clean_scrub_stamp":"2024-04-17 22:28:58.239187","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":"25","reported_epoch":"27","state":"active+clean","last_fresh":"2024-04-17 22:31:41.538254","last_change":"2024-04-17 22:28:59.253392","last_active":"2024-04-17 22:31:41.538254","last_peered":"2024-04-17 22:31:41.538254","last_clean":"2024-04-17 22:31:41.538254","last_became_active":"2024-04-17 22:28:59.253049","last_became_peered":"2024-04-17 22:28:59.253049","last_unstale":"2024-04-17 22:31:41.538254","last_undegraded":"2024-04-17 22:31:41.538254","last_fullsized":"2024-04-17 22:31:41.538254","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:28:58.239187","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:28:58.239187","last_clean_scrub_stamp":"2024-04-17 22:28:58.239187","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":"25","reported_epoch":"27","state":"active+clean","last_fresh":"2024-04-17 22:31:41.539152","last_change":"2024-04-17 22:28:59.254907","last_active":"2024-04-17 22:31:41.539152","last_peered":"2024-04-17 22:31:41.539152","last_clean":"2024-04-17 22:31:41.539152","last_became_active":"2024-04-17 22:28:59.254633","last_became_peered":"2024-04-17 22:28:59.254633","last_unstale":"2024-04-17 22:31:41.539152","last_undegraded":"2024-04-17 22:31:41.539152","last_fullsized":"2024-04-17 22:31:41.539152","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:28:58.239187","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:28:58.239187","last_clean_scrub_stamp":"2024-04-17 22:28:58.239187","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":"25","reported_epoch":"27","state":"active+clean","last_fresh":"2024-04-17 22:31:41.637872","last_change":"2024-04-17 22:28:59.254217","last_active":"2024-04-17 22:31:41.637872","last_peered":"2024-04-17 22:31:41.637872","last_clean":"2024-04-17 22:31:41.637872","last_became_active":"2024-04-17 22:28:59.253877","last_became_peered":"2024-04-17 22:28:59.253877","last_unstale":"2024-04-17 22:31:41.637872","last_undegraded":"2024-04-17 22:31:41.637872","last_fullsized":"2024-04-17 22:31:41.637872","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:28:58.239187","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:28:58.239187","last_clean_scrub_stamp":"2024-04-17 22:28:58.239187","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":"25","reported_epoch":"27","state":"active+clean","last_fresh":"2024-04-17 22:31:41.637690","last_change":"2024-04-17 22:28:59.254830","last_active":"2024-04-17 22:31:41.637690","last_peered":"2024-04-17 22:31:41.637690","last_clean":"2024-04-17 22:31:41.637690","last_became_active":"2024-04-17 22:28:59.254505","last_became_peered":"2024-04-17 22:28:59.254505","last_unstale":"2024-04-17 22:31:41.637690","last_undegraded":"2024-04-17 22:31:41.637690","last_fullsized":"2024-04-17 22:31:41.637690","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:28:58.239187","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:28:58.239187","last_clean_scrub_stamp":"2024-04-17 22:28:58.239187","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":"25","reported_epoch":"27","state":"active+clean","last_fresh":"2024-04-17 22:31:41.544192","last_change":"2024-04-17 22:28:59.254055","last_active":"2024-04-17 22:31:41.544192","last_peered":"2024-04-17 22:31:41.544192","last_clean":"2024-04-17 22:31:41.544192","last_became_active":"2024-04-17 22:28:59.253750","last_became_peered":"2024-04-17 22:28:59.253750","last_unstale":"2024-04-17 22:31:41.544192","last_undegraded":"2024-04-17 22:31:41.544192","last_fullsized":"2024-04-17 22:31:41.544192","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:28:58.239187","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:28:58.239187","last_clean_scrub_stamp":"2024-04-17 22:28:58.239187","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705702,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"kb":94371840,"kb_used":1048848,"kb_used_data":248,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93322992,"statfs":{"total":96636764160,"available":95562743808,"internally_reserved":1073741824,"allocated":253952,"data_stored":79982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 17 22:30:59 2024","interfaces":[{"interface":"back","average":{"1min":0.308,"5min":0.30399999999999999,"15min":0.30399999999999999},"min":{"1min":0.20399999999999999,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"last":0.22900000000000001},{"interface":"front","average":{"1min":0.30199999999999999,"5min":0.314,"15min":0.316},"min":{"1min":0.21099999999999999,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.44600000000000001,"5min":0.52400000000000002,"15min":0.52400000000000002},"last":0.23699999999999999}]},{"osd":1,"last update":"Wed Apr 17 22:30:59 2024","interfaces":[{"interface":"back","average":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"min":{"1min":0.223,"5min":0.19600000000000001,"15min":0.19600000000000001},"max":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"last":0.26500000000000001},{"interface":"front","average":{"1min":0.33400000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"min":{"1min":0.249,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.438,"5min":0.55300000000000005,"15min":0.55300000000000005},"last":0.254}]},{"osd":2,"last update":"Wed Apr 17 22:30:59 2024","interfaces":[{"interface":"back","average":{"1min":0.33800000000000002,"5min":0.34799999999999998,"15min":0.34999999999999998},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.46200000000000002,"5min":0.50600000000000001,"15min":0.50600000000000001},"last":0.27100000000000002},{"interface":"front","average":{"1min":0.32800000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"min":{"1min":0.23200000000000001,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.46899999999999997,"5min":0.47699999999999998,"15min":0.47699999999999998},"last":0.218}]}]},{"osd":2,"up_from":9,"seq":38654705702,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"kb":94371840,"kb_used":1048840,"kb_used_data":248,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323000,"statfs":{"total":96636764160,"available":95562752000,"internally_reserved":1073741824,"allocated":253952,"data_stored":79982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 17 22:30:58 2024","interfaces":[{"interface":"back","average":{"1min":0.30099999999999999,"5min":0.28499999999999998,"15min":0.28299999999999997},"min":{"1min":0.22600000000000001,"5min":0.112,"15min":0.112},"max":{"1min":0.41499999999999998,"5min":0.46400000000000002,"15min":0.46400000000000002},"last":0.26000000000000001},{"interface":"front","average":{"1min":0.33600000000000002,"5min":0.314,"15min":0.31},"min":{"1min":0.27600000000000002,"5min":0.155,"15min":0.155},"max":{"1min":0.40400000000000003,"5min":0.49199999999999999,"15min":0.49199999999999999},"last":0.26600000000000001}]},{"osd":1,"last update":"Wed Apr 17 22:30:58 2024","interfaces":[{"interface":"back","average":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"min":{"1min":0.214,"5min":0.17899999999999999,"15min":0.17899999999999999},"max":{"1min":0.377,"5min":0.52900000000000003,"15min":0.52900000000000003},"last":0.27800000000000002},{"interface":"front","average":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"min":{"1min":0.245,"5min":0.192,"15min":0.192},"max":{"1min":0.38800000000000001,"5min":0.47199999999999998,"15min":0.47199999999999998},"last":0.23499999999999999}]},{"osd":3,"last update":"Wed Apr 17 22:30:58 2024","interfaces":[{"interface":"back","average":{"1min":0.30399999999999999,"5min":0.29599999999999999,"15min":0.29499999999999998},"min":{"1min":0.24199999999999999,"5min":0.191,"15min":0.191},"max":{"1min":0.42599999999999999,"5min":0.45600000000000002,"15min":0.45600000000000002},"last":0.28999999999999998},{"interface":"front","average":{"1min":0.31900000000000001,"5min":0.311,"15min":0.31},"min":{"1min":0.26200000000000001,"5min":0.214,"15min":0.214},"max":{"1min":0.41299999999999998,"5min":0.48699999999999999,"15min":0.48699999999999999},"last":0.30099999999999999}]}]},{"osd":0,"up_from":9,"seq":38654705702,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"kb":94371840,"kb_used":1048832,"kb_used_data":248,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323008,"statfs":{"total":96636764160,"available":95562760192,"internally_reserved":1073741824,"allocated":253952,"data_stored":79982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Apr 17 22:30:56 2024","interfaces":[{"interface":"back","average":{"1min":0.248,"5min":0.28299999999999997,"15min":0.28899999999999998},"min":{"1min":0.159,"5min":0.159,"15min":0.159},"max":{"1min":0.33200000000000002,"5min":0.40899999999999997,"15min":0.40899999999999997},"last":0.245},{"interface":"front","average":{"1min":0.26000000000000001,"5min":0.29199999999999998,"15min":0.29699999999999999},"min":{"1min":0.189,"5min":0.16600000000000001,"15min":0.16600000000000001},"max":{"1min":0.35399999999999998,"5min":0.42999999999999999,"15min":0.42999999999999999},"last":0.249}]},{"osd":2,"last update":"Wed Apr 17 22:30:56 2024","interfaces":[{"interface":"back","average":{"1min":0.28899999999999998,"5min":0.32100000000000001,"15min":0.32700000000000001},"min":{"1min":0.218,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.39500000000000002,"5min":0.48599999999999999,"15min":0.48599999999999999},"last":0.27200000000000002},{"interface":"front","average":{"1min":0.27800000000000002,"5min":0.317,"15min":0.32300000000000001},"min":{"1min":0.19400000000000001,"5min":0.19400000000000001,"15min":0.19400000000000001},"max":{"1min":0.371,"5min":0.60699999999999998,"15min":0.60699999999999998},"last":0.22900000000000001}]},{"osd":3,"last update":"Wed Apr 17 22:30:56 2024","interfaces":[{"interface":"back","average":{"1min":0.311,"5min":0.33400000000000002,"15min":0.33800000000000002},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.40600000000000003,"5min":0.505,"15min":0.505},"last":0.26000000000000001},{"interface":"front","average":{"1min":0.28599999999999998,"5min":0.32000000000000001,"15min":0.32600000000000001},"min":{"1min":0.20000000000000001,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.38300000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"last":0.219}]}]},{"osd":1,"up_from":9,"seq":38654705701,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"kb":94371840,"kb_used":1048832,"kb_used_data":248,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323008,"statfs":{"total":96636764160,"available":95562760192,"internally_reserved":1073741824,"allocated":253952,"data_stored":79982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 17 22:31:00 2024","interfaces":[{"interface":"back","average":{"1min":0.26900000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"min":{"1min":0.128,"5min":0.128,"15min":0.128},"max":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"last":0.252},{"interface":"front","average":{"1min":0.28000000000000003,"5min":0.27600000000000002,"15min":0.27600000000000002},"min":{"1min":0.19400000000000001,"5min":0.191,"15min":0.191},"max":{"1min":0.441,"5min":0.441,"15min":0.441},"last":0.26100000000000001}]},{"osd":2,"last update":"Wed Apr 17 22:31:00 2024","interfaces":[{"interface":"back","average":{"1min":0.28599999999999998,"5min":0.27900000000000003,"15min":0.27800000000000002},"min":{"1min":0.153,"5min":0.153,"15min":0.153},"max":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"last":0.374},{"interface":"front","average":{"1min":0.28599999999999998,"5min":0.29099999999999998,"15min":0.29199999999999998},"min":{"1min":0.18099999999999999,"5min":0.18099999999999999,"15min":0.18099999999999999},"max":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"last":0.27300000000000002}]},{"osd":3,"last update":"Wed Apr 17 22:31:00 2024","interfaces":[{"interface":"back","average":{"1min":0.30499999999999999,"5min":0.29699999999999999,"15min":0.29499999999999998},"min":{"1min":0.23300000000000001,"5min":0.218,"15min":0.218},"max":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"last":0.33800000000000002},{"interface":"front","average":{"1min":0.309,"5min":0.315,"15min":0.316},"min":{"1min":0.245,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"last":0.28799999999999998}]}]}]}} 2024-04-17T22:31:48.373 INFO:tasks.ceph:pgid 1.5 last_scrub_stamp 2024-04-17 22:28:58.239187 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=22, tm_min=28, tm_sec=58, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=22, tm_min=31, tm_sec=45, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T22:31:48.374 INFO:tasks.ceph:pgid 1.4 last_scrub_stamp 2024-04-17 22:28:58.239187 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=22, tm_min=28, tm_sec=58, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=22, tm_min=31, tm_sec=45, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T22:31:48.374 INFO:tasks.ceph:pgid 1.3 last_scrub_stamp 2024-04-17 22:28:58.239187 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=22, tm_min=28, tm_sec=58, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=22, tm_min=31, tm_sec=45, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T22:31:48.374 INFO:tasks.ceph:pgid 1.2 last_scrub_stamp 2024-04-17 22:28:58.239187 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=22, tm_min=28, tm_sec=58, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=22, tm_min=31, tm_sec=45, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T22:31:48.374 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-04-17 22:28:58.239187 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=22, tm_min=28, tm_sec=58, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=22, tm_min=31, tm_sec=45, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T22:31:48.374 INFO:tasks.ceph:pgid 1.1 last_scrub_stamp 2024-04-17 22:28:58.239187 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=22, tm_min=28, tm_sec=58, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=22, tm_min=31, tm_sec=45, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T22:31:48.374 INFO:tasks.ceph:pgid 1.6 last_scrub_stamp 2024-04-17 22:28:58.239187 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=22, tm_min=28, tm_sec=58, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=22, tm_min=31, tm_sec=45, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T22:31:48.374 INFO:tasks.ceph:pgid 1.7 last_scrub_stamp 2024-04-17 22:28:58.239187 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=22, tm_min=28, tm_sec=58, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=22, tm_min=31, tm_sec=45, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T22:31:48.374 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-04-17T22:32:08.376 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-17T22:32:08.517 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:08.530 7f7682d6a700 1 Processor -- start 2024-04-17T22:32:08.517 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:08.530 7f7682d6a700 1 -- start start 2024-04-17T22:32:08.517 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:08.530 7f7682d6a700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f767c10e260 0x7f767c10e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:32:08.518 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:08.530 7f7682d6a700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f767c10ece0 0x7f767c0f1750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:32:08.518 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:08.530 7f7682d6a700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f767c0f1d80 0x7f767c1aa4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:32:08.518 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:08.530 7f7682d6a700 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f767c0f9f30 con 0x7f767c10ece0 2024-04-17T22:32:08.518 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:08.530 7f7682d6a700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f767c0f9050 con 0x7f767c10e260 2024-04-17T22:32:08.518 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:08.530 7f7682d6a700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f767c0fa1d0 con 0x7f767c0f1d80 2024-04-17T22:32:08.518 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:08.531 7f767b7fe700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f767c10e260 0x7f767c10e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:32:08.518 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:08.531 7f767bfff700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f767c0f1d80 0x7f767c1aa4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:32:08.518 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:08.531 7f767affd700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f767c10ece0 0x7f767c0f1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:32:08.518 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:08.531 7f767bfff700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f767c0f1d80 0x7f767c1aa4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:36882/0 (socket says 172.21.15.102:36882) 2024-04-17T22:32:08.518 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:08.531 7f767b7fe700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f767c10e260 0x7f767c10e6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:56802/0 (socket says 172.21.15.102:56802) 2024-04-17T22:32:08.518 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:08.531 7f767bfff700 1 -- 172.21.15.102:0/3477394142 learned_addr learned my addr 172.21.15.102:0/3477394142 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:32:08.518 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:08.531 7f767affd700 1 -- 172.21.15.102:0/3477394142 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f767c0f1d80 msgr2=0x7f767c1aa4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:32:08.518 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:08.531 7f767affd700 1 --2- 172.21.15.102:0/3477394142 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f767c0f1d80 0x7f767c1aa4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:32:08.519 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:08.531 7f767affd700 1 -- 172.21.15.102:0/3477394142 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f767c10e260 msgr2=0x7f767c10e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:32:08.519 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:08.531 7f767affd700 1 --2- 172.21.15.102:0/3477394142 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f767c10e260 0x7f767c10e6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:32:08.519 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:08.531 7f767affd700 1 -- 172.21.15.102:0/3477394142 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f767c0f21d0 con 0x7f767c10ece0 2024-04-17T22:32:08.519 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:08.531 7f767bfff700 1 --2- 172.21.15.102:0/3477394142 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f767c0f1d80 0x7f767c1aa4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:32:08.519 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:08.531 7f767affd700 1 --2- 172.21.15.102:0/3477394142 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f767c10ece0 0x7f767c0f1750 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 rx=0x7f767c0f46c0 tx=0x7f766400ba50).ready entity=mon.0 client_cookie=10ba6f493cd6d8e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:32:08.519 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:08.531 7f767a7fc700 1 -- 172.21.15.102:0/3477394142 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f767c0f21d0 con 0x7f767c10ece0 2024-04-17T22:32:08.519 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:08.531 7f767a7fc700 1 -- 172.21.15.102:0/3477394142 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f766400d570 con 0x7f767c10ece0 2024-04-17T22:32:08.519 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:08.531 7f767a7fc700 1 -- 172.21.15.102:0/3477394142 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f766400d940 con 0x7f767c10ece0 2024-04-17T22:32:08.519 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:08.531 7f7682d6a700 1 -- 172.21.15.102:0/3477394142 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f767c10ece0 msgr2=0x7f767c0f1750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:32:08.519 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:08.531 7f7682d6a700 1 --2- 172.21.15.102:0/3477394142 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f767c10ece0 0x7f767c0f1750 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 rx=0x7f767c0f46c0 tx=0x7f766400ba50).stop 2024-04-17T22:32:08.519 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:08.532 7f7682d6a700 1 -- 172.21.15.102:0/3477394142 shutdown_connections 2024-04-17T22:32:08.519 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:08.532 7f7682d6a700 1 --2- 172.21.15.102:0/3477394142 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f767c0f1d80 0x7f767c1aa4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:32:08.519 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:08.532 7f7682d6a700 1 --2- 172.21.15.102:0/3477394142 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f767c10e260 0x7f767c10e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:32:08.520 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:08.532 7f7682d6a700 1 --2- 172.21.15.102:0/3477394142 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f767c10ece0 0x7f767c0f1750 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:32:08.520 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:08.532 7f7682d6a700 1 -- 172.21.15.102:0/3477394142 >> 172.21.15.102:0/3477394142 conn(0x7f767c1138c0 msgr2=0x7f767c0f4f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:32:08.520 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:08.532 7f7682d6a700 1 -- 172.21.15.102:0/3477394142 shutdown_connections 2024-04-17T22:32:08.520 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:08.532 7f7682d6a700 1 -- 172.21.15.102:0/3477394142 wait complete. 2024-04-17T22:32:08.541 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:08.555 7f7682d6a700 1 Processor -- start 2024-04-17T22:32:08.541 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:08.555 7f7682d6a700 1 -- start start 2024-04-17T22:32:08.542 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:08.555 7f7682d6a700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f767c0de420 0x7f767c0ca2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:32:08.542 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:08.555 7f7682d6a700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f767c0ca910 0x7f767c1b3be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:32:08.542 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:08.555 7f7682d6a700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f767c0cad60 0x7f767c1b4210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:32:08.542 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:08.555 7f7682d6a700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f767c08f2e0 con 0x7f767c0de420 2024-04-17T22:32:08.542 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:08.555 7f7682d6a700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f767c0d1170 con 0x7f767c0ca910 2024-04-17T22:32:08.542 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:08.555 7f7682d6a700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f767c0d1500 con 0x7f767c0cad60 2024-04-17T22:32:08.542 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:08.555 7f767affd700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f767c0ca910 0x7f767c1b3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:32:08.542 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:08.555 7f767b7fe700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f767c0de420 0x7f767c0ca2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:32:08.542 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:08.555 7f767affd700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f767c0ca910 0x7f767c1b3be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:56810/0 (socket says 172.21.15.102:56810) 2024-04-17T22:32:08.542 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:08.555 7f767b7fe700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f767c0de420 0x7f767c0ca2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:40110/0 (socket says 172.21.15.102:40110) 2024-04-17T22:32:08.542 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:08.555 7f767bfff700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f767c0cad60 0x7f767c1b4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:32:08.542 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:08.555 7f767affd700 1 -- 172.21.15.102:0/2500664380 learned_addr learned my addr 172.21.15.102:0/2500664380 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:32:08.542 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:08.556 7f767bfff700 1 -- 172.21.15.102:0/2500664380 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f767c0ca910 msgr2=0x7f767c1b3be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:32:08.543 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:08.556 7f767bfff700 1 --2- 172.21.15.102:0/2500664380 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f767c0ca910 0x7f767c1b3be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:32:08.543 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:08.556 7f767bfff700 1 -- 172.21.15.102:0/2500664380 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f767c0de420 msgr2=0x7f767c0ca2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:32:08.543 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:08.556 7f767bfff700 1 --2- 172.21.15.102:0/2500664380 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f767c0de420 0x7f767c0ca2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:32:08.543 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:08.556 7f767bfff700 1 -- 172.21.15.102:0/2500664380 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f767c1b4840 con 0x7f767c0cad60 2024-04-17T22:32:08.543 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:08.556 7f767affd700 1 --2- 172.21.15.102:0/2500664380 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f767c0ca910 0x7f767c1b3be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:32:08.543 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:08.556 7f767b7fe700 1 --2- 172.21.15.102:0/2500664380 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f767c0de420 0x7f767c0ca2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:32:08.543 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:08.556 7f767bfff700 1 --2- 172.21.15.102:0/2500664380 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f767c0cad60 0x7f767c1b4210 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 rx=0x7f767c0d6580 tx=0x7f766c00edc0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:32:08.543 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:08.556 7f7678ff9700 1 -- 172.21.15.102:0/2500664380 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f767c1b4840 con 0x7f767c0cad60 2024-04-17T22:32:08.543 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:08.556 7f7678ff9700 1 -- 172.21.15.102:0/2500664380 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f766c00c900 con 0x7f767c0cad60 2024-04-17T22:32:08.543 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:08.556 7f7682d6a700 1 -- 172.21.15.102:0/2500664380 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f767c1b4aa0 con 0x7f767c0cad60 2024-04-17T22:32:08.543 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:08.556 7f7678ff9700 1 -- 172.21.15.102:0/2500664380 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f766c00f040 con 0x7f767c0cad60 2024-04-17T22:32:08.543 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:08.556 7f7682d6a700 1 -- 172.21.15.102:0/2500664380 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f767c1bb5c0 con 0x7f767c0cad60 2024-04-17T22:32:08.543 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:08.556 7f7678ff9700 1 -- 172.21.15.102:0/2500664380 <== mon.2 v2:172.21.15.102:3302/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7f766c022060 con 0x7f767c0cad60 2024-04-17T22:32:08.544 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:08.556 7f7678ff9700 1 -- 172.21.15.102:0/2500664380 <== mon.2 v2:172.21.15.102:3302/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 2417+0+0 (secure 0 0 0) 0x7f766c010040 con 0x7f767c0cad60 2024-04-17T22:32:08.544 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:08.557 7f7682d6a700 1 -- 172.21.15.102:0/2500664380 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f767c0f0820 con 0x7f767c0cad60 2024-04-17T22:32:08.545 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:08.560 7f7678ff9700 1 -- 172.21.15.102:0/2500664380 <== mon.2 v2:172.21.15.102:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+110455 (secure 0 0 0) 0x7f766c00c6f0 con 0x7f767c0cad60 2024-04-17T22:32:08.646 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:08.661 7f7682d6a700 1 -- 172.21.15.102:0/2500664380 --> v1:172.21.15.102:6816/42360 -- command(tid 0: {"prefix": "pg dump", "target": ["mgr", ""], "format": "json"}) v1 -- 0x7f767c08a300 con 0x7f76600145a0 2024-04-17T22:32:08.647 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:08.662 7f7678ff9700 1 -- 172.21.15.102:0/2500664380 <== mgr.4104 v1:172.21.15.102:6816/42360 1 ==== command_reply(tid 0: 0 dumped all) v1 ==== 18+0+32409 (unknown 2965378022 0 3733544200) 0x7f767400bba0 con 0x7f76600145a0 2024-04-17T22:32:08.648 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-17T22:32:08.649 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:08.664 7f7682d6a700 1 -- 172.21.15.102:0/2500664380 >> v1:172.21.15.102:6816/42360 conn(0x7f76600145a0 legacy=0x7f7660016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:32:08.649 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:08.664 7f7682d6a700 1 -- 172.21.15.102:0/2500664380 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f767c0cad60 msgr2=0x7f767c1b4210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:32:08.649 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:08.664 7f7682d6a700 1 --2- 172.21.15.102:0/2500664380 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f767c0cad60 0x7f767c1b4210 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 rx=0x7f767c0d6580 tx=0x7f766c00edc0).stop 2024-04-17T22:32:08.649 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:08.664 7f7682d6a700 1 -- 172.21.15.102:0/2500664380 shutdown_connections 2024-04-17T22:32:08.649 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:08.664 7f7682d6a700 1 --2- 172.21.15.102:0/2500664380 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f767c0cad60 0x7f767c1b4210 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:32:08.649 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:08.664 7f7682d6a700 1 --2- 172.21.15.102:0/2500664380 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f767c0de420 0x7f767c0ca2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:32:08.649 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:08.664 7f7682d6a700 1 --2- 172.21.15.102:0/2500664380 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f767c0ca910 0x7f767c1b3be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:32:08.649 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:08.664 7f7682d6a700 1 -- 172.21.15.102:0/2500664380 >> 172.21.15.102:0/2500664380 conn(0x7f767c1a7050 msgr2=0x7f767c0d69d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:32:08.649 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:08.664 7f7682d6a700 1 -- 172.21.15.102:0/2500664380 shutdown_connections 2024-04-17T22:32:08.649 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:08.664 7f7682d6a700 1 -- 172.21.15.102:0/2500664380 wait complete. 2024-04-17T22:32:08.650 INFO:teuthology.orchestra.run.smithi102.stderr:dumped all 2024-04-17T22:32:08.659 INFO:teuthology.orchestra.run.smithi102.stdout:{"pg_ready":true,"pg_map":{"version":121,"stamp":"2024-04-17 22:32:07.227126","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":4,"num_per_pool_osds":4,"kb":377487360,"kb_used":4195352,"kb_used_data":992,"kb_used_omap":0,"kb_used_meta":4194304,"kb_avail":373292008,"statfs":{"total":386547056640,"available":382251016192,"internally_reserved":4294967296,"allocated":1015808,"data_stored":319928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4294967296},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-2286,"num_objects":-21,"num_object_clones":0,"num_object_copies":-42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-44,"num_write_kb":-13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.309956"},"pg_stats":[{"pgid":"1.5","version":"0'0","reported_seq":"28","reported_epoch":"28","state":"active+clean","last_fresh":"2024-04-17 22:31:47.800008","last_change":"2024-04-17 22:31:47.800052","last_active":"2024-04-17 22:31:47.800008","last_peered":"2024-04-17 22:31:47.800008","last_clean":"2024-04-17 22:31:47.800008","last_became_active":"2024-04-17 22:28:59.253641","last_became_peered":"2024-04-17 22:28:59.253641","last_unstale":"2024-04-17 22:31:47.800008","last_undegraded":"2024-04-17 22:31:47.800008","last_fullsized":"2024-04-17 22:31:47.800008","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:47.800008","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:47.800008","last_clean_scrub_stamp":"2024-04-17 22:31:47.800008","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":"28","reported_epoch":"28","state":"active+clean","last_fresh":"2024-04-17 22:32:01.750332","last_change":"2024-04-17 22:32:01.750392","last_active":"2024-04-17 22:32:01.750332","last_peered":"2024-04-17 22:32:01.750332","last_clean":"2024-04-17 22:32:01.750332","last_became_active":"2024-04-17 22:28:59.255411","last_became_peered":"2024-04-17 22:28:59.255411","last_unstale":"2024-04-17 22:32:01.750332","last_undegraded":"2024-04-17 22:32:01.750332","last_fullsized":"2024-04-17 22:32:01.750332","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:32:01.750332","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:32:01.750332","last_clean_scrub_stamp":"2024-04-17 22:32:01.750332","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":"28","reported_epoch":"28","state":"active+clean","last_fresh":"2024-04-17 22:31:55.787702","last_change":"2024-04-17 22:31:55.787756","last_active":"2024-04-17 22:31:55.787702","last_peered":"2024-04-17 22:31:55.787702","last_clean":"2024-04-17 22:31:55.787702","last_became_active":"2024-04-17 22:28:59.252345","last_became_peered":"2024-04-17 22:28:59.252345","last_unstale":"2024-04-17 22:31:55.787702","last_undegraded":"2024-04-17 22:31:55.787702","last_fullsized":"2024-04-17 22:31:55.787702","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:55.787702","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:55.787702","last_clean_scrub_stamp":"2024-04-17 22:31:55.787702","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":"28","reported_epoch":"28","state":"active+clean","last_fresh":"2024-04-17 22:31:49.134492","last_change":"2024-04-17 22:31:49.134544","last_active":"2024-04-17 22:31:49.134492","last_peered":"2024-04-17 22:31:49.134492","last_clean":"2024-04-17 22:31:49.134492","last_became_active":"2024-04-17 22:28:59.253049","last_became_peered":"2024-04-17 22:28:59.253049","last_unstale":"2024-04-17 22:31:49.134492","last_undegraded":"2024-04-17 22:31:49.134492","last_fullsized":"2024-04-17 22:31:49.134492","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:49.134492","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:49.134492","last_clean_scrub_stamp":"2024-04-17 22:31:49.134492","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":"28","reported_epoch":"28","state":"active+clean","last_fresh":"2024-04-17 22:31:48.804285","last_change":"2024-04-17 22:31:48.804331","last_active":"2024-04-17 22:31:48.804285","last_peered":"2024-04-17 22:31:48.804285","last_clean":"2024-04-17 22:31:48.804285","last_became_active":"2024-04-17 22:28:59.254633","last_became_peered":"2024-04-17 22:28:59.254633","last_unstale":"2024-04-17 22:31:48.804285","last_undegraded":"2024-04-17 22:31:48.804285","last_fullsized":"2024-04-17 22:31:48.804285","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:48.804285","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:48.804285","last_clean_scrub_stamp":"2024-04-17 22:31:48.804285","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":"28","reported_epoch":"28","state":"active+clean","last_fresh":"2024-04-17 22:31:55.720568","last_change":"2024-04-17 22:31:55.720632","last_active":"2024-04-17 22:31:55.720568","last_peered":"2024-04-17 22:31:55.720568","last_clean":"2024-04-17 22:31:55.720568","last_became_active":"2024-04-17 22:28:59.253877","last_became_peered":"2024-04-17 22:28:59.253877","last_unstale":"2024-04-17 22:31:55.720568","last_undegraded":"2024-04-17 22:31:55.720568","last_fullsized":"2024-04-17 22:31:55.720568","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:55.720568","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:55.720568","last_clean_scrub_stamp":"2024-04-17 22:31:55.720568","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":"28","reported_epoch":"28","state":"active+clean","last_fresh":"2024-04-17 22:32:03.722111","last_change":"2024-04-17 22:32:03.722169","last_active":"2024-04-17 22:32:03.722111","last_peered":"2024-04-17 22:32:03.722111","last_clean":"2024-04-17 22:32:03.722111","last_became_active":"2024-04-17 22:28:59.254505","last_became_peered":"2024-04-17 22:28:59.254505","last_unstale":"2024-04-17 22:32:03.722111","last_undegraded":"2024-04-17 22:32:03.722111","last_fullsized":"2024-04-17 22:32:03.722111","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:32:03.722111","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:32:03.722111","last_clean_scrub_stamp":"2024-04-17 22:32:03.722111","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":"28","reported_epoch":"28","state":"active+clean","last_fresh":"2024-04-17 22:31:56.788489","last_change":"2024-04-17 22:31:56.788556","last_active":"2024-04-17 22:31:56.788489","last_peered":"2024-04-17 22:31:56.788489","last_clean":"2024-04-17 22:31:56.788489","last_became_active":"2024-04-17 22:28:59.253750","last_became_peered":"2024-04-17 22:28:59.253750","last_unstale":"2024-04-17 22:31:56.788489","last_undegraded":"2024-04-17 22:31:56.788489","last_fullsized":"2024-04-17 22:31:56.788489","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17 22:31:56.788489","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17 22:31:56.788489","last_clean_scrub_stamp":"2024-04-17 22:31:56.788489","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705706,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"kb":94371840,"kb_used":1048848,"kb_used_data":248,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93322992,"statfs":{"total":96636764160,"available":95562743808,"internally_reserved":1073741824,"allocated":253952,"data_stored":79982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 17 22:32:00 2024","interfaces":[{"interface":"back","average":{"1min":0.28399999999999997,"5min":0.29999999999999999,"15min":0.30199999999999999},"min":{"1min":0.22900000000000001,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.45600000000000002,"5min":0.47299999999999998,"15min":0.47299999999999998},"last":0.224},{"interface":"front","average":{"1min":0.27600000000000002,"5min":0.30499999999999999,"15min":0.313},"min":{"1min":0.20499999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.41399999999999998,"5min":0.52400000000000002,"15min":0.52400000000000002},"last":0.23699999999999999}]},{"osd":1,"last update":"Wed Apr 17 22:32:00 2024","interfaces":[{"interface":"back","average":{"1min":0.29799999999999999,"5min":0.32100000000000001,"15min":0.32500000000000001},"min":{"1min":0.214,"5min":0.19600000000000001,"15min":0.19600000000000001},"max":{"1min":0.44500000000000001,"5min":0.48199999999999998,"15min":0.48199999999999998},"last":0.29199999999999998},{"interface":"front","average":{"1min":0.29599999999999999,"5min":0.32400000000000001,"15min":0.32800000000000001},"min":{"1min":0.19900000000000001,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.441,"5min":0.55300000000000005,"15min":0.55300000000000005},"last":0.25700000000000001}]},{"osd":2,"last update":"Wed Apr 17 22:32:00 2024","interfaces":[{"interface":"back","average":{"1min":0.313,"5min":0.34000000000000002,"15min":0.34699999999999998},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.46000000000000002,"5min":0.50600000000000001,"15min":0.50600000000000001},"last":0.28000000000000003},{"interface":"front","average":{"1min":0.28799999999999998,"5min":0.318,"15min":0.32300000000000001},"min":{"1min":0.20300000000000001,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.45100000000000001,"5min":0.47699999999999998,"15min":0.47699999999999998},"last":0.27300000000000002}]}]},{"osd":2,"up_from":9,"seq":38654705706,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"kb":94371840,"kb_used":1048840,"kb_used_data":248,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323000,"statfs":{"total":96636764160,"available":95562752000,"internally_reserved":1073741824,"allocated":253952,"data_stored":79982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 17 22:32:01 2024","interfaces":[{"interface":"back","average":{"1min":0.253,"5min":0.28000000000000003,"15min":0.28100000000000003},"min":{"1min":0.17799999999999999,"5min":0.112,"15min":0.112},"max":{"1min":0.33600000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.28199999999999997,"5min":0.309,"15min":0.309},"min":{"1min":0.23300000000000001,"5min":0.155,"15min":0.155},"max":{"1min":0.375,"5min":0.49199999999999999,"15min":0.49199999999999999},"last":0.41099999999999998}]},{"osd":1,"last update":"Wed Apr 17 22:32:01 2024","interfaces":[{"interface":"back","average":{"1min":0.252,"5min":0.28899999999999998,"15min":0.29499999999999998},"min":{"1min":0.19800000000000001,"5min":0.17899999999999999,"15min":0.17899999999999999},"max":{"1min":0.33300000000000002,"5min":0.52900000000000003,"15min":0.52900000000000003},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.26100000000000001,"5min":0.29399999999999998,"15min":0.29999999999999999},"min":{"1min":0.20699999999999999,"5min":0.192,"15min":0.192},"max":{"1min":0.35599999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"last":0.378}]},{"osd":3,"last update":"Wed Apr 17 22:32:01 2024","interfaces":[{"interface":"back","average":{"1min":0.25800000000000001,"5min":0.28899999999999998,"15min":0.29299999999999998},"min":{"1min":0.19800000000000001,"5min":0.191,"15min":0.191},"max":{"1min":0.38400000000000001,"5min":0.45600000000000002,"15min":0.45600000000000002},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.27700000000000002,"5min":0.30499999999999999,"15min":0.308},"min":{"1min":0.22500000000000001,"5min":0.214,"15min":0.214},"max":{"1min":0.36599999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"last":0.36699999999999999}]}]},{"osd":0,"up_from":9,"seq":38654705706,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"kb":94371840,"kb_used":1048832,"kb_used_data":248,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323008,"statfs":{"total":96636764160,"available":95562760192,"internally_reserved":1073741824,"allocated":253952,"data_stored":79982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Apr 17 22:31:56 2024","interfaces":[{"interface":"back","average":{"1min":0.247,"5min":0.27400000000000002,"15min":0.28599999999999998},"min":{"1min":0.151,"5min":0.151,"15min":0.151},"max":{"1min":0.36699999999999999,"5min":0.40899999999999997,"15min":0.40899999999999997},"last":0.28699999999999998},{"interface":"front","average":{"1min":0.23899999999999999,"5min":0.27900000000000003,"15min":0.29299999999999998},"min":{"1min":0.14499999999999999,"5min":0.14499999999999999,"15min":0.14499999999999999},"max":{"1min":0.36699999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"last":0.33600000000000002}]},{"osd":2,"last update":"Wed Apr 17 22:31:56 2024","interfaces":[{"interface":"back","average":{"1min":0.26600000000000001,"5min":0.309,"15min":0.32300000000000001},"min":{"1min":0.188,"5min":0.188,"15min":0.188},"max":{"1min":0.40500000000000003,"5min":0.48599999999999999,"15min":0.48599999999999999},"last":0.29699999999999999},{"interface":"front","average":{"1min":0.249,"5min":0.30099999999999999,"15min":0.318},"min":{"1min":0.16700000000000001,"5min":0.16700000000000001,"15min":0.16700000000000001},"max":{"1min":0.39300000000000002,"5min":0.60699999999999998,"15min":0.60699999999999998},"last":0.26500000000000001}]},{"osd":3,"last update":"Wed Apr 17 22:31:56 2024","interfaces":[{"interface":"back","average":{"1min":0.28399999999999997,"5min":0.32300000000000001,"15min":0.33400000000000002},"min":{"1min":0.20300000000000001,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.39200000000000002,"5min":0.505,"15min":0.505},"last":0.29299999999999998},{"interface":"front","average":{"1min":0.25600000000000001,"5min":0.30499999999999999,"15min":0.32100000000000001},"min":{"1min":0.16,"5min":0.16,"15min":0.16},"max":{"1min":0.38300000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"last":0.314}]}]},{"osd":1,"up_from":9,"seq":38654705705,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"kb":94371840,"kb_used":1048832,"kb_used_data":248,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323008,"statfs":{"total":96636764160,"available":95562760192,"internally_reserved":1073741824,"allocated":253952,"data_stored":79982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 17 22:32:01 2024","interfaces":[{"interface":"back","average":{"1min":0.38700000000000001,"5min":0.29499999999999998,"15min":0.28100000000000003},"min":{"1min":0.17399999999999999,"5min":0.128,"15min":0.128},"max":{"1min":2.782,"5min":2.782,"15min":2.782},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.436,"5min":0.308,"15min":0.28599999999999998},"min":{"1min":0.189,"5min":0.189,"15min":0.189},"max":{"1min":3.7250000000000001,"5min":3.7250000000000001,"15min":3.7250000000000001},"last":0.38400000000000001}]},{"osd":2,"last update":"Wed Apr 17 22:32:01 2024","interfaces":[{"interface":"back","average":{"1min":0.38100000000000001,"5min":0.29999999999999999,"15min":0.28499999999999998},"min":{"1min":0.17899999999999999,"5min":0.153,"15min":0.153},"max":{"1min":2.3100000000000001,"5min":2.3100000000000001,"15min":2.3100000000000001},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.25800000000000001,"5min":0.28399999999999997,"15min":0.28999999999999998},"min":{"1min":0.16,"5min":0.16,"15min":0.16},"max":{"1min":0.42699999999999999,"5min":0.45600000000000002,"15min":0.45600000000000002},"last":0.371}]},{"osd":3,"last update":"Wed Apr 17 22:32:01 2024","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.40600000000000003,"15min":0.33200000000000002},"min":{"1min":0.16500000000000001,"5min":0.16500000000000001,"15min":0.16500000000000001},"max":{"1min":11.692,"5min":11.692,"15min":11.692},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.36199999999999999,"15min":0.33200000000000002},"min":{"1min":0.20200000000000001,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":5.3730000000000002,"5min":5.3730000000000002,"15min":5.3730000000000002},"last":0.39300000000000002}]}]}]}} 2024-04-17T22:32:08.661 DEBUG:teuthology.orchestra.run.smithi102:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-04-17T22:32:08.788 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:08.800 7f022c3d1700 1 Processor -- start 2024-04-17T22:32:08.788 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:08.801 7f022c3d1700 1 -- start start 2024-04-17T22:32:08.788 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:08.801 7f022c3d1700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f02240f5840 0x7f022410b010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:32:08.788 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:08.801 7f022c3d1700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f022410b640 0x7f022419faf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:32:08.788 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:08.801 7f022c3d1700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f022410ba90 0x7f02241a0120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:32:08.788 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:08.801 7f022c3d1700 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f022410bee0 con 0x7f022410ba90 2024-04-17T22:32:08.788 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:08.801 7f022c3d1700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02240fa370 con 0x7f02240f5840 2024-04-17T22:32:08.788 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:08.801 7f022c3d1700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0224103b40 con 0x7f022410b640 2024-04-17T22:32:08.788 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:08.801 7f02289cb700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f022410b640 0x7f022419faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:32:08.788 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:08.801 7f02299cd700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f022410ba90 0x7f02241a0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:32:08.788 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:08.801 7f02289cb700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f022410b640 0x7f022419faf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:36908/0 (socket says 172.21.15.102:36908) 2024-04-17T22:32:08.788 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:08.801 7f02289cb700 1 -- 172.21.15.102:0/4235071161 learned_addr learned my addr 172.21.15.102:0/4235071161 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:32:08.789 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:08.801 7f02291cc700 1 --2- 172.21.15.102:0/4235071161 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f02240f5840 0x7f022410b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:32:08.789 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:08.801 7f02299cd700 1 -- 172.21.15.102:0/4235071161 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f022410b640 msgr2=0x7f022419faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:32:08.789 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:08.801 7f02299cd700 1 --2- 172.21.15.102:0/4235071161 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f022410b640 0x7f022419faf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:32:08.789 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:08.801 7f02299cd700 1 -- 172.21.15.102:0/4235071161 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f02240f5840 msgr2=0x7f022410b010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:32:08.789 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:08.801 7f02299cd700 1 --2- 172.21.15.102:0/4235071161 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f02240f5840 0x7f022410b010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:32:08.789 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:08.801 7f02299cd700 1 -- 172.21.15.102:0/4235071161 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02241a0750 con 0x7f022410ba90 2024-04-17T22:32:08.789 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:08.801 7f02291cc700 1 --2- 172.21.15.102:0/4235071161 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f02240f5840 0x7f022410b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:32:08.789 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:08.801 7f02289cb700 1 --2- 172.21.15.102:0/4235071161 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f022410b640 0x7f022419faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:32:08.789 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:08.802 7f02299cd700 1 --2- 172.21.15.102:0/4235071161 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f022410ba90 0x7f02241a0120 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 rx=0x7f02240fa520 tx=0x7f021c009d40).ready entity=mon.0 client_cookie=101d66a7b407b519 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:32:08.789 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:08.802 7f0213fff700 1 -- 172.21.15.102:0/4235071161 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f02241a0750 con 0x7f022410ba90 2024-04-17T22:32:08.789 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:08.802 7f0213fff700 1 -- 172.21.15.102:0/4235071161 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f021c00b7c0 con 0x7f022410ba90 2024-04-17T22:32:08.789 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:08.802 7f0213fff700 1 -- 172.21.15.102:0/4235071161 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f021c00b9a0 con 0x7f022410ba90 2024-04-17T22:32:08.789 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:08.802 7f022c3d1700 1 -- 172.21.15.102:0/4235071161 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f022410ba90 msgr2=0x7f02241a0120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:32:08.789 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:08.802 7f022c3d1700 1 --2- 172.21.15.102:0/4235071161 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f022410ba90 0x7f02241a0120 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 rx=0x7f02240fa520 tx=0x7f021c009d40).stop 2024-04-17T22:32:08.789 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:08.802 7f022c3d1700 1 -- 172.21.15.102:0/4235071161 shutdown_connections 2024-04-17T22:32:08.790 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:08.802 7f022c3d1700 1 --2- 172.21.15.102:0/4235071161 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f022410ba90 0x7f02241a0120 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:32:08.790 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:08.802 7f022c3d1700 1 --2- 172.21.15.102:0/4235071161 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f02240f5840 0x7f022410b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:32:08.790 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:08.802 7f022c3d1700 1 --2- 172.21.15.102:0/4235071161 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f022410b640 0x7f022419faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:32:08.790 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:08.802 7f022c3d1700 1 -- 172.21.15.102:0/4235071161 >> 172.21.15.102:0/4235071161 conn(0x7f02241138a0 msgr2=0x7f02240f4f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:32:08.790 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:08.802 7f022c3d1700 1 -- 172.21.15.102:0/4235071161 shutdown_connections 2024-04-17T22:32:08.790 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:08.802 7f022c3d1700 1 -- 172.21.15.102:0/4235071161 wait complete. 2024-04-17T22:32:08.810 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:08.825 7f022c3d1700 1 Processor -- start 2024-04-17T22:32:08.810 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:08.825 7f022c3d1700 1 -- start start 2024-04-17T22:32:08.812 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:08.826 7f022c3d1700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f02240d2640 0x7f02240dcbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:32:08.812 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:08.826 7f022c3d1700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f02240dd200 0x7f02241ab0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:32:08.813 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:08.826 7f022c3d1700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f02240dd650 0x7f02241ab710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:32:08.813 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:08.826 7f022c3d1700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02240c4090 con 0x7f02240d2640 2024-04-17T22:32:08.813 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:08.826 7f022c3d1700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02240f2310 con 0x7f02240dd200 2024-04-17T22:32:08.813 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:08.826 7f022c3d1700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f02241abd40 con 0x7f02240dd650 2024-04-17T22:32:08.813 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:08.826 7f02291cc700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f02240d2640 0x7f02240dcbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:32:08.813 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:08.826 7f02289cb700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f02240dd200 0x7f02241ab0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:32:08.813 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:08.826 7f02291cc700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f02240d2640 0x7f02240dcbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:40124/0 (socket says 172.21.15.102:40124) 2024-04-17T22:32:08.813 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:08.826 7f02291cc700 1 -- 172.21.15.102:0/126775779 learned_addr learned my addr 172.21.15.102:0/126775779 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:32:08.813 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:08.826 7f02299cd700 1 --2- 172.21.15.102:0/126775779 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f02240dd650 0x7f02241ab710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:32:08.813 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:08.826 7f02291cc700 1 -- 172.21.15.102:0/126775779 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f02240dd650 msgr2=0x7f02241ab710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:32:08.813 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:08.826 7f02291cc700 1 --2- 172.21.15.102:0/126775779 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f02240dd650 0x7f02241ab710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:32:08.813 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:08.826 7f02291cc700 1 -- 172.21.15.102:0/126775779 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f02240dd200 msgr2=0x7f02241ab0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:32:08.813 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:08.826 7f02291cc700 1 --2- 172.21.15.102:0/126775779 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f02240dd200 0x7f02241ab0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:32:08.814 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:08.826 7f02291cc700 1 -- 172.21.15.102:0/126775779 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02241b9170 con 0x7f02240d2640 2024-04-17T22:32:08.814 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:08.826 7f02289cb700 1 --2- 172.21.15.102:0/126775779 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f02240dd200 0x7f02241ab0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:32:08.814 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:08.826 7f02291cc700 1 --2- 172.21.15.102:0/126775779 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f02240d2640 0x7f02240dcbd0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 rx=0x7f02240c44a0 tx=0x7f021400ed00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:32:08.814 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:08.826 7f02127fc700 1 -- 172.21.15.102:0/126775779 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f02241b9170 con 0x7f02240d2640 2024-04-17T22:32:08.814 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:08.826 7f02127fc700 1 -- 172.21.15.102:0/126775779 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f021400c940 con 0x7f02240d2640 2024-04-17T22:32:08.814 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:08.826 7f022c3d1700 1 -- 172.21.15.102:0/126775779 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02241b9370 con 0x7f02240d2640 2024-04-17T22:32:08.814 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:08.826 7f02127fc700 1 -- 172.21.15.102:0/126775779 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f021400f040 con 0x7f02240d2640 2024-04-17T22:32:08.814 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:08.827 7f022c3d1700 1 -- 172.21.15.102:0/126775779 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02241b9750 con 0x7f02240d2640 2024-04-17T22:32:08.814 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:08.827 7f02127fc700 1 -- 172.21.15.102:0/126775779 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7f0214011070 con 0x7f02240d2640 2024-04-17T22:32:08.814 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:08.827 7f02127fc700 1 -- 172.21.15.102:0/126775779 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 2417+0+0 (secure 0 0 0) 0x7f0214010040 con 0x7f02240d2640 2024-04-17T22:32:08.814 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:08.828 7f022c3d1700 1 -- 172.21.15.102:0/126775779 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02240f0800 con 0x7f02240d2640 2024-04-17T22:32:08.815 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:08.831 7f02127fc700 1 -- 172.21.15.102:0/126775779 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+110455 (secure 0 0 0) 0x7f021400c540 con 0x7f02240d2640 2024-04-17T22:32:08.920 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:08.935 7f022c3d1700 1 -- 172.21.15.102:0/126775779 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7f02240e21f0 con 0x7f02240d2640 2024-04-17T22:32:08.922 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:08.938 7f02127fc700 1 -- 172.21.15.102:0/126775779 <== mon.0 v2:172.21.15.102:3300/0 7 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f021401ad50 con 0x7f02240d2640 2024-04-17T22:32:08.923 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:08.938 7f02127fc700 1 -- 172.21.15.102:0/126775779 <== mon.0 v2:172.21.15.102:3300/0 8 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v4)=0 v4) v1 ==== 123+0+0 (secure 0 0 0) 0x7f0214000c30 con 0x7f02240d2640 2024-04-17T22:32:08.925 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:08.940 7f022c3d1700 1 -- 172.21.15.102:0/126775779 >> v1:172.21.15.102:6816/42360 conn(0x7f02040145a0 legacy=0x7f0204016af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:32:08.925 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:08.940 7f022c3d1700 1 -- 172.21.15.102:0/126775779 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f02240d2640 msgr2=0x7f02240dcbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:32:08.925 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:08.940 7f022c3d1700 1 --2- 172.21.15.102:0/126775779 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f02240d2640 0x7f02240dcbd0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 rx=0x7f02240c44a0 tx=0x7f021400ed00).stop 2024-04-17T22:32:08.925 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:08.940 7f022c3d1700 1 -- 172.21.15.102:0/126775779 shutdown_connections 2024-04-17T22:32:08.925 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:08.940 7f022c3d1700 1 --2- 172.21.15.102:0/126775779 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f02240dd650 0x7f02241ab710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:32:08.926 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:08.940 7f022c3d1700 1 --2- 172.21.15.102:0/126775779 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f02240d2640 0x7f02240dcbd0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:32:08.926 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:08.940 7f022c3d1700 1 --2- 172.21.15.102:0/126775779 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f02240dd200 0x7f02241ab0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:32:08.926 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:08.940 7f022c3d1700 1 -- 172.21.15.102:0/126775779 >> 172.21.15.102:0/126775779 conn(0x7f02240c5ac0 msgr2=0x7f02241a4920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:32:08.926 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:08.940 7f022c3d1700 1 -- 172.21.15.102:0/126775779 shutdown_connections 2024-04-17T22:32:08.926 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:08.940 7f022c3d1700 1 -- 172.21.15.102:0/126775779 wait complete. 2024-04-17T22:32:08.936 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-17T22:32:08.937 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T22:32:09.063 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:09.076 7f5bc8243700 1 Processor -- start 2024-04-17T22:32:09.063 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:09.076 7f5bc8243700 1 -- start start 2024-04-17T22:32:09.063 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:09.077 7f5bc8243700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f5bc01a6440 0x7f5bc01a88a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:32:09.064 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:09.077 7f5bc8243700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5bc01a8ed0 0x7f5bc01ab330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-04-17T22:32:09.064 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:09.077 7f5bc8243700 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5bc010ee20 con 0x7f5bc010e1d0 2024-04-17T22:32:09.064 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:09.077 7f5bc8243700 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5bc00f9ea0 con 0x7f5bc01a6440 2024-04-17T22:32:09.064 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:09.077 7f5bc8243700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5bc00f8fc0 con 0x7f5bc01a8ed0 2024-04-17T22:32:09.064 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:09.077 7f5bc583f700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5bc01a8ed0 0x7f5bc01ab330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:32:09.064 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:09.077 7f5bc583f700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5bc01a8ed0 0x7f5bc01ab330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:36922/0 (socket says 172.21.15.102:36922) 2024-04-17T22:32:09.064 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:09.077 7f5bc483d700 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f5bc01a6440 0x7f5bc01a88a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:32:09.064 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:09.077 7f5bc583f700 1 -- 172.21.15.102:0/3606244418 learned_addr learned my addr 172.21.15.102:0/3606244418 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:32:09.064 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:09.077 7f5bc583f700 1 -- 172.21.15.102:0/3606244418 >> v2:172.21.15.102:3300/0 conn(0x7f5bc01a6440 msgr2=0x7f5bc01a88a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T22:32:09.064 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:09.077 7f5bc583f700 1 --2- 172.21.15.102:0/3606244418 >> v2:172.21.15.102:3300/0 conn(0x7f5bc01a6440 0x7f5bc01a88a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:32:09.064 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:09.077 7f5bc583f700 1 -- 172.21.15.102:0/3606244418 >> v1:172.21.15.102:6789/0 conn(0x7f5bc010e1d0 legacy=0x7f5bc010e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:32:09.064 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:09.077 7f5bc583f700 1 -- 172.21.15.102:0/3606244418 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bc01ac970 con 0x7f5bc01a8ed0 2024-04-17T22:32:09.064 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:09.077 7f5bb3fff700 1 -- 172.21.15.102:0/3606244418 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1088304089 0 0) 0x7f5bb4001110 con 0x7f5bc010e1d0 2024-04-17T22:32:09.064 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:09.077 7f5bc483d700 1 --2- 172.21.15.102:0/3606244418 >> v2:172.21.15.102:3300/0 conn(0x7f5bc01a6440 0x7f5bc01a88a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:32:09.065 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:09.077 7f5bc583f700 1 --2- 172.21.15.102:0/3606244418 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5bc01a8ed0 0x7f5bc01ab330 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 rx=0x7f5bc00f4400 tx=0x7f5bb8009d90).ready entity=mon.2 client_cookie=9433b01c4a11e14d server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:32:09.065 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:09.077 7f5bb3fff700 1 -- 172.21.15.102:0/3606244418 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f5bc01ac970 con 0x7f5bc01a8ed0 2024-04-17T22:32:09.065 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:09.077 7f5bb3fff700 1 -- 172.21.15.102:0/3606244418 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f5bb800b840 con 0x7f5bc01a8ed0 2024-04-17T22:32:09.065 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:09.077 7f5bb3fff700 1 -- 172.21.15.102:0/3606244418 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f5bb800ba20 con 0x7f5bc01a8ed0 2024-04-17T22:32:09.065 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:09.078 7f5bc8243700 1 -- 172.21.15.102:0/3606244418 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5bc01a8ed0 msgr2=0x7f5bc01ab330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:32:09.065 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:09.078 7f5bc8243700 1 --2- 172.21.15.102:0/3606244418 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5bc01a8ed0 0x7f5bc01ab330 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 rx=0x7f5bc00f4400 tx=0x7f5bb8009d90).stop 2024-04-17T22:32:09.065 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:09.078 7f5bc8243700 1 -- 172.21.15.102:0/3606244418 shutdown_connections 2024-04-17T22:32:09.065 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:09.078 7f5bc8243700 1 --2- 172.21.15.102:0/3606244418 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5bc01a8ed0 0x7f5bc01ab330 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:32:09.065 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:09.078 7f5bc8243700 1 --2- 172.21.15.102:0/3606244418 >> v2:172.21.15.102:3300/0 conn(0x7f5bc01a6440 0x7f5bc01a88a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-04-17T22:32:09.065 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:09.078 7f5bc8243700 1 -- 172.21.15.102:0/3606244418 >> 172.21.15.102:0/3606244418 conn(0x7f5bc0113830 msgr2=0x7f5bc00f4ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:32:09.065 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:09.078 7f5bc8243700 1 -- 172.21.15.102:0/3606244418 shutdown_connections 2024-04-17T22:32:09.065 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:09.078 7f5bc8243700 1 -- 172.21.15.102:0/3606244418 wait complete. 2024-04-17T22:32:09.088 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:09.101 7f5bc8243700 1 Processor -- start 2024-04-17T22:32:09.088 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:09.101 7f5bc8243700 1 -- start start 2024-04-17T22:32:09.088 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:09.101 7f5bc8243700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5bc00d5310 0x7f5bc00d5760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:32:09.088 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:09.101 7f5bc8243700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5bc00d5d90 0x7f5bc01b4230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:32:09.089 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:09.101 7f5bc8243700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5bc01b4860 0x7f5bc01b4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-04-17T22:32:09.089 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:09.101 7f5bc8243700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bc0089b50 con 0x7f5bc00d5310 2024-04-17T22:32:09.089 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:09.101 7f5bc8243700 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bc00fa140 con 0x7f5bc00d5d90 2024-04-17T22:32:09.089 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:09.101 7f5bc8243700 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5bc00cb5c0 con 0x7f5bc01b4860 2024-04-17T22:32:09.089 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:09.102 7f5bc503e700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5bc00d5310 0x7f5bc00d5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:32:09.089 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:09.102 7f5bc503e700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5bc00d5310 0x7f5bc00d5760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:40146/0 (socket says 172.21.15.102:40146) 2024-04-17T22:32:09.089 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:09.102 7f5bc483d700 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5bc00d5d90 0x7f5bc01b4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:32:09.089 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:09.102 7f5bc503e700 1 -- 172.21.15.102:0/188318915 learned_addr learned my addr 172.21.15.102:0/188318915 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-17T22:32:09.089 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:09.102 7f5bc583f700 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5bc01b4860 0x7f5bc01b4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-17T22:32:09.089 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:09.102 7f5bc503e700 1 -- 172.21.15.102:0/188318915 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5bc01b4860 msgr2=0x7f5bc01b4cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:32:09.089 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:09.102 7f5bc503e700 1 --2- 172.21.15.102:0/188318915 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5bc01b4860 0x7f5bc01b4cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:32:09.089 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:09.102 7f5bc503e700 1 -- 172.21.15.102:0/188318915 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5bc00d5d90 msgr2=0x7f5bc01b4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:32:09.089 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:09.102 7f5bc503e700 1 --2- 172.21.15.102:0/188318915 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5bc00d5d90 0x7f5bc01b4230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:32:09.090 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:09.102 7f5bc503e700 1 -- 172.21.15.102:0/188318915 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bc01bba30 con 0x7f5bc00d5310 2024-04-17T22:32:09.090 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:09.102 7f5bc483d700 1 --2- 172.21.15.102:0/188318915 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5bc00d5d90 0x7f5bc01b4230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:32:09.090 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:09.102 7f5bc583f700 1 --2- 172.21.15.102:0/188318915 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5bc01b4860 0x7f5bc01b4cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-04-17T22:32:09.090 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:09.102 7f5bc503e700 1 --2- 172.21.15.102:0/188318915 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5bc00d5310 0x7f5bc00d5760 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 rx=0x7f5bc00c71c0 tx=0x7f5bb400c870).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T22:32:09.090 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:09.102 7f5bb27fc700 1 -- 172.21.15.102:0/188318915 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f5bc01bba30 con 0x7f5bc00d5310 2024-04-17T22:32:09.090 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:09.102 7f5bb27fc700 1 -- 172.21.15.102:0/188318915 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f5bb4007240 con 0x7f5bc00d5310 2024-04-17T22:32:09.090 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:09.102 7f5bc8243700 1 -- 172.21.15.102:0/188318915 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5bc01bbc30 con 0x7f5bc00d5310 2024-04-17T22:32:09.090 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:09.102 7f5bb27fc700 1 -- 172.21.15.102:0/188318915 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 377+0+0 (secure 0 0 0) 0x7f5bb4006710 con 0x7f5bc00d5310 2024-04-17T22:32:09.090 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:09.102 7f5bc8243700 1 -- 172.21.15.102:0/188318915 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bc01bbe30 con 0x7f5bc00d5310 2024-04-17T22:32:09.090 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:09.103 7f5bb27fc700 1 -- 172.21.15.102:0/188318915 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 3) v1 ==== 14991+0+0 (secure 0 0 0) 0x7f5bb401a020 con 0x7f5bc00d5310 2024-04-17T22:32:09.090 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:09.103 7f5bb27fc700 1 -- 172.21.15.102:0/188318915 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 2417+0+0 (secure 0 0 0) 0x7f5bb4017c90 con 0x7f5bc00d5310 2024-04-17T22:32:09.090 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:09.103 7f5bc8243700 1 -- 172.21.15.102:0/188318915 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5bc00f0790 con 0x7f5bc00d5310 2024-04-17T22:32:09.091 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:09.107 7f5bb27fc700 1 -- 172.21.15.102:0/188318915 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+110455 (secure 0 0 0) 0x7f5bb4007030 con 0x7f5bc00d5310 2024-04-17T22:32:09.227 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:09.242 7f5bc8243700 1 -- 172.21.15.102:0/188318915 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5bc00f54c0 con 0x7f5bc00d5310 2024-04-17T22:32:09.227 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:09.242 7f5bb27fc700 1 -- 172.21.15.102:0/188318915 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) v1 ==== 94+0+782 (secure 0 0 0) 0x7f5bb4000c30 con 0x7f5bc00d5310 2024-04-17T22:32:09.228 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-17T22:32:09.228 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":14,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5251,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.102:6817/3506554335","addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6817","nonce":3506554335}]},"export_targets":[],"features":4611087854035861503,"flags":0,"epoch":14}],"filesystems":[]} 2024-04-17T22:32:09.229 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:09.244 7f5bc8243700 1 -- 172.21.15.102:0/188318915 >> v1:172.21.15.102:6816/42360 conn(0x7f5ba00146b0 legacy=0x7f5ba0016c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:32:09.229 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:09.244 7f5bc8243700 1 -- 172.21.15.102:0/188318915 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5bc00d5310 msgr2=0x7f5bc00d5760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T22:32:09.229 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:09.244 7f5bc8243700 1 --2- 172.21.15.102:0/188318915 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5bc00d5310 0x7f5bc00d5760 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 rx=0x7f5bc00c71c0 tx=0x7f5bb400c870).stop 2024-04-17T22:32:09.229 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:09.244 7f5bc8243700 1 -- 172.21.15.102:0/188318915 shutdown_connections 2024-04-17T22:32:09.229 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:09.244 7f5bc8243700 1 --2- 172.21.15.102:0/188318915 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5bc01b4860 0x7f5bc01b4cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:32:09.229 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:09.244 7f5bc8243700 1 --2- 172.21.15.102:0/188318915 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5bc00d5310 0x7f5bc00d5760 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:32:09.229 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:09.244 7f5bc8243700 1 --2- 172.21.15.102:0/188318915 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5bc00d5d90 0x7f5bc01b4230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-04-17T22:32:09.229 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:09.244 7f5bc8243700 1 -- 172.21.15.102:0/188318915 >> 172.21.15.102:0/188318915 conn(0x7f5bc00c7cd0 msgr2=0x7f5bc00d6340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T22:32:09.229 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:09.244 7f5bc8243700 1 -- 172.21.15.102:0/188318915 shutdown_connections 2024-04-17T22:32:09.229 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-17 22:32:09.244 7f5bc8243700 1 -- 172.21.15.102:0/188318915 wait complete. 2024-04-17T22:32:09.230 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 14 2024-04-17T22:32:09.239 DEBUG:tasks.cephfs.filesystem:already dead... 2024-04-17T22:32:09.239 INFO:teuthology.misc:Shutting down mds daemons... 2024-04-17T22:32:09.240 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-17T22:32:09.240 INFO:teuthology.orchestra.run:waiting for 300 2024-04-17T22:32:09.247 INFO:tasks.ceph.mds.a:Stopped 2024-04-17T22:32:09.247 INFO:teuthology.misc:Shutting down osd daemons... 2024-04-17T22:32:09.248 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-04-17T22:32:09.248 INFO:teuthology.orchestra.run:waiting for 300 2024-04-17T22:32:09.311 INFO:tasks.ceph.osd.0:Stopped 2024-04-17T22:32:09.312 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-04-17T22:32:09.312 INFO:teuthology.orchestra.run:waiting for 300 2024-04-17T22:32:09.382 INFO:tasks.ceph.osd.1:Stopped 2024-04-17T22:32:09.383 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-04-17T22:32:09.383 INFO:teuthology.orchestra.run:waiting for 300 2024-04-17T22:32:09.450 INFO:tasks.ceph.osd.2:Stopped 2024-04-17T22:32:09.450 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-04-17T22:32:09.451 INFO:teuthology.orchestra.run:waiting for 300 2024-04-17T22:32:09.514 INFO:tasks.ceph.osd.3:Stopped 2024-04-17T22:32:09.514 INFO:teuthology.misc:Shutting down mgr daemons... 2024-04-17T22:32:09.515 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-04-17T22:32:09.515 INFO:teuthology.orchestra.run:waiting for 300 2024-04-17T22:32:09.572 INFO:tasks.ceph.mgr.x:Stopped 2024-04-17T22:32:09.572 INFO:teuthology.misc:Shutting down mon daemons... 2024-04-17T22:32:09.572 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-04-17T22:32:09.572 INFO:teuthology.orchestra.run:waiting for 300 2024-04-17T22:32:09.622 INFO:tasks.ceph.mon.a:Stopped 2024-04-17T22:32:09.622 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-04-17T22:32:09.622 INFO:teuthology.orchestra.run:waiting for 300 2024-04-17T22:32:09.671 INFO:tasks.ceph.mon.b:Stopped 2024-04-17T22:32:09.672 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-04-17T22:32:09.672 INFO:teuthology.orchestra.run:waiting for 300 2024-04-17T22:32:09.721 INFO:tasks.ceph.mon.c:Stopped 2024-04-17T22:32:09.721 INFO:tasks.ceph:Checking cluster log for badness... 2024-04-17T22:32:09.721 DEBUG:teuthology.orchestra.run.smithi102:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v 'overall HEALTH_' | egrep -v '\(FS_' | egrep -v '\(MDS_' | egrep -v '\(OSD_' | egrep -v '\(MON_DOWN\)' | egrep -v '\(CACHE_POOL_' | egrep -v '\(POOL_' | egrep -v '\(MGR_DOWN\)' | egrep -v '\(PG_' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v 'Monitor daemon marked osd' | egrep -v 'Behind on trimming' | egrep -v 'Manager daemon' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | head -n 1 2024-04-17T22:32:09.796 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi102.front.sepia.ceph.com 2024-04-17T22:32:09.796 DEBUG:teuthology.orchestra.run.smithi102:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-04-17T22:32:10.057 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi102.front.sepia.ceph.com 2024-04-17T22:32:10.064 DEBUG:teuthology.orchestra.run.smithi102:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-04-17T22:32:10.139 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi102.front.sepia.ceph.com 2024-04-17T22:32:10.140 DEBUG:teuthology.orchestra.run.smithi102:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-04-17T22:32:10.213 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi102.front.sepia.ceph.com 2024-04-17T22:32:10.214 DEBUG:teuthology.orchestra.run.smithi102:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-04-17T22:32:10.309 INFO:tasks.ceph:Archiving mon data... 2024-04-17T22:32:10.310 DEBUG:teuthology.misc:Transferring archived files from smithi102:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/teuthology-2024-04-17_21:24:02-fs-squid-distro-default-smithi/7661154/data/mon.a.tgz 2024-04-17T22:32:10.310 DEBUG:teuthology.orchestra.run.smithi102:> mktemp 2024-04-17T22:32:10.327 INFO:teuthology.orchestra.run.smithi102.stdout:/tmp/tmp.uwjH9v8wdl 2024-04-17T22:32:10.327 DEBUG:teuthology.orchestra.run.smithi102:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.uwjH9v8wdl 2024-04-17T22:32:10.492 DEBUG:teuthology.orchestra.run.smithi102:> sudo chmod 0666 /tmp/tmp.uwjH9v8wdl 2024-04-17T22:32:10.578 DEBUG:teuthology.orchestra.remote:smithi102:/tmp/tmp.uwjH9v8wdl is 322KB 2024-04-17T22:32:10.674 DEBUG:teuthology.orchestra.run.smithi102:> rm -fr /tmp/tmp.uwjH9v8wdl 2024-04-17T22:32:10.691 DEBUG:teuthology.misc:Transferring archived files from smithi102:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/teuthology-2024-04-17_21:24:02-fs-squid-distro-default-smithi/7661154/data/mon.b.tgz 2024-04-17T22:32:10.692 DEBUG:teuthology.orchestra.run.smithi102:> mktemp 2024-04-17T22:32:10.747 INFO:teuthology.orchestra.run.smithi102.stdout:/tmp/tmp.O8p0gjVqWX 2024-04-17T22:32:10.748 DEBUG:teuthology.orchestra.run.smithi102:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.O8p0gjVqWX 2024-04-17T22:32:10.904 DEBUG:teuthology.orchestra.run.smithi102:> sudo chmod 0666 /tmp/tmp.O8p0gjVqWX 2024-04-17T22:32:10.992 DEBUG:teuthology.orchestra.remote:smithi102:/tmp/tmp.O8p0gjVqWX is 413KB 2024-04-17T22:32:11.087 DEBUG:teuthology.orchestra.run.smithi102:> rm -fr /tmp/tmp.O8p0gjVqWX 2024-04-17T22:32:11.104 DEBUG:teuthology.misc:Transferring archived files from smithi102:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/teuthology-2024-04-17_21:24:02-fs-squid-distro-default-smithi/7661154/data/mon.c.tgz 2024-04-17T22:32:11.105 DEBUG:teuthology.orchestra.run.smithi102:> mktemp 2024-04-17T22:32:11.161 INFO:teuthology.orchestra.run.smithi102.stdout:/tmp/tmp.ShRUSFo9jE 2024-04-17T22:32:11.161 DEBUG:teuthology.orchestra.run.smithi102:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.ShRUSFo9jE 2024-04-17T22:32:11.317 DEBUG:teuthology.orchestra.run.smithi102:> sudo chmod 0666 /tmp/tmp.ShRUSFo9jE 2024-04-17T22:32:11.405 DEBUG:teuthology.orchestra.remote:smithi102:/tmp/tmp.ShRUSFo9jE is 414KB 2024-04-17T22:32:11.501 DEBUG:teuthology.orchestra.run.smithi102:> rm -fr /tmp/tmp.ShRUSFo9jE 2024-04-17T22:32:11.519 INFO:tasks.ceph:Cleaning ceph cluster... 2024-04-17T22:32:11.519 DEBUG:teuthology.orchestra.run.smithi102:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-17T22:32:11.561 DEBUG:teuthology.orchestra.run.smithi190:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-17T22:32:11.643 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-17T22:32:11.665 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-17T22:32:11.665 INFO:tasks.ceph:Archiving crash dumps... 2024-04-17T22:32:11.667 DEBUG:teuthology.misc:Transferring archived files from smithi102:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-04-17_21:24:02-fs-squid-distro-default-smithi/7661154/remote/smithi102/crash 2024-04-17T22:32:11.668 DEBUG:teuthology.orchestra.run.smithi102:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-17T22:32:11.710 DEBUG:teuthology.misc:Transferring archived files from smithi190:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-04-17_21:24:02-fs-squid-distro-default-smithi/7661154/remote/smithi190/crash 2024-04-17T22:32:11.711 DEBUG:teuthology.orchestra.run.smithi190:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-17T22:32:11.753 INFO:tasks.ceph:Compressing logs... 2024-04-17T22:32:11.753 DEBUG:teuthology.orchestra.run.smithi102:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-17T22:32:11.756 DEBUG:teuthology.orchestra.run.smithi190:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-17T22:32:11.821 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47625.log 2024-04-17T22:32:11.821 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54712.log 2024-04-17T22:32:11.821 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.47625.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48108.log 2024-04-17T22:32:11.822 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.54712.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.47625.log.gz 2024-04-17T22:32:11.822 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51730.log 2024-04-17T22:32:11.822 INFO:teuthology.orchestra.run.smithi102.stderr: 86.1% -- replaced with /var/log/ceph/ceph-client.admin.54712.log.gz 2024-04-17T22:32:11.822 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.48108.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45073.log 2024-04-17T22:32:11.822 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.51730.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.48108.log.gz 2024-04-17T22:32:11.822 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49191.log 2024-04-17T22:32:11.822 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.45073.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.51730.log.gz 2024-04-17T22:32:11.822 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45302.log 2024-04-17T22:32:11.822 INFO:teuthology.orchestra.run.smithi102.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.45073.log.gz 2024-04-17T22:32:11.822 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.49191.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51331.log 2024-04-17T22:32:11.822 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.45302.log: gzip -5 --verbose 88.9%-- /var/log/ceph/ceph-client.admin.49803.log -- replaced with /var/log/ceph/ceph-client.admin.49191.log.gz 2024-04-17T22:32:11.822 INFO:teuthology.orchestra.run.smithi102.stderr: 2024-04-17T22:32:11.822 INFO:teuthology.orchestra.run.smithi102.stderr: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.45302.log.gz 2024-04-17T22:32:11.823 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.51331.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46765.log 2024-04-17T22:32:11.823 INFO:teuthology.orchestra.run.smithi102.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.51331.log.gz 2024-04-17T22:32:11.823 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52821.log 2024-04-17T22:32:11.823 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.49803.log: /var/log/ceph/ceph-client.admin.46765.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49599.log 2024-04-17T22:32:11.823 INFO:teuthology.orchestra.run.smithi102.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.49803.log.gz 2024-04-17T22:32:11.823 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.52821.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44093.log 2024-04-17T22:32:11.823 INFO:teuthology.orchestra.run.smithi102.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.46765.log.gz 2024-04-17T22:32:11.823 INFO:teuthology.orchestra.run.smithi102.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.52821.log.gz 2024-04-17T22:32:11.823 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.49599.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49854.log 2024-04-17T22:32:11.823 INFO:teuthology.orchestra.run.smithi102.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.49599.log.gz 2024-04-17T22:32:11.823 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.44093.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51059.log 2024-04-17T22:32:11.823 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.49854.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48516.log 2024-04-17T22:32:11.823 INFO:teuthology.orchestra.run.smithi102.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.44093.log.gz 2024-04-17T22:32:11.823 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.51059.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.49854.log.gz 2024-04-17T22:32:11.824 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46348.log 2024-04-17T22:32:11.824 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.48516.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.51059.log.gz 2024-04-17T22:32:11.824 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54814.log 2024-04-17T22:32:11.824 INFO:teuthology.orchestra.run.smithi102.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.48516.log.gz 2024-04-17T22:32:11.824 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.46348.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49038.log 2024-04-17T22:32:11.824 INFO:teuthology.orchestra.run.smithi102.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.46348.log.gz 2024-04-17T22:32:11.824 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.54814.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51679.log 2024-04-17T22:32:11.824 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.49038.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.54814.log.gz 2024-04-17T22:32:11.824 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53623.log 2024-04-17T22:32:11.824 INFO:teuthology.orchestra.run.smithi102.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.49038.log.gz 2024-04-17T22:32:11.824 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.51679.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46816.log 2024-04-17T22:32:11.824 INFO:teuthology.orchestra.run.smithi102.stderr: 87.6%/var/log/ceph/ceph-client.admin.53623.log: -- replaced with /var/log/ceph/ceph-client.admin.51679.log.gz 2024-04-17T22:32:11.824 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44195.log 2024-04-17T22:32:11.824 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.46816.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.53623.log.gz 2024-04-17T22:32:11.824 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48363.log 2024-04-17T22:32:11.825 INFO:teuthology.orchestra.run.smithi102.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.46816.log.gz 2024-04-17T22:32:11.825 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.44195.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47774.log 2024-04-17T22:32:11.825 INFO:teuthology.orchestra.run.smithi102.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.44195.log.gz 2024-04-17T22:32:11.825 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.48363.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51212.log 2024-04-17T22:32:11.825 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.47774.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.48363.log.gz 2024-04-17T22:32:11.825 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48465.log 2024-04-17T22:32:11.825 INFO:teuthology.orchestra.run.smithi102.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.47774.log.gz 2024-04-17T22:32:11.825 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.51212.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45508.log 2024-04-17T22:32:11.825 INFO:teuthology.orchestra.run.smithi102.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.51212.log.gz 2024-04-17T22:32:11.825 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.48465.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46603.log 2024-04-17T22:32:11.825 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.45508.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.48465.log.gz 2024-04-17T22:32:11.825 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51382.log 2024-04-17T22:32:11.825 INFO:teuthology.orchestra.run.smithi102.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.45508.log.gz 2024-04-17T22:32:11.825 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.46603.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52138.log 2024-04-17T22:32:11.826 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.51382.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53114.log 2024-04-17T22:32:11.826 INFO:teuthology.orchestra.run.smithi102.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.46603.log.gz 2024-04-17T22:32:11.826 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.52138.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.51382.log.gz 2024-04-17T22:32:11.826 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49293.log 2024-04-17T22:32:11.826 INFO:teuthology.orchestra.run.smithi102.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.52138.log.gz 2024-04-17T22:32:11.826 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45710.log 2024-04-17T22:32:11.826 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.53114.log: /var/log/ceph/ceph-client.admin.49293.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48210.log 2024-04-17T22:32:11.826 INFO:teuthology.orchestra.run.smithi102.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.53114.log.gz 2024-04-17T22:32:11.826 INFO:teuthology.orchestra.run.smithi102.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.49293.log.gz 2024-04-17T22:32:11.826 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.45710.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51110.log 2024-04-17T22:32:11.826 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.48210.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.45710.log.gz 2024-04-17T22:32:11.826 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47019.log 2024-04-17T22:32:11.826 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.51110.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.48210.log.gz 2024-04-17T22:32:11.826 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-04-17T22:32:11.826 INFO:teuthology.orchestra.run.smithi102.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.51110.log.gz 2024-04-17T22:32:11.827 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.47019.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44144.log 2024-04-17T22:32:11.827 INFO:teuthology.orchestra.run.smithi102.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.47019.log.gz/var/log/ceph/ceph-osd.2.log: 2024-04-17T22:32:11.827 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50957.log 2024-04-17T22:32:11.827 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.44144.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46714.log 2024-04-17T22:32:11.827 INFO:teuthology.orchestra.run.smithi102.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.44144.log.gz 2024-04-17T22:32:11.827 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.50957.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44402.log 2024-04-17T22:32:11.827 INFO:teuthology.orchestra.run.smithi102.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.50957.log.gz 2024-04-17T22:32:11.827 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.46714.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54965.log 2024-04-17T22:32:11.827 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.44402.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43621.log 2024-04-17T22:32:11.827 INFO:teuthology.orchestra.run.smithi102.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.46714.log.gz 2024-04-17T22:32:11.827 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.54965.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.44402.log.gz 2024-04-17T22:32:11.827 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51008.log 2024-04-17T22:32:11.827 INFO:teuthology.orchestra.run.smithi102.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.54965.log.gz 2024-04-17T22:32:11.827 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.43621.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44767.log 2024-04-17T22:32:11.828 INFO:teuthology.orchestra.run.smithi102.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.43621.log.gz 2024-04-17T22:32:11.828 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51526.log 2024-04-17T22:32:11.828 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.51008.log: /var/log/ceph/ceph-client.admin.44767.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54048.log 2024-04-17T22:32:11.828 INFO:teuthology.orchestra.run.smithi102.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.51008.log.gz 2024-04-17T22:32:11.828 INFO:teuthology.orchestra.run.smithi102.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.44767.log.gz 2024-04-17T22:32:11.828 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.51526.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49497.log 2024-04-17T22:32:11.828 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48414.log 2024-04-17T22:32:11.828 INFO:teuthology.orchestra.run.smithi102.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.51526.log.gz 2024-04-17T22:32:11.828 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.54048.log: /var/log/ceph/ceph-client.admin.49497.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51781.log 2024-04-17T22:32:11.828 INFO:teuthology.orchestra.run.smithi102.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.54048.log.gz 2024-04-17T22:32:11.828 INFO:teuthology.orchestra.run.smithi102.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.49497.log.gz 2024-04-17T22:32:11.828 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44716.log 2024-04-17T22:32:11.828 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.48414.log: /var/log/ceph/ceph-client.admin.51781.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45175.log 2024-04-17T22:32:11.828 INFO:teuthology.orchestra.run.smithi102.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.48414.log.gz 2024-04-17T22:32:11.829 INFO:teuthology.orchestra.run.smithi102.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.51781.log.gz 2024-04-17T22:32:11.829 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.44716.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43888.log 2024-04-17T22:32:11.829 INFO:teuthology.orchestra.run.smithi102.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.44716.log.gz 2024-04-17T22:32:11.829 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.45175.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50550.log 2024-04-17T22:32:11.829 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.43888.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48936.log 2024-04-17T22:32:11.829 INFO:teuthology.orchestra.run.smithi102.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.45175.log.gz 2024-04-17T22:32:11.829 INFO:teuthology.orchestra.run.smithi102.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.43888.log.gz 2024-04-17T22:32:11.829 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.50550.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52617.log 2024-04-17T22:32:11.829 INFO:teuthology.orchestra.run.smithi102.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.50550.log.gz 2024-04-17T22:32:11.829 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.48936.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45991.log 2024-04-17T22:32:11.829 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.52617.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42699.log 2024-04-17T22:32:11.829 INFO:teuthology.orchestra.run.smithi102.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.48936.log.gz 2024-04-17T22:32:11.829 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.45991.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.52617.log.gz 2024-04-17T22:32:11.829 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47070.log 2024-04-17T22:32:11.829 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.42699.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.45991.log.gz 2024-04-17T22:32:11.830 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54610.log 2024-04-17T22:32:11.830 INFO:teuthology.orchestra.run.smithi102.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.42699.log.gz 2024-04-17T22:32:11.830 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-04-17T22:32:11.830 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.47070.log: /var/log/ceph/ceph-client.admin.54610.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54099.log 2024-04-17T22:32:11.830 INFO:teuthology.orchestra.run.smithi102.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.47070.log.gz 2024-04-17T22:32:11.830 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph.audit.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.54610.log.gz 2024-04-17T22:32:11.830 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47445.log 2024-04-17T22:32:11.830 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose /var/log/ceph/ceph-client.admin.54099.log: -- /var/log/ceph/ceph-client.admin.46093.log 2024-04-17T22:32:11.830 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.47445.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47223.log 2024-04-17T22:32:11.830 INFO:teuthology.orchestra.run.smithi102.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.54099.log.gz 2024-04-17T22:32:11.830 INFO:teuthology.orchestra.run.smithi102.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.47445.log.gz 2024-04-17T22:32:11.830 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.46093.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50855.log 2024-04-17T22:32:11.830 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48312.log 2024-04-17T22:32:11.831 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.47223.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.46093.log.gz 2024-04-17T22:32:11.831 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.50855.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.47223.log.gz 2024-04-17T22:32:11.831 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44818.log 2024-04-17T22:32:11.831 INFO:teuthology.orchestra.run.smithi102.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.50855.log.gz 2024-04-17T22:32:11.831 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.48312.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48822.log 2024-04-17T22:32:11.831 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.44818.log: 95.1% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-04-17T22:32:11.831 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44920.log 2024-04-17T22:32:11.831 INFO:teuthology.orchestra.run.smithi102.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.48312.log.gz 2024-04-17T22:32:11.831 INFO:teuthology.orchestra.run.smithi102.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.44818.log.gz 2024-04-17T22:32:11.831 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.48822.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49140.log 2024-04-17T22:32:11.831 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.44920.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50601.log 2024-04-17T22:32:11.831 INFO:teuthology.orchestra.run.smithi102.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.48822.log.gz 2024-04-17T22:32:11.832 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.49140.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.44920.log.gz 2024-04-17T22:32:11.832 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47622.log 2024-04-17T22:32:11.832 INFO:teuthology.orchestra.run.smithi102.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.49140.log.gz 2024-04-17T22:32:11.832 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.50601.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50092.log 2024-04-17T22:32:11.832 INFO:teuthology.orchestra.run.smithi102.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.50601.log.gz 2024-04-17T22:32:11.832 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.47622.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.40170.log 2024-04-17T22:32:11.832 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.50092.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49446.log 2024-04-17T22:32:11.832 INFO:teuthology.orchestra.run.smithi102.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.47622.log.gz 2024-04-17T22:32:11.832 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph.tmp-client.admin.40170.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.50092.log.gz 2024-04-17T22:32:11.832 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49701.log 2024-04-17T22:32:11.832 INFO:teuthology.orchestra.run.smithi102.stderr: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.40170.log.gz 2024-04-17T22:32:11.832 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.49446.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46552.log 2024-04-17T22:32:11.832 INFO:teuthology.orchestra.run.smithi102.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.49446.log.gz 2024-04-17T22:32:11.832 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.49701.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45456.log 2024-04-17T22:32:11.832 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.46552.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.49701.log.gz 2024-04-17T22:32:11.833 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52189.log 2024-04-17T22:32:11.833 INFO:teuthology.orchestra.run.smithi102.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.46552.log.gz 2024-04-17T22:32:11.833 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.45456.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44971.log 2024-04-17T22:32:11.833 INFO:teuthology.orchestra.run.smithi102.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.45456.log.gz 2024-04-17T22:32:11.833 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.52189.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51628.log 2024-04-17T22:32:11.833 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.44971.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.52189.log.gz 2024-04-17T22:32:11.833 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47829.log 2024-04-17T22:32:11.833 INFO:teuthology.orchestra.run.smithi102.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.44971.log.gz 2024-04-17T22:32:11.833 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.51628.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48618.log 2024-04-17T22:32:11.833 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.47829.log: gzip 88.9%-5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.51628.log.gz-- 2024-04-17T22:32:11.833 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-osd.3.log 2024-04-17T22:32:11.833 INFO:teuthology.orchestra.run.smithi102.stderr: 88.3%/var/log/ceph/ceph-client.admin.48618.log: -- replaced with /var/log/ceph/ceph-client.admin.47829.log.gz 2024-04-17T22:32:11.833 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45301.log 2024-04-17T22:32:11.833 INFO:teuthology.orchestra.run.smithi102.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.48618.log.gz 2024-04-17T22:32:11.834 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50397.log 2024-04-17T22:32:11.834 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.45301.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52036.log 2024-04-17T22:32:11.834 INFO:teuthology.orchestra.run.smithi102.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.45301.log.gz 2024-04-17T22:32:11.834 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.50397.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52923.log 2024-04-17T22:32:11.834 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53063.log 2024-04-17T22:32:11.834 INFO:teuthology.orchestra.run.smithi102.stderr: 87.7%/var/log/ceph/ceph-client.admin.52036.log: -- replaced with /var/log/ceph/ceph-client.admin.50397.log.gz 2024-04-17T22:32:11.834 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.52923.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.52036.log.gz 2024-04-17T22:32:11.834 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47496.log 2024-04-17T22:32:11.834 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45712.log 2024-04-17T22:32:11.834 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.53063.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.52923.log.gz 2024-04-17T22:32:11.834 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.47496.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51161.log 2024-04-17T22:32:11.834 INFO:teuthology.orchestra.run.smithi102.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.53063.log.gz 2024-04-17T22:32:11.834 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.45712.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.47496.log.gz 2024-04-17T22:32:11.834 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-04-17T22:32:11.834 INFO:teuthology.orchestra.run.smithi102.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.45712.log.gz 2024-04-17T22:32:11.835 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.51161.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45303.log 2024-04-17T22:32:11.835 INFO:teuthology.orchestra.run.smithi102.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.51161.log.gz 2024-04-17T22:32:11.835 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48159.log 2024-04-17T22:32:11.835 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-osd.0.log: /var/log/ceph/ceph-client.admin.45303.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50346.log 2024-04-17T22:32:11.835 INFO:teuthology.orchestra.run.smithi102.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.45303.log.gz 2024-04-17T22:32:11.835 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.48159.log: /var/log/ceph/ceph-client.admin.50346.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.48159.log.gz 2024-04-17T22:32:11.835 INFO:teuthology.orchestra.run.smithi102.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.50346.log.gz 2024-04-17T22:32:11.835 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54406.log 2024-04-17T22:32:11.835 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47624.log 2024-04-17T22:32:11.835 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.54406.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54304.log 2024-04-17T22:32:11.835 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.47624.log: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.54406.log.gz 2024-04-17T22:32:11.835 INFO:teuthology.orchestra.run.smithi102.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.47624.log.gz 2024-04-17T22:32:11.835 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.54304.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.54304.log.gz 2024-04-17T22:32:11.835 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53267.log 2024-04-17T22:32:11.836 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52770.log 2024-04-17T22:32:11.836 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.53267.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49089.log 2024-04-17T22:32:11.836 INFO:teuthology.orchestra.run.smithi102.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.53267.log.gz 2024-04-17T22:32:11.836 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.52770.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54201.log 2024-04-17T22:32:11.836 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50906.log 2024-04-17T22:32:11.836 INFO:teuthology.orchestra.run.smithi102.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.52770.log.gz 2024-04-17T22:32:11.836 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.49089.log: /var/log/ceph/ceph-client.admin.54201.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51985.log 2024-04-17T22:32:11.836 INFO:teuthology.orchestra.run.smithi102.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.49089.log.gz 2024-04-17T22:32:11.836 INFO:teuthology.orchestra.run.smithi102.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.54201.log.gz 2024-04-17T22:32:11.836 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.50906.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51264.log 2024-04-17T22:32:11.836 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45889.log 2024-04-17T22:32:11.836 INFO:teuthology.orchestra.run.smithi102.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.50906.log.gz 2024-04-17T22:32:11.836 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.51985.log: /var/log/ceph/ceph-client.admin.51264.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47980.log 2024-04-17T22:32:11.836 INFO:teuthology.orchestra.run.smithi102.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.51985.log.gz 2024-04-17T22:32:11.837 INFO:teuthology.orchestra.run.smithi102.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.51264.log.gz 2024-04-17T22:32:11.837 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.45889.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47281.log 2024-04-17T22:32:11.837 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.47980.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.45889.log.gz 2024-04-17T22:32:11.837 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53878.log 2024-04-17T22:32:11.837 INFO:teuthology.orchestra.run.smithi102.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.47980.log.gz 2024-04-17T22:32:11.837 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.47281.log: /var/log/ceph/ceph-client.admin.53878.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.47281.log.gz 2024-04-17T22:32:11.837 INFO:teuthology.orchestra.run.smithi102.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.53878.log.gz 2024-04-17T22:32:11.837 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44665.log 2024-04-17T22:32:11.837 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52293.log 2024-04-17T22:32:11.837 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.44665.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45124.log 2024-04-17T22:32:11.837 INFO:teuthology.orchestra.run.smithi102.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.44665.log.gz 2024-04-17T22:32:11.837 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.52293.log: /var/log/ceph/ceph-client.admin.45124.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.52293.log.gz 2024-04-17T22:32:11.837 INFO:teuthology.orchestra.run.smithi102.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.45124.log.gz 2024-04-17T22:32:11.837 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52087.log 2024-04-17T22:32:11.837 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45711.log 2024-04-17T22:32:11.838 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.52087.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51577.log 2024-04-17T22:32:11.838 INFO:teuthology.orchestra.run.smithi102.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.52087.log.gz 2024-04-17T22:32:11.838 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.45711.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53317.log 2024-04-17T22:32:11.838 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50007.log 2024-04-17T22:32:11.838 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.51577.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.45711.log.gz 2024-04-17T22:32:11.838 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.53317.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43819.log 2024-04-17T22:32:11.838 INFO:teuthology.orchestra.run.smithi102.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.51577.log.gz 2024-04-17T22:32:11.838 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.50007.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.53317.log.gz 2024-04-17T22:32:11.838 INFO:teuthology.orchestra.run.smithi102.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.50007.log.gz 2024-04-17T22:32:11.838 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.43819.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.43819.log.gz 2024-04-17T22:32:11.838 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46450.log 2024-04-17T22:32:11.838 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42798.log 2024-04-17T22:32:11.838 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.46450.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48987.log 2024-04-17T22:32:11.839 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.42798.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43692.log 2024-04-17T22:32:11.839 INFO:teuthology.orchestra.run.smithi102.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.46450.log.gz 2024-04-17T22:32:11.839 INFO:teuthology.orchestra.run.smithi102.stderr: 87.5%/var/log/ceph/ceph-client.admin.48987.log: -- replaced with /var/log/ceph/ceph-client.admin.42798.log.gz 2024-04-17T22:32:11.839 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53572.log 2024-04-17T22:32:11.839 INFO:teuthology.orchestra.run.smithi102.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.48987.log.gz 2024-04-17T22:32:11.839 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.43692.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45940.log 2024-04-17T22:32:11.839 INFO:teuthology.orchestra.run.smithi102.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.43692.log.gz 2024-04-17T22:32:11.839 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.53572.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44563.log 2024-04-17T22:32:11.839 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- 87.3%/var/log/ceph/ceph-client.admin.47172.log 2024-04-17T22:32:11.839 INFO:teuthology.orchestra.run.smithi102.stderr: -- replaced with /var/log/ceph/ceph-client.admin.53572.log.gz 2024-04-17T22:32:11.839 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.45940.log: /var/log/ceph/ceph-client.admin.44563.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.45940.log.gz 2024-04-17T22:32:11.839 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49344.log 2024-04-17T22:32:11.839 INFO:teuthology.orchestra.run.smithi102.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.44563.log.gz 2024-04-17T22:32:11.839 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.47172.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52668.log 2024-04-17T22:32:11.839 INFO:teuthology.orchestra.run.smithi102.stderr: 87.3%gzip -- replaced with /var/log/ceph/ceph-client.admin.47172.log.gz-5 2024-04-17T22:32:11.840 INFO:teuthology.orchestra.run.smithi102.stderr:--verbose -- /var/log/ceph/ceph-client.admin.46969.log 2024-04-17T22:32:11.840 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.49344.log: /var/log/ceph/ceph-client.admin.52668.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.49344.log.gz 2024-04-17T22:32:11.840 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.46969.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.52668.log.gz 2024-04-17T22:32:11.840 INFO:teuthology.orchestra.run.smithi102.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.46969.log.gz 2024-04-17T22:32:11.840 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49905.log 2024-04-17T22:32:11.840 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48057.log 2024-04-17T22:32:11.840 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.49905.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47981.log 2024-04-17T22:32:11.840 INFO:teuthology.orchestra.run.smithi102.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.49905.log.gz 2024-04-17T22:32:11.840 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.48057.log: /var/log/ceph/ceph-client.admin.47981.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.48057.log.gz 2024-04-17T22:32:11.840 INFO:teuthology.orchestra.run.smithi102.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.47981.log.gz 2024-04-17T22:32:11.840 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47777.log 2024-04-17T22:32:11.840 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47121.log 2024-04-17T22:32:11.840 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.47777.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53776.log 2024-04-17T22:32:11.840 INFO:teuthology.orchestra.run.smithi102.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.47777.log.gz 2024-04-17T22:32:11.840 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.47121.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-04-17T22:32:11.841 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51458.log 2024-04-17T22:32:11.841 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.53776.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.47121.log.gz 2024-04-17T22:32:11.844 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.40530.log 2024-04-17T22:32:11.844 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.42802.log 2024-04-17T22:32:11.845 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.42143.log 2024-04-17T22:32:11.845 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.0.40530.log: /var/log/ceph/ceph-client.0.42802.log: /var/log/ceph/ceph-client.0.42143.log: 87.9% -- replaced with /var/log/ceph/ceph-client.0.40530.log.gz 2024-04-17T22:32:11.845 INFO:teuthology.orchestra.run.smithi190.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.0.42143.log.gz 2024-04-17T22:32:11.845 INFO:teuthology.orchestra.run.smithi190.stderr: 91.6% -- replaced with /var/log/ceph/ceph-client.0.42802.log.gz 2024-04-17T22:32:11.845 INFO:teuthology.orchestra.run.smithi190.stderr: 2024-04-17T22:32:11.845 INFO:teuthology.orchestra.run.smithi190.stderr:real 0m0.014s 2024-04-17T22:32:11.845 INFO:teuthology.orchestra.run.smithi190.stderr:user 0m0.011s 2024-04-17T22:32:11.845 INFO:teuthology.orchestra.run.smithi190.stderr:sys 0m0.016s 2024-04-17T22:32:11.848 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47623.log 2024-04-17T22:32:11.848 INFO:teuthology.orchestra.run.smithi102.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.53776.log.gz 2024-04-17T22:32:11.848 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.51458.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54508.log 2024-04-17T22:32:11.848 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.47623.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.51458.log.gz 2024-04-17T22:32:11.848 INFO:teuthology.orchestra.run.smithi102.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.47623.log.gz 2024-04-17T22:32:11.848 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.54508.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.54508.log.gz 2024-04-17T22:32:11.848 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47828.log 2024-04-17T22:32:11.848 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49395.log 2024-04-17T22:32:11.848 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.47828.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53997.log 2024-04-17T22:32:11.849 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.49395.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.47828.log.gz 2024-04-17T22:32:11.849 INFO:teuthology.orchestra.run.smithi102.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.49395.log.gz 2024-04-17T22:32:11.849 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.53997.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.53997.log.gz 2024-04-17T22:32:11.849 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51832.log 2024-04-17T22:32:11.849 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47394.log 2024-04-17T22:32:11.849 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.51832.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53827.log 2024-04-17T22:32:11.849 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.47394.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.51832.log.gz 2024-04-17T22:32:11.849 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.53827.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.47394.log.gz 2024-04-17T22:32:11.849 INFO:teuthology.orchestra.run.smithi102.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.53827.log.gz 2024-04-17T22:32:11.849 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46246.log 2024-04-17T22:32:11.849 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54763.log 2024-04-17T22:32:11.849 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.46246.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45022.log 2024-04-17T22:32:11.850 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.54763.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.46246.log.gz 2024-04-17T22:32:11.850 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.45022.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.54763.log.gz 2024-04-17T22:32:11.850 INFO:teuthology.orchestra.run.smithi102.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.45022.log.gz 2024-04-17T22:32:11.850 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52346.log 2024-04-17T22:32:11.850 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53419.log 2024-04-17T22:32:11.850 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.52346.log: /var/log/ceph/ceph-client.admin.53419.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.52346.log.gz 2024-04-17T22:32:11.850 INFO:teuthology.orchestra.run.smithi102.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.53419.log.gz 2024-04-17T22:32:11.850 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48669.log 2024-04-17T22:32:11.850 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50499.log 2024-04-17T22:32:11.850 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44614.log 2024-04-17T22:32:11.850 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.48669.log: /var/log/ceph/ceph-client.admin.50499.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.48669.log.gz 2024-04-17T22:32:11.850 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.44614.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.50499.log.gz 2024-04-17T22:32:11.851 INFO:teuthology.orchestra.run.smithi102.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.44614.log.gz 2024-04-17T22:32:11.851 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50143.log 2024-04-17T22:32:11.851 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54252.log 2024-04-17T22:32:11.851 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.50143.log: /var/log/ceph/ceph-client.admin.54252.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.50143.log.gz 2024-04-17T22:32:11.851 INFO:teuthology.orchestra.run.smithi102.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.54252.log.gz 2024-04-17T22:32:11.851 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50448.log 2024-04-17T22:32:11.851 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43743.log 2024-04-17T22:32:11.851 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.50448.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45454.log 2024-04-17T22:32:11.851 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.43743.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.50448.log.gz 2024-04-17T22:32:11.851 INFO:teuthology.orchestra.run.smithi102.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.43743.log.gz 2024-04-17T22:32:11.851 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.45454.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.45454.log.gz 2024-04-17T22:32:11.851 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-04-17T22:32:11.851 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49650.log 2024-04-17T22:32:11.851 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44246.log 2024-04-17T22:32:11.852 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.49650.log: /var/log/ceph/ceph-client.admin.44246.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.49650.log.gz 2024-04-17T22:32:11.852 INFO:teuthology.orchestra.run.smithi102.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.44246.log.gz 2024-04-17T22:32:11.852 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2024-04-17T22:32:11.852 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-04-17T22:32:11.852 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-mds.a.log: /var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44349.log 2024-04-17T22:32:11.852 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.44349.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.44349.log.gz 2024-04-17T22:32:11.852 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48261.log 2024-04-17T22:32:11.853 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.48261.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.48261.log.gz 2024-04-17T22:32:11.853 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46918.log 2024-04-17T22:32:11.853 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.46918.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.46918.log.gz 2024-04-17T22:32:11.853 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51883.log 2024-04-17T22:32:11.853 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.51883.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.51883.log.gz 2024-04-17T22:32:11.853 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45507.log 2024-04-17T22:32:11.853 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.45507.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.45507.log.gz 2024-04-17T22:32:11.853 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43990.log 2024-04-17T22:32:11.854 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.43990.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.43990.log.gz 2024-04-17T22:32:11.854 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52719.log 2024-04-17T22:32:11.854 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.52719.log: 92.5% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-04-17T22:32:11.854 INFO:teuthology.orchestra.run.smithi102.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.52719.log.gz 2024-04-17T22:32:11.854 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49242.log 2024-04-17T22:32:11.855 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53521.log 2024-04-17T22:32:11.855 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.49242.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.49242.log.gz 2024-04-17T22:32:11.855 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.53521.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.53521.log.gz 2024-04-17T22:32:11.856 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54865.log 2024-04-17T22:32:11.856 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46195.log 2024-04-17T22:32:11.856 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.54865.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.54865.log.gz 2024-04-17T22:32:11.857 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.46195.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.46195.log.gz 2024-04-17T22:32:11.857 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50702.log 2024-04-17T22:32:11.858 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.50702.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.50702.log.gz 2024-04-17T22:32:11.858 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46663.log 2024-04-17T22:32:11.859 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.46663.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.46663.log.gz 2024-04-17T22:32:11.859 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-04-17T22:32:11.859 INFO:teuthology.orchestra.run.smithi102.stderr: 90.5% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2024-04-17T22:32:11.862 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46297.log 2024-04-17T22:32:11.864 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46042.log 2024-04-17T22:32:11.865 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.46297.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.46297.log.gz 2024-04-17T22:32:11.865 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49548.log 2024-04-17T22:32:11.866 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.46042.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.46042.log.gz 2024-04-17T22:32:11.866 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.49548.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52240.log 2024-04-17T22:32:11.866 INFO:teuthology.orchestra.run.smithi102.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.49548.log.gz 2024-04-17T22:32:11.867 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.52240.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.52240.log.gz 2024-04-17T22:32:11.867 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52566.log 2024-04-17T22:32:11.868 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.52566.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.52566.log.gz 2024-04-17T22:32:11.868 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50245.log 2024-04-17T22:32:11.869 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.50245.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.50245.log.gz 2024-04-17T22:32:11.869 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53725.log 2024-04-17T22:32:11.870 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.53725.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.53725.log.gz 2024-04-17T22:32:11.870 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50296.log 2024-04-17T22:32:11.871 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.50296.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.50296.log.gz 2024-04-17T22:32:11.871 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54559.log 2024-04-17T22:32:11.872 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.54559.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.54559.log.gz 2024-04-17T22:32:11.872 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42952.log 2024-04-17T22:32:11.873 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.42952.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.42952.log.gz 2024-04-17T22:32:11.873 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50804.log 2024-04-17T22:32:11.874 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.50804.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.50804.log.gz 2024-04-17T22:32:11.874 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52872.log 2024-04-17T22:32:11.875 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.52872.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.52872.log.gz 2024-04-17T22:32:11.875 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-04-17T22:32:11.876 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph.log: 94.0% -- replaced with /var/log/ceph/ceph.log.gz 2024-04-17T22:32:11.877 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43939.log 2024-04-17T22:32:11.878 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.43939.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.43939.log.gz 2024-04-17T22:32:11.878 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50753.log 2024-04-17T22:32:11.879 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.50753.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.50753.log.gz 2024-04-17T22:32:11.879 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42749.log 2024-04-17T22:32:11.879 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.42749.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.42749.log.gz 2024-04-17T22:32:11.880 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43054.log 2024-04-17T22:32:11.881 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.43054.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.43054.log.gz 2024-04-17T22:32:11.881 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53470.log 2024-04-17T22:32:11.882 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.53470.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.53470.log.gz 2024-04-17T22:32:11.882 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49956.log 2024-04-17T22:32:11.883 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.49956.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.49956.log.gz 2024-04-17T22:32:11.883 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44461.log 2024-04-17T22:32:11.883 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.44461.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.44461.log.gz 2024-04-17T22:32:11.884 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49752.log 2024-04-17T22:32:11.884 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.49752.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.49752.log.gz 2024-04-17T22:32:11.885 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53165.log 2024-04-17T22:32:11.885 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.53165.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.53165.log.gz 2024-04-17T22:32:11.886 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54150.log 2024-04-17T22:32:11.889 INFO:teuthology.orchestra.run.smithi102.stderr: 94.5% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-04-17T22:32:11.889 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.54150.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48771.log 2024-04-17T22:32:11.889 INFO:teuthology.orchestra.run.smithi102.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.54150.log.gz 2024-04-17T22:32:11.889 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46867.log 2024-04-17T22:32:11.889 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.48771.log: 87.4%/var/log/ceph/ceph-client.admin.46867.log: -- replaced with /var/log/ceph/ceph-client.admin.48771.log.gz 2024-04-17T22:32:11.889 INFO:teuthology.orchestra.run.smithi102.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.46867.log.gz 2024-04-17T22:32:11.889 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51934.log 2024-04-17T22:32:11.889 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46501.log 2024-04-17T22:32:11.889 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.51934.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.51934.log.gz 2024-04-17T22:32:11.890 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.46501.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.46501.log.gz 2024-04-17T22:32:11.890 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53674.log 2024-04-17T22:32:11.890 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48567.log 2024-04-17T22:32:11.896 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.53674.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.53674.log.gz 2024-04-17T22:32:11.896 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.48567.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.48567.log.gz 2024-04-17T22:32:11.896 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44512.log 2024-04-17T22:32:11.897 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46399.log 2024-04-17T22:32:11.897 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.44512.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.44512.log.gz 2024-04-17T22:32:11.897 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.46399.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.46399.log.gz 2024-04-17T22:32:11.897 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45304.log 2024-04-17T22:32:11.897 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47332.log 2024-04-17T22:32:11.897 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.45304.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.45304.log.gz 2024-04-17T22:32:11.898 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53368.log 2024-04-17T22:32:11.898 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.47332.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.47332.log.gz 2024-04-17T22:32:11.898 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45634.log 2024-04-17T22:32:11.898 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.53368.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.53368.log.gz 2024-04-17T22:32:11.898 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50651.log 2024-04-17T22:32:11.898 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.45634.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.45634.log.gz 2024-04-17T22:32:11.898 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52405.log 2024-04-17T22:32:11.899 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.50651.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.50651.log.gz 2024-04-17T22:32:11.899 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44869.log 2024-04-17T22:32:11.899 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.52405.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.52405.log.gz 2024-04-17T22:32:11.899 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42354.log 2024-04-17T22:32:11.899 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.44869.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.44869.log.gz 2024-04-17T22:32:11.899 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44296.log 2024-04-17T22:32:11.902 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.42354.log: 93.2% -- replaced with /var/log/ceph/ceph-client.admin.42354.log.gz 2024-04-17T22:32:11.903 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48720.log 2024-04-17T22:32:11.903 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.44296.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.44296.log.gz 2024-04-17T22:32:11.903 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.48720.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.48720.log.gz 2024-04-17T22:32:11.903 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54661.log 2024-04-17T22:32:11.903 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42648.log 2024-04-17T22:32:11.904 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.54661.log: /var/log/ceph/ceph-client.admin.42648.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.54661.log.gz 2024-04-17T22:32:11.904 INFO:teuthology.orchestra.run.smithi102.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.42648.log.gz 2024-04-17T22:32:11.904 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48884.log 2024-04-17T22:32:11.904 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54355.log 2024-04-17T22:32:11.904 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.48884.log: /var/log/ceph/ceph-client.admin.54355.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.48884.log.gz 2024-04-17T22:32:11.904 INFO:teuthology.orchestra.run.smithi102.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.54355.log.gz 2024-04-17T22:32:11.905 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-04-17T22:32:11.905 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45838.log 2024-04-17T22:32:11.906 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-osd.1.log: /var/log/ceph/ceph-client.admin.45838.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.45838.log.gz 2024-04-17T22:32:11.906 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46144.log 2024-04-17T22:32:11.906 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.46144.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.46144.log.gz 2024-04-17T22:32:11.906 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52515.log 2024-04-17T22:32:11.906 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.52515.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.52515.log.gz 2024-04-17T22:32:11.906 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53937.log 2024-04-17T22:32:11.906 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.53937.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.53937.log.gz 2024-04-17T22:32:11.906 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54914.log 2024-04-17T22:32:11.906 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.54914.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.54914.log.gz 2024-04-17T22:32:11.906 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52985.log 2024-04-17T22:32:11.906 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.52985.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.52985.log.gz 2024-04-17T22:32:11.906 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50194.log 2024-04-17T22:32:11.907 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.50194.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.50194.log.gz 2024-04-17T22:32:11.907 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52456.log 2024-04-17T22:32:11.908 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.52456.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.52456.log.gz 2024-04-17T22:32:11.908 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44042.log 2024-04-17T22:32:11.908 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.44042.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.44042.log.gz 2024-04-17T22:32:11.908 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53216.log 2024-04-17T22:32:11.909 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.53216.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.53216.log.gz 2024-04-17T22:32:11.909 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54457.log 2024-04-17T22:32:11.910 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.54457.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.54457.log.gz 2024-04-17T22:32:11.915 INFO:teuthology.orchestra.run.smithi102.stderr: 91.5% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-04-17T22:32:11.925 INFO:teuthology.orchestra.run.smithi102.stderr: 91.4% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-04-17T22:32:11.940 INFO:teuthology.orchestra.run.smithi102.stderr: 94.5% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-04-17T22:32:11.946 INFO:teuthology.orchestra.run.smithi102.stderr: 94.6% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-04-17T22:32:11.988 INFO:teuthology.orchestra.run.smithi102.stderr: 90.7% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-04-17T22:32:11.991 INFO:teuthology.orchestra.run.smithi102.stderr: 94.5% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-04-17T22:32:11.993 INFO:teuthology.orchestra.run.smithi102.stderr: 2024-04-17T22:32:11.993 INFO:teuthology.orchestra.run.smithi102.stderr:real 0m0.223s 2024-04-17T22:32:11.994 INFO:teuthology.orchestra.run.smithi102.stderr:user 0m0.896s 2024-04-17T22:32:11.994 INFO:teuthology.orchestra.run.smithi102.stderr:sys 0m0.153s 2024-04-17T22:32:11.994 INFO:tasks.ceph:Archiving logs... 2024-04-17T22:32:11.994 DEBUG:teuthology.misc:Transferring archived files from smithi102:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-04-17_21:24:02-fs-squid-distro-default-smithi/7661154/remote/smithi102/log 2024-04-17T22:32:11.995 DEBUG:teuthology.orchestra.run.smithi102:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-17T22:32:12.350 DEBUG:teuthology.misc:Transferring archived files from smithi190:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-04-17_21:24:02-fs-squid-distro-default-smithi/7661154/remote/smithi190/log 2024-04-17T22:32:12.351 DEBUG:teuthology.orchestra.run.smithi190:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-17T22:32:12.386 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-17T22:32:12.412 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-17T22:32:12.412 DEBUG:teuthology.orchestra.run.smithi102:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-17T22:32:12.414 DEBUG:teuthology.orchestra.run.smithi190:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-17T22:32:12.536 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-17T22:32:12.547 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-17T22:32:12.547 DEBUG:teuthology.orchestra.run.smithi102:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-17T22:32:12.550 DEBUG:teuthology.orchestra.run.smithi190:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-17T22:32:12.564 INFO:teuthology.orchestra.run.smithi102.stderr:bash: ntpq: command not found 2024-04-17T22:32:12.568 INFO:teuthology.orchestra.run.smithi102.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-17T22:32:12.568 INFO:teuthology.orchestra.run.smithi102.stdout:=============================================================================== 2024-04-17T22:32:12.568 INFO:teuthology.orchestra.run.smithi102.stdout:^* hv01.front.sepia.ceph.com 3 6 377 12 +796us[ +798us] +/- 39ms 2024-04-17T22:32:12.568 INFO:teuthology.orchestra.run.smithi102.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 10 -925us[ -925us] +/- 53ms 2024-04-17T22:32:12.568 INFO:teuthology.orchestra.run.smithi102.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 10 -157us[ -157us] +/- 81ms 2024-04-17T22:32:12.568 INFO:teuthology.orchestra.run.smithi102.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-04-17T22:32:12.569 INFO:teuthology.orchestra.run.smithi190.stderr:bash: ntpq: command not found 2024-04-17T22:32:12.573 INFO:teuthology.orchestra.run.smithi190.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-17T22:32:12.574 INFO:teuthology.orchestra.run.smithi190.stdout:=============================================================================== 2024-04-17T22:32:12.574 INFO:teuthology.orchestra.run.smithi190.stdout:^* hv01.front.sepia.ceph.com 3 6 377 11 +796us[ +801us] +/- 39ms 2024-04-17T22:32:12.574 INFO:teuthology.orchestra.run.smithi190.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 12 -915us[ -910us] +/- 53ms 2024-04-17T22:32:12.574 INFO:teuthology.orchestra.run.smithi190.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 10 -164us[ -164us] +/- 81ms 2024-04-17T22:32:12.574 INFO:teuthology.orchestra.run.smithi190.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-04-17T22:32:12.575 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-17T22:32:12.586 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-17T22:32:12.587 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-17T22:32:12.617 DEBUG:teuthology.orchestra.run.smithi102:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-17T22:32:12.681 DEBUG:teuthology.orchestra.run.smithi190:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-17T22:32:12.750 DEBUG:teuthology.orchestra.run.smithi102:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-04-17T22:32:12.779 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-17T22:32:12.780 DEBUG:teuthology.orchestra.run.smithi190:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-04-17T22:32:12.810 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-17T22:32:12.811 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-17T22:32:12.822 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-17T22:32:12.859 INFO:teuthology.task.internal:Duration was 695.220427 seconds 2024-04-17T22:32:12.859 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-17T22:32:12.891 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-17T22:32:12.891 DEBUG:teuthology.orchestra.run.smithi102:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-17T22:32:12.893 DEBUG:teuthology.orchestra.run.smithi190:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-17T22:32:12.936 INFO:teuthology.orchestra.run.smithi102.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-17T22:32:12.941 INFO:teuthology.orchestra.run.smithi190.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-17T22:32:13.305 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-17T22:32:13.305 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi102.front.sepia.ceph.com 2024-04-17T22:32:13.306 DEBUG:teuthology.orchestra.run.smithi102:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-17T22:32:13.328 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi190.front.sepia.ceph.com 2024-04-17T22:32:13.328 DEBUG:teuthology.orchestra.run.smithi190:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-17T22:32:13.352 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-17T22:32:13.352 DEBUG:teuthology.orchestra.run.smithi102:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-17T22:32:13.371 DEBUG:teuthology.orchestra.run.smithi190:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-17T22:32:13.434 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-17T22:32:13.434 DEBUG:teuthology.orchestra.run.smithi102:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-17T22:32:13.454 DEBUG:teuthology.orchestra.run.smithi190:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-17T22:32:13.543 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-17T22:32:13.555 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-17T22:32:13.555 DEBUG:teuthology.orchestra.run.smithi102:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-17T22:32:13.582 DEBUG:teuthology.orchestra.run.smithi190:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-17T22:32:13.608 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-17T22:32:13.621 DEBUG:teuthology.orchestra.run.smithi102:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-17T22:32:13.624 DEBUG:teuthology.orchestra.run.smithi190:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-17T22:32:13.647 INFO:teuthology.orchestra.run.smithi102.stdout:kernel.core_pattern = core 2024-04-17T22:32:13.675 INFO:teuthology.orchestra.run.smithi190.stdout:kernel.core_pattern = core 2024-04-17T22:32:13.688 DEBUG:teuthology.orchestra.run.smithi102:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-17T22:32:13.733 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-17T22:32:13.734 DEBUG:teuthology.orchestra.run.smithi190:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-17T22:32:13.750 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-17T22:32:13.751 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-17T22:32:13.764 INFO:teuthology.task.internal:Transferring archived files... 2024-04-17T22:32:13.765 DEBUG:teuthology.misc:Transferring archived files from smithi102:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-04-17_21:24:02-fs-squid-distro-default-smithi/7661154/remote/smithi102 2024-04-17T22:32:13.765 DEBUG:teuthology.orchestra.run.smithi102:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-17T22:32:13.811 DEBUG:teuthology.misc:Transferring archived files from smithi190:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-04-17_21:24:02-fs-squid-distro-default-smithi/7661154/remote/smithi190 2024-04-17T22:32:13.811 DEBUG:teuthology.orchestra.run.smithi190:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-17T22:32:13.849 INFO:teuthology.task.internal:Removing archive directory... 2024-04-17T22:32:13.849 DEBUG:teuthology.orchestra.run.smithi102:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-17T22:32:13.851 DEBUG:teuthology.orchestra.run.smithi190:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-17T22:32:13.900 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-17T22:32:13.914 INFO:teuthology.task.internal:Not uploading archives. 2024-04-17T22:32:13.914 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-17T22:32:13.933 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-17T22:32:13.934 DEBUG:teuthology.orchestra.run.smithi102:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-17T22:32:13.936 DEBUG:teuthology.orchestra.run.smithi190:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-17T22:32:13.951 INFO:teuthology.orchestra.run.smithi102.stdout: 526076 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 17 22:32 /home/ubuntu/cephtest 2024-04-17T22:32:13.958 INFO:teuthology.orchestra.run.smithi190.stdout: 525357 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 17 22:32 /home/ubuntu/cephtest 2024-04-17T22:32:13.959 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-17T22:32:13.972 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-17T22:32:14.037 INFO:teuthology.run:Summary data: description: fs/upgrade/upgraded_client/{bluestore-bitmap branch/nautilus centos_8.stream clusters/{1-mds-1-client-micro} conf/{client mds mgr mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} tasks/{0-install 1-mount/mount/fuse 2-clients/fuse-upgrade 3-workload/new_ops/{newops}}} duration: 695.220427274704 flavor: default owner: scheduled_teuthology@teuthology success: true 2024-04-17T22:32:14.037 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-17T22:32:14.118 INFO:teuthology.run:pass